# Data Quality and Duplicate Records: How to Find, Merge and Prevent Duplicate Customer and Product Records

> What is data quality and how is it measured? Find duplicate customer and product records by tax number, merge them safely and stop repeats in 7 steps.

Source: https://digitalbridge.com.tr/en/blog/data-quality-duplicates.html
Publisher: Digital Bridge Teknoloji (https://digitalbridge.com.tr)
Topic: Data & Analytics
Published: 2026-09-27 · Updated: 2026-10-22

[Data quality](https://digitalbridge.com.tr/en/glossary.html#data-quality) is whether a record is accurate, complete, unique, valid and current enough for its job, and duplicate records are its most common failure: the same customer, supplier or product entered twice. Duplicates are found with strong keys like tax numbers plus name and address matching, merged with the owning team and prevented by entry rules and regular measurement.

## Key takeaways

- Data quality is tracked along five measurable dimensions: uniqueness, completeness, validity, consistency and timeliness.
- Duplicate records are found first with strong keys such as tax number, national ID number or barcode, then by name and address similarity.
- Automatic matching only produces a candidate list; the team that uses the record decides on merges, and duplicates are deactivated, not deleted.
- Thomas C. Redman, writing in MIT Sloan Management Review in 2017, estimates bad data costs most companies 15% to 25% of revenue.
- The lasting fix is prevention at source: mandatory fields, a "similar record exists" warning, an owner for each data area and monthly quality measurement.

## What is a duplicate record and how does it happen?

A duplicate record is the same customer, supplier, product or employee recorded more than once, in one system or across several. In the customer list, "ABC Automotive Industry and Trade Ltd.", "ABC AUTOMOTIVE LTD" and "Abc Automotive Industry" are the same company. One was created by sales, one by accounts and the third automatically by the webshop integration. None of them is "wrong" on its own; the problem is that all three exist at once.

The usual causes:

- **Creating a record without searching.** The user cannot find the existing record, so they create a new one; accents, abbreviations and spacing defeat the search.
- **Several entry points.** CRM, [ERP (enterprise resource planning)](https://digitalbridge.com.tr/en/glossary.html#erp), webshop and [marketplace integration](https://digitalbridge.com.tr/en/blog/api-integration-explained.html) each create the same customer separately.
- **Missing mandatory fields.** If the tax number or national ID number is optional, there is no key to guarantee uniqueness.
- **Migrations and mergers.** Data from the old system is loaded without being matched to records already in the new one.
- **Data with no owner.** When nobody is responsible for customer records being right, everyone waits for someone else to fix them.

The same happens with product records: an item has one code in the warehouse, another in sales and a third name online. In staff data, a name spelt two ways breaks the link between payroll and the [access control and timesheet system](https://digitalbridge.com.tr/en/blog/timesheet-overtime-calculation-turkey.html).

## What does poor data quality cost?

The cost of duplicate and faulty records never appears as one line item; it is split up and spread across every department:

> Data quality expert Thomas C. Redman estimates the cost of bad data at **15% to 25% of revenue** for most companies, and reckons two-thirds of those costs can be identified and eliminated permanently. ([MIT Sloan Management Review, "Seizing Opportunity in Data Quality", 2017](https://sloanreview.mit.edu/article/seizing-opportunity-in-data-quality/))

Redman's figure is an expert estimate, but the day-to-day version will be familiar to any company: invoices raised against the wrong account, a credit limit exceeded because one customer's exposure is split across three records, stock counted twice, the same person receiving a campaign three times, and marketing and finance reporting different numbers of "active customers". On the marketing side, merging one person's records across channels into a single profile is the job of a [customer data platform (CDP)](https://digitalbridge.com.tr/en/blog/customer-data-platform-cdp.html).

The scale of the duplicate problem grows with the amount of data companies keep in systems. According to TurkStat, in 2025 28.3% of Turkish enterprises with 10+ employees used ERP software and 12.0% used CRM ([TurkStat ICT Usage Survey in Enterprises 2025](https://veriportali.tuik.gov.tr/tr/press/54012)). Customer and product data in these companies often lives in more than one system; without matching rules, duplicates are inevitable.

Data integrity is not only damaged by poor entry. TurkStat reports that 5.6% of Turkish enterprises had data lost or corrupted due to malware or unauthorised access in 2025 ([TurkStat ICT Usage Survey in Enterprises 2026](https://veriportali.tuik.gov.tr/tr/press/58021)). Where nobody tracks who changed which record and when, corruption is harder both to spot and to reverse. The backup routine that makes reversal possible is covered in [ransomware and 3-2-1 backups](https://digitalbridge.com.tr/en/blog/ransomware-backup-3-2-1.html).

Another cost of bad data is wasted investment. Moving to a new ERP with uncleaned data carries the old problems into the new system, which is why data migration ranks high in our article on [why ERP projects fail](https://digitalbridge.com.tr/en/blog/why-erp-projects-fail.html).

Reporting is hit directly by bad data too. When sales and finance bring conflicting customer counts or revenue figures to the monthly meeting, the cause is often one company held on three separate records. A BI dashboard built on duplicate data just shows the wrong answer faster and more attractively, so data cleansing is one of the first steps in [moving from Excel reports to BI](https://digitalbridge.com.tr/en/blog/from-excel-reports-to-bi.html).

## How is data quality measured? Five core dimensions

Data quality should be handled not as "good" or "bad" but through measurable dimensions. The table below shows the question each dimension asks and how to measure it on customer records:

| Dimension | Question | Example measure for customer records |
|---|---|---|
| Uniqueness | Is each entity recorded once? | Number of records sharing a tax number |
| Completeness | Are mandatory fields filled? | Share of records missing tax number, email or address |
| Validity | Does the value follow the rules? | Tax numbers with the wrong number of digits, invalid IBAN formats |
| Consistency | Do the systems agree? | Customers whose name or address differs between ERP and CRM |
| Timeliness | Is the record still correct? | Records shown as active with no transactions for two years |

Data quality measurements are not one-offs; they are tracked monthly, and every area has an owner. How to set up those owners, definitions and decision rules is covered in our [data governance framework](https://digitalbridge.com.tr/en/blog/data-governance-framework.html) guide.

## How do you clean up duplicate records? Seven steps

Cleaning up duplicate records is a process that starts with measurement and ends with prevention at source. The seven steps below are written for customer records; the same sequence works for product and staff records:

1. **Choose the scope.** Start with the data area that hurts the business most — usually customer records, followed by product records.
2. **Measure the starting point.** Establish baseline figures for uniqueness, completeness and validity. A clean-up that starts without measurement cannot show progress either.
3. **Define matching rules.** If a strong key (tax number, national ID, barcode) matches, the records are very likely the same. Without one, compare name, address and phone number after normalising case, local characters, suffixes such as "Ltd." and spacing.
4. **Validate candidates with the business.** Automatic matching only produces a candidate list. The decision to merge is made with the sales, accounts or purchasing team that uses the record; accounts with open balances or open orders need particular care.
5. **Pick the master record and merge.** Decide which record survives and how transactions on the others are carried over. Deactivating a duplicate and pointing it to the master, rather than deleting it, preserves the trail of past invoices. Turning that one-off decision into an ongoing process is what [master data management (MDM)](https://digitalbridge.com.tr/en/blog/master-data-management-guide.html) is about.
6. **Fix entry at source.** Define mandatory fields, tax number validation, a "similar record exists" warning and a single role authorised to create records. Integrations should search for an existing record before creating a new one. Critical fields such as a supplier's IBAN should never change without a second approval; fake "our bank details have changed" emails target exactly this gap ([bank detail change email fraud](https://digitalbridge.com.tr/en/blog/bank-detail-change-email-fraud.html)).
7. **Assign owners and monitor.** Give each data area an owner and a steward; when quality measures drop below the agreed level, the steward is alerted.

### Matching pitfalls in Turkish data

Automatic matching on Turkish company names and addresses commonly goes wrong in three places:

- **Character differences.** Turkish "İ/I", "ı/i", "Ş/S" and "Ğ/G" variations turn one company name into two in the system's eyes. All text should be normalised to a single form before comparison.
- **Company-type suffixes.** Similarity scores are misleading unless suffixes such as "Ltd. Şti." (limited company), "A.Ş." (joint-stock company) and "San. ve Tic." (industry and trade) are stripped first.
- **Branches and sole traders.** Two records with the same tax number are not always duplicates; branches with different delivery addresses may be kept apart deliberately. For sole traders the tax identifier is the owner's national ID number, which is personal data, so access to it during matching should be restricted.

### A data dictionary: definitions are a quality issue too

Some "inconsistencies" are really differences of definition. If marketing counts a customer as active when they have ordered in the last 12 months and finance counts one with an open balance, both numbers are right but cannot be compared. A data dictionary with written definitions of "active customer", "net sales" and "stock on hand" ends that argument for good.

The definitions in a data dictionary underpin the KPIs on a management dashboard; we show how a definitions glossary works on screen in our guide to [building a management dashboard](https://digitalbridge.com.tr/en/blog/bi-dashboard-kpis.html). How business terms belong in a business glossary while technical table and field definitions belong in a data dictionary is set out in [data catalog and data dictionary](https://digitalbridge.com.tr/en/blog/data-catalog-data-dictionary.html).

Clean data is also a precondition for AI projects. Gartner predicted that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025 due to poor data quality, inadequate risk controls, escalating costs or unclear business value ([Gartner press release, July 2024](https://www.gartner.com/en/newsroom/press-releases/2024-07-29-gartner-predicts-30-percent-of-generative-ai-projects-will-be-abandoned-after-proof-of-concept-by-end-of-2025)). We explain how to label training examples consistently in [data labelling for AI](https://digitalbridge.com.tr/en/blog/data-labelling-for-ai.html), and how record accuracy affects store orders in [retail inventory optimisation](https://digitalbridge.com.tr/en/blog/retail-inventory-optimisation.html).

Duplicate master data costs most in processes that send data to outside parties. [Automotive EDI despatch advice](https://digitalbridge.com.tr/en/blog/automotive-edi-integration.html) and the textile [digital product passport](https://digitalbridge.com.tr/en/blog/digital-product-passport-textiles.html) are two current examples.

## How does Digital Bridge approach data quality work?

In our [data governance and data quality](https://digitalbridge.com.tr/en/data-analytics/data-governance-quality.html) work, the aim is not to sell you new software but to make the data in your existing systems reliable:

- **Data inventory and flow map.** We map which data is created in which system, where it is copied and which reports it feeds; spreadsheets kept outside the official systems become visible too.
- **Cleansing and merging.** We find spelling variations, missing fields and repeated records, and make merge decisions together with your business teams. Name, address and tax number fields are brought into a single format.
- **Prevention at source.** In your [ERP](https://digitalbridge.com.tr/en/software-solutions/erp-enterprise-resource-planning.html) and [CRM](https://digitalbridge.com.tr/en/software-solutions/crm-customer-relationship-management.html) systems we design record-creation rules, mandatory fields and approval flows with you, and stop integrations creating new records without first searching for existing ones.
- **Ongoing measurement.** Completeness, uniqueness, validity and consistency are measured and reported regularly; when an area falls below the agreed level, its owner is alerted. Where several systems feed a [data warehouse](https://digitalbridge.com.tr/en/data-analytics/data-warehouse.html), faulty records are caught at load time and held back before they reach a report — the architecture is covered in our [data warehouse and ETL guide](https://digitalbridge.com.tr/en/blog/data-warehouse-etl-guide.html).
- **Personal data inventory.** We record which personal data is held in which system, for what purpose and for how long; duplicate records also mean unnecessary copies of personal data. For the full compliance picture under KVKK, Türkiye's Personal Data Protection Law (the local counterpart of GDPR), we work through our [data protection compliance](https://digitalbridge.com.tr/en/consultancy/data-protection-compliance.html) service.

Duplication is not limited to database records; documents suffer from it too, with copies such as "quote_final" and "quote_final2" leaving nobody sure which is current. In SmartFiles, part of the [Smart360](https://digitalbridge.com.tr/en/smart360.html) suite, a file uploaded to the same folder under the same name is never overwritten but stored as a new version, and earlier versions can be downloaded or restored.

A [BI dashboard](https://digitalbridge.com.tr/en/data-analytics/bi-dashboard.html) built on clean data means meetings discuss the decision rather than whether the number is right. We do not sell off-the-shelf packages; after a needs analysis we provide a written proposal covering scope, phases and cost. For neighbouring topics such as data warehousing, KPIs and GIS, see [our full Data & Analytics guide](https://digitalbridge.com.tr/en/blog/topic/veri.html).

## A first data quality check you can do today

Here is a simple check to size the duplicate problem today: export the customer list from your ERP and sort it by tax number. Count how many records share a tax number and how many have none. Those two figures show the size of the problem and where the clean-up should start. [Get in touch](https://digitalbridge.com.tr/en/contact.html) and we will review the results with you and draw up a cleansing and prevention plan for your first data area.

## Frequently Asked Questions

### What is data quality?

Data quality is whether a record is accurate, complete, unique, valid and current enough for the job it is used for. It is not an abstract idea; it is measured along uniqueness, completeness, validity, consistency and timeliness. The most common quality problem is the duplicate record, where the same customer, supplier or product is held more than once; wrong invoices, split credit exposure and conflicting reports usually start there.

### How is data quality measured?

Data quality is measured along dimensions such as uniqueness, completeness, validity, consistency and timeliness. For customer records, for example, the number of records sharing a tax number, the share with no tax number and the number of customers whose name differs between ERP and CRM are reported regularly. Each measure is tracked monthly and each data area has an owner, so improvement or decline shows up as a number.

### How do you find duplicate records?

Duplicate records are first searched for using strong keys such as tax number, national ID number or barcode; if the key matches, the records are very likely the same. Without a key, name, address and phone similarity is used. Before comparing, case, Turkish characters, company-type suffixes such as "Ltd. Şti." and spacing should be normalised; otherwise one company looks like two.

### Is it safe to merge duplicate records automatically?

Automatic matching is fine for producing a candidate list, but the merge decision should be made with the business team. A wrong merge on accounts with open balances, open orders or different delivery addresses can disrupt invoicing and account tracking. The safe route is to give the business team a list of high-scoring candidates and merge only those they approve.

### Should we delete duplicate customer records?

Usually not. To preserve the trail of past invoices and transactions, a duplicate is deactivated and pointed to the master record rather than deleted. Whether transactions are moved across depends on your accounting and audit requirements. New transactions should be blocked on the deactivated record, and anyone searching for it should be pointed to the master; otherwise the duplicate soon reappears.

### Should we clean our data before moving to a new ERP?

Yes. Migrating uncleaned data carries old problems into the new system, where they are harder to fix after go-live. A migration is a good opportunity to measure and deduplicate customer and product records and to set entry rules in the new system. Leaving dormant records behind rather than migrating them is part of the clean-up too.

### Who should be responsible for data quality?

IT provides the tools, but the business team that uses the data is responsible for its accuracy. Naming an owner for each data area (for example, the finance manager for customer records) and a steward who makes day-to-day corrections is what makes quality last. The owner approves record-creation rules and receives the alert when quality measures drop.

### What is the difference between data quality and data governance?

Data quality is the measure of how accurate, complete and unique data is for the job it is used for. Data governance is the framework that makes that quality last: it sets owners for each data area, shared definitions of terms, record-creation rules and the decision process. A clean-up done without governance soon has to be repeated, because the entry points that create duplicates stay open.