Data quality is whether a record is accurate, complete, unique, valid and current enough for its job, and duplicate records are its most common failure: the same customer, supplier or product entered twice. Duplicates are found with strong keys like tax numbers plus name and address matching, merged with the owning team and prevented by entry rules and regular measurement.
How duplicate records happen
In the customer list, "ABC Automotive Industry and Trade Ltd.", "ABC AUTOMOTIVE LTD" and "Abc Automotive Industry" are the same company. One was created by sales, one by accounts and the third automatically by the webshop integration. None of them is "wrong" on its own; the problem is that all three exist at once.
The usual causes:
- Creating a record without searching. The user cannot find the existing record, so they create a new one; accents, abbreviations and spacing defeat the search.
- Several entry points. CRM, ERP, webshop and marketplace integration each create the same customer separately.
- Missing mandatory fields. If the tax number or national ID number is optional, there is no key to guarantee uniqueness.
- Migrations and mergers. Data from the old system is loaded without being matched to records already in the new one.
- Data with no owner. When nobody is responsible for customer records being right, everyone waits for someone else to fix them.
The same happens with product records: an item has one code in the warehouse, another in sales and a third name online. In staff data, a name spelt two ways breaks the link between payroll and the access control and timesheet system.
The cost of leaving it
The cost of duplicate and faulty records never appears as one line item; it is split up and spread across every department:
Data quality expert Thomas C. Redman estimates the cost of bad data at 15% to 25% of revenue for most companies, and reckons two-thirds of those costs can be identified and eliminated permanently. (MIT Sloan Management Review, "Seizing Opportunity in Data Quality", 2017)
It is an expert estimate, but the day-to-day version is familiar: invoices raised against the wrong account, a credit limit exceeded because one customer's exposure is split across three records, stock counted twice, the same person receiving a campaign three times, and marketing and finance reporting different numbers of "active customers". On the marketing side, merging one person's records across channels into a single profile is the job of a customer data platform (CDP).
The problem grows with the amount of data companies keep in systems. According to TurkStat, in 2025 28.3% of Turkish enterprises with 10+ employees used ERP software and 12.0% used CRM (TurkStat ICT Usage Survey in Enterprises 2025). Customer and product data in these companies often lives in more than one system; without matching rules, duplicates are inevitable.
Data integrity is not only damaged by poor entry. TurkStat reports that 5.6% of Turkish enterprises had data lost or corrupted due to malware or unauthorised access in 2025 (TurkStat ICT Usage Survey in Enterprises 2026). Where nobody tracks who changed which record and when, corruption is harder both to spot and to reverse. The backup routine that makes reversal possible is covered in ransomware and 3-2-1 backups.
Then there is wasted investment. Moving to a new ERP with uncleaned data carries the old problems into the new system, which is why data migration ranks high in our article on why ERP projects fail.
Reporting suffers in the same way. When sales and finance bring conflicting customer counts or revenue figures to the monthly meeting, the cause is often one company held on three separate records. A BI dashboard built on duplicate data just shows the wrong answer faster and more attractively, so data cleansing is one of the first steps in moving from Excel reports to BI.
Five dimensions of data quality
Quality should be handled not as "good" or "bad" but through measurable dimensions:
| Dimension | Question | Example measure for customer records |
|---|---|---|
| Uniqueness | Is each entity recorded once? | Number of records sharing a tax number |
| Completeness | Are mandatory fields filled? | Share of records missing tax number, email or address |
| Validity | Does the value follow the rules? | Tax numbers with the wrong number of digits, invalid IBAN formats |
| Consistency | Do the systems agree? | Customers whose name or address differs between ERP and CRM |
| Timeliness | Is the record still correct? | Records shown as active with no transactions for two years |
These are not one-off measurements; they are tracked monthly, and every area has an owner. How to set up those owners, definitions and decision rules is covered in our data governance framework guide.
Seven steps to clean up duplicate records
- Choose the scope. Start with the data area that hurts the business most — usually customer records, followed by product records.
- Measure the starting point. Establish baseline figures for uniqueness, completeness and validity. A clean-up that starts without measurement cannot show progress either.
- Define matching rules. If a strong key (tax number, national ID, barcode) matches, the records are very likely the same. Without one, compare name, address and phone number after normalising case, local characters, suffixes such as "Ltd." and spacing.
- Validate candidates with the business. Automatic matching only produces a candidate list. The decision to merge is made with the sales, accounts or purchasing team that uses the record; accounts with open balances or open orders need particular care.
- Pick the master record and merge. Decide which record survives and how transactions on the others are carried over. Deactivating a duplicate and pointing it to the master, rather than deleting it, preserves the trail of past invoices. Turning that one-off decision into an ongoing process is what master data management (MDM) is about.
- Fix entry at source. Define mandatory fields, tax number validation, a "similar record exists" warning and a single role authorised to create records. Integrations should search for an existing record before creating a new one. Critical fields such as a supplier's IBAN should never change without a second approval; fake "our bank details have changed" emails target exactly this gap (bank detail change email fraud).
- Assign owners and monitor. Give each data area an owner and a steward; when quality measures drop below the agreed level, the steward is alerted.
Matching pitfalls in Turkish data
- Character differences. Turkish "İ/I", "ı/i", "Ş/S" and "Ğ/G" variations turn one company name into two in the system's eyes. All text should be normalised to a single form before comparison.
- Company-type suffixes. Similarity scores are misleading unless suffixes such as "Ltd. Şti." (limited company), "A.Ş." (joint-stock company) and "San. ve Tic." (industry and trade) are stripped first.
- Branches and sole traders. Two records with the same tax number are not always duplicates; branches with different delivery addresses may be kept apart deliberately. For sole traders the tax identifier is the owner's national ID number, which is personal data, so access to it during matching should be restricted.
A data dictionary: definitions are a quality issue too
Some "inconsistencies" are really differences of definition. If marketing counts a customer as active when they have ordered in the last 12 months and finance counts one with an open balance, both numbers are right but cannot be compared. A data dictionary with written definitions of "active customer", "net sales" and "stock on hand" ends that argument for good. The same definitions underpin the KPIs on a management dashboard; we show how a definitions glossary works on screen in our guide to building a management dashboard. How business terms belong in a business glossary while technical table and field definitions belong in a data dictionary is set out in data catalog and data dictionary.
Clean data is also a precondition for AI projects. We explain how to label training examples consistently in data labelling for AI, and how record accuracy affects store orders in retail inventory optimisation.
Duplicate master data costs most in processes that send data to outside parties. Automotive EDI despatch advice and the textile digital product passport are two current examples.
How we approach it at Digital Bridge
In our data governance and data quality work, the aim is not to sell you new software but to make the data in your existing systems reliable:
- Data inventory and flow map. We map which data is created in which system, where it is copied and which reports it feeds; spreadsheets kept outside the official systems become visible too.
- Cleansing and merging. We find spelling variations, missing fields and repeated records, and make merge decisions together with your business teams. Name, address and tax number fields are brought into a single format.
- Prevention at source. In your ERP and CRM systems we design record-creation rules, mandatory fields and approval flows with you, and stop integrations creating new records without first searching for existing ones.
- Ongoing measurement. Completeness, uniqueness, validity and consistency are measured and reported regularly; when an area falls below the agreed level, its owner is alerted. Where several systems feed a data warehouse, faulty records are caught at load time and held back before they reach a report — the architecture is covered in our data warehouse and ETL guide.
- Personal data inventory. We record which personal data is held in which system, for what purpose and for how long; duplicate records also mean unnecessary copies of personal data. For the full compliance picture under KVKK, Türkiye's Personal Data Protection Law (the local counterpart of GDPR), we work through our data protection compliance service.
Duplication is not limited to database records; documents suffer from it too, with copies such as "quote_final" and "quote_final2" leaving nobody sure which is current. In SmartFiles, part of the Smart360 suite, a file uploaded under the same name is never overwritten but stored as a new version, and earlier versions can be downloaded or restored.
A BI dashboard built on clean data means meetings discuss the decision rather than whether the number is right. We do not sell off-the-shelf packages; after a needs analysis we provide a written proposal covering scope, phases and cost. For neighbouring topics such as data warehousing, KPIs and GIS, see our full Data & Analytics guide.
Next step
Here is a check you can do today: export the customer list from your ERP and sort it by tax number. Count how many records share a tax number and how many have none. Those two figures show the size of the problem and where the clean-up should start. Get in touch and we will review the results with you and draw up a cleansing and prevention plan for your first data area.