Knowing how to collect data is one thing. Knowing how to make it relevant is another. And this is all the work of the people working in the data quality management department. Key point expressed by Benjamin, data analyst at Advalo.
“It is essential to have “clean” data. That’s the purpose of data quality management. If you have your own, accurate data, you can set up effective marketing actions..."
Benjamin, Team Leader Data Analyst at Advalo.
What is your definition of data quality management?
Control over data. It means ensuring its quality, consistency and relevance. When we collect data from a customer, I would say that there is first of all a data management task: managing the customer database, harmonizing the data... Then a data quality task: ensuring the relevance of the data we are reporting, checking that there are no inconsistencies.
Can you give an example?
If I see there was a transaction on a Sunday in a physical location, but the store is closed that day. That constitutes an irregularity. There is an error in the uploaded data. So we scrutinize the data every day to ensure that it is error free. Previously, a large part of the work was done by humans, but today machine learning algorithms are being developed to automatically detect these anomalies.
Our differentiation is based on our expertise in creating these algorithms!
Does it take time?
Yes! I usually say that we clean the data, and cleaning is time-consuming. For example, the titles on loyalty cards can be complex to manage. A customer sends us "Mister" with an "M"; another writes out "Mister” in full, another writes "mister", ... We need to harmonize all this, and it goes through the implementation of an ontology (defining a fixed scheme for our databases) and standardization of the data. And that takes a long time. That's 80% of our work! But it is essential. The "cleaner" the data, the more reliable our reports, statistics and forecasts.
What is the purpose of data quality management?
At Advalo, we have decided to do ecological, sustainable marketing. It may surprise you, but that is what we do. Rather than receiving 2 emails a day from a brand, our tools, combined with data, help our customers target the right people, at the right time, with the right support and content. And with the rise of artificial intelligence, all this tends to happen automatically. The essential prerequisite is to have "clean" data. That’s the purpose of data quality management. If you have your own, accurate data, you can set up effective marketing actions.
Can you give us an example?
For example, you can target on Facebook all the people our models detect as "intending" to buy. We must then be confident that the data on which our model is based are correct to make it relevant and durable over time. This is why it is important to have qualitative, clean, cleaned data.
What do you think are the keys to good data quality management?
First of all, the quality of the incoming data has to be good. To this end, we assist companies to develop specifications for the data they wish to receive. We then ensure that the data collected is consistent with their own metrics (average basket, time spent, pages viewed, etc.).
Secondly, I think it is essential to set up rigorous processes with algorithms that run in real time. Because if there is an anomaly, we must be aware of it as soon as possible so as not to include erroneous data.
Finally, being surrounded by statistical experts capable of developing strategies is essential.