Consistent, accurate, and high-quality data is one of the biggest promises we make here at QES, but what does it really mean to have high-quality data, and how do we go about collecting it?
In this four-part blog series on data quality, we’ll be discussing exactly that.
Consistency Is Key
First, what is quality data, and why is it important? It seems obvious that good data should be accurate, but confirming that accuracy is not as straightforward. “The key to good data is reproducibility,” Says QES Pavement Engineer Ryan Finley, “If a team can go out and collect new sets of numbers but come up with the same results, that’s verified.” In practice, we use statistical analysis to confirm the quality of our data. We deliver data at a 95% confidence level, meaning 95% of all of our data is reproducible.
There are three keys to obtaining accurate, reproducible data that meets this high confidence standard:
- Consistency between different data collectors/measuring tools
- Consistency between a single collector’s/tool’s measurements over time.
- Agreement with historical records.
Creating Actionable Datasets
At QES, we take pride in our ability to collect, provide, and verify high-quality data, but it is more than a matter of personal pride. According to QES Vice President of Pavement Engineering Doug Frith, “The quality of the collected data determines the quality of any subsequent work related to that data.” A pavement management system requires good inputs in order to produce accurate results: the wrong maintenance treatments can result in either overspending on maintenance where it is unnecessary, or underspending in areas that are in need, requiring more maintenance and higher costs down the road.
Here are the steps QES takes to ensure the highest quality data is obtained for the client to allow for the most successful, long-term projects.
Deep Standardized Training
Before any technician begins collecting data for a client, they go through our rigorous training process. This involves standardized training and distress identification, and the completion of several local test sites. Our technicians are matched against our company standards in order to ensure their ratings remain consistent, subsequently technicians are also compared against each other to limit the variation between them.
This complete training of our data collectors before they enter the field and a robust system of QA checks during and after field collection helps ensure the data is collected with consistency and variability.
Field Checks
Once in the field, we use a system of QA checks to ensure all ratings remain up to our quality standards over the course of the project. To ensure technicians have a consistent level of quality from start to finish on a project, they will occasionally resample sections they have already rated. If the technician’s new ratings don’t match the old ones, then the data needs to be collected again. Additionally, technicians resample a section of each other’s ratings, to provide another level of data quality. “The idea is to check often and catch any errors early on,” says Finley, “This means if errors do occur, they do not affect large portions of the data.”
Final Check
Upon completion of data collection, a final check is performed against historical data sets. We usually expect new data to match the historical trends. “Having historical data to compare to allows us to use projection models, and makes the data more valuable for the long term,” says Finley. If new data doesn’t match the historical trends, it provides another opportunity for us to double check and verify the data. “Historical outlier data may be perfectly legitimate, if there was unrecorded maintenance for another issue.” Verifying and then inputting new data allows us to fill in missing details on Pavement Management Systems and improve the datasets for the future.”
Stay tuned for the rest of our data quality series, in which we examine the benefits of QES’s data practices:
- How Samples and Statistics Make Raw Data Better
- Who Watches the Watchmen: Verifying Data Quality in Construction Inspection
- Making Better Data for the Future