Talk:Quality control coding

During our shear probe group meeting on March 28, 2022, we discussed the possibility of using a different flagging scheme for the shear probe data. Suggestions were to use a binary system that allows to relate the flagging number to one or more specific data quality control tests that would be unique even if multible quality control tests were failed by the data set. However, this flagging scheme would only apply to our data and international data centres or data display display servers (e.g. https://www.ocean-ops.org/board) would not recognize this. Perhaps a solution would be to use a two-level scheme, as also suggeted by the Intergovernmental Oceanographic Commission (IOC). On the primary level, we use a common quality flagging as shown below. The secondary level could then be a unique number that points to the quality test the data has failed. The flagging scheme below is taken from a best practice IOC document (link). The secondary level could then complement the primary level flags by reporting the results of specific QC tests performed and data processing history. As suggested by IOC best practices "The secondary level content varies in number and description and is chosen by those who implement the scheme, representing information on the applied quality tests (e.g., excessive spike check, regional data range check) and data processing history (e.g., interpolated values, corrected values)."

IOC Quality flagging primary level

Value	Primary-level flag short name	Definition
1	Good	Passed documented required QC tests
2	Not evaluated, not available or unknown	Used for data when no QC test performed or the information on quality is not available
3	Questionable/suspect	Failed non-critical documented metric or subjective test(s)
4	Bad	Failed critical documented QC test(s) or as assigned by the data provider
9	Missing data	Used as place holder when data are missing

Would that work?

---

Below are some old notes from the main page. They might still come in handy at a later stage: [Amelie]

Ocean Sites Providing quality-control flags according to Ocean Sites is encouraged. These are described at Ocean Sites for quailty control (QC) coding. This flagging scheme is mostly compatible with the primary level flagging recommended by Intergovernmental Oceanographic Commission of UNESCO (2013). However, only the flags of 0, 1, and 4 make sense for dissipation estimates derived from shear-probe data.

Flag	Meaning	Comment
0	unknown	No QC was performed.
1	good data	All QC tests passed.
2	probably good data	Data have failed one or more QC tests but detailed examination after processing (e.g. by visual examination) suggests data is good.
3	potentially correctable bad data	These data are not to be used without scientific correction or re-calibration (e.g. uncertain shear sensor sensitivity).
4	bad data	Data have failed one or more tests.
5	-	Not used
6	-	Not used
7	nominal value	Data were not observed but reported (e.g. instrument target depth.).
8	interpolated value	Missing data may be interpolated from neighboring data in space or time.
9	missing value	This is a fill value

[Rolf left the stuff below in place because I do not know what to do with it.]

Climate and Forecast Metadata Convention (CF) requires that QC flags carry attributes. In netCDF (Network Common Data Form) data files, the following information for quality control flagging should be provided for each data variable <PARAM>.

<PARAM>_QC
<PARAM>_QC:long_name = “quality flag of <PARAM>”;
<PARAM>_QC:conventions = “OceanSITES QC Flags”;
<PARAM>_QC:flag_values = 0, 1, 2, 3, 4, 7, 8, 9;
<PARAM>_QC:flag_meanings = “0:unknown 1:good_data 2:probably_good_data 3:potentially_correctable_bad_data 4:bad_data 7:nominal_value 8:interpolated_value 9:missing_value”