Quality control coding: Difference between revisions

From Atomix
No edit summary
Rolf (talk | contribs)
No edit summary
Line 1: Line 1:
'''Example: DEFINE A NAME'''
'''Example: DEFINE A NAME'''


copied from code--- to be tidied up:
The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.
Provide a Q flag for every dissipation estimate
 
Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:
 
Q =  
Q =  
       0, if all checks pass
       0, if all checks pass
       1, if FOM > FOM_limit
       1, if FOM > FOM_limit
       2, if despike_fraction > despike_fraction_limit
       2, if despike_fraction > despike_fraction_limit
       4, if |log(e_max)-log(e_min)|> 5 sigma
       4, if |log(e_max)-log(e_min)|> diss_ratio_limit <math>\times \sigma_{\ln\varepsilon}</math>
 
      8, if despike_iterations > despike_iterations_limit


Combinations (sums) are:
The Q flags are combined by their addition.
      3, FOM and despike fraction fail
For example a Q value of 3 mens that the dissipation estimated failed both FOM_limit test and the despike_fraction test.
      5, FOM and sigma fail
A value of 15 means that all tests failed.
      7, all fail
A failure of any one test (<math>Q\ne0</math>) means that a dissipation test should not be trusted.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.


This allows one to identify the unique ....
This allows one to identify the unique ....
Line 30: Line 34:
       999, all bad
       999, all bad


However, this is already provided by the above recommended Q flags.


'''Example: Ocean Sites'''
'''Example: Ocean Sites'''
 
Providing quality-control flags according to Ocean Sites is encouraged.
One option is to follow [http://www.oceansites.org/ Ocean Sites] for quailty control (QC) coding. The flagging scheme is mostly compatible with the primary level flagging recommended by [http://www.ioccp.org/images/D4standards/IOC-OceanDataStandards54-3-2013.pdf Intergovernmental Oceanographic Commission of UNESCO (2013)].  
These are described at [http://www.oceansites.org/ Ocean Sites] for quailty control (QC) coding.  
This flagging scheme is mostly compatible with the primary level flagging recommended by [http://www.ioccp.org/images/D4standards/IOC-OceanDataStandards54-3-2013.pdf Intergovernmental Oceanographic Commission of UNESCO (2013)].
However, only the flags of 0, 1, and 4 make sense for dissipation estimates derived from shear-probe data.





Revision as of 00:12, 24 May 2022

Example: DEFINE A NAME

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites Ocean Sites for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag. The numerical values of the Q flags are as follows:

Q =

      0, if all checks pass
      1, if FOM > FOM_limit
      2, if despike_fraction > despike_fraction_limit
      4, if |log(e_max)-log(e_min)|> diss_ratio_limit [math]\displaystyle{ \times \sigma_{\ln\varepsilon} }[/math]
      8, if despike_iterations > despike_iterations_limit

The Q flags are combined by their addition. For example a Q value of 3 mens that the dissipation estimated failed both FOM_limit test and the despike_fraction test. A value of 15 means that all tests failed. A failure of any one test ([math]\displaystyle{ Q\ne0 }[/math]) means that a dissipation test should not be trusted. The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.

This allows one to identify the unique ....

Another flag that can be used (in addition to the Quality flag) is GOOD PROBE: GOOD_PROBE

      0, all probes are good
      1, sh1 only
      2, sh2 only
      3, sh1 and sh2
      4, sh3 only
      5, sh1 and sh3
      6, sh2 and sh3
      7, sh1, sh2 and sh3
      8, sh4 only
      999, all bad

However, this is already provided by the above recommended Q flags.

Example: Ocean Sites Providing quality-control flags according to Ocean Sites is encouraged. These are described at Ocean Sites for quailty control (QC) coding. This flagging scheme is mostly compatible with the primary level flagging recommended by Intergovernmental Oceanographic Commission of UNESCO (2013). However, only the flags of 0, 1, and 4 make sense for dissipation estimates derived from shear-probe data.


Flag Meaning Comment
0 unknown No QC was performed.
1 good data All QC tests passed.
2 probably good data Data have failed one or more QC tests but detailed examination after processing (e.g. by visual examination) suggests data is good.
3 potentially correctable bad data These data are not to be used without scientific correction or re-calibration (e.g. uncertain shear sensor sensitivity).
4 bad data Data have failed one or more tests.
5 - Not used
6 - Not used
7 nominal value Data were not observed but reported (e.g. instrument target depth.).
8 interpolated value Missing data may be interpolated from neighboring data in space or time.
9 missing value This is a fill value


Climate and Forecast Metadata Convention (CF) requires that QC flags carry attributes. In netCDF (Network Common Data Form) data files, the following information for quality control flagging should be provided for each data variable <PARAM>.

<PARAM>_QC
<PARAM>_QC:long_name = “quality flag of <PARAM>”;
<PARAM>_QC:conventions = “OceanSITES QC Flags”;
<PARAM>_QC:flag_values = 0, 1, 2, 3, 4, 7, 8, 9;
<PARAM>_QC:flag_meanings = “0:unknown 1:good_data 2:probably_good_data 3:potentially_correctable_bad_data 4:bad_data 7:nominal_value 8:interpolated_value 9:missing_value”