Quality control coding: Difference between revisions

From Atomix
← Older edit
VisualWikitext
Rolf (talk | contribs)
415 edits
No edit summary
No edit summary
 
(25 intermediate revisions by 3 users not shown)
Line 1: Line 1:
'''[In progress]
'''
'''Shear-probe quality-control flags'''
'''Shear-probe quality-control flags'''


The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.
The Q (quality control) flags associated with shear-probe measurements are bitwise flags with boolean values which are CF compliant. More information available here: https://mplnet.gsfc.nasa.gov/about-flags


Every dissipation estimate from every probe must have Q flag.
Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:
The numerical values of the Q flags are as follows:
Q =
      0, if all checks pass
      1, if FOM > FOM_limit
      2, if despike_fraction > despike_fraction_limit
      4, if |log(e_max)-log(e_min)|> diss_ratio_limit <math>\times \sigma_{\ln\varepsilon}</math>
      8, if despike_iterations > despike_iterations_limit
The Q flags are combined by their addition.
For example a Q value of 3 means that the dissipation estimated failed both FOM_limit test and the despike_fraction test.
A value of 15 means that all tests failed.
A failure of any one test (<math>Q\ne0</math>) means that a dissipation test should not be trusted.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.
Another flag that can be used (in addition to the Quality flag) is GOOD PROBE:
GOOD_PROBE
      0, all probes are good
      1, sh1 only
      2, sh2 only
      3, sh1 and sh2
      4, sh3 only
      5, sh1 and sh3
      6, sh2 and sh3
      7, sh1, sh2 and sh3
      8, sh4 only
      999, all bad
However, this is already provided by the above recommended Q flags.
'''Ocean Sites'''
Providing quality-control flags according to Ocean Sites is encouraged.
These are described at [http://www.oceansites.org/ Ocean Sites] for quailty control (QC) coding.
This flagging scheme is mostly compatible with the primary level flagging recommended by [http://www.ioccp.org/images/D4standards/IOC-OceanDataStandards54-3-2013.pdf Intergovernmental Oceanographic Commission of UNESCO (2013)].
However, only the flags of 0, 1, and 4 make sense for dissipation estimates derived from shear-probe data.


{| class="wikitable"  
{| class="wikitable"  
|-
|-
! Flag
! Flag Mask
! Meaning
! Bit
! Comment
! Flag Meaning
! Example threshold value
| Ex: True =1  / False =0
| Ex: Q value
|-
|-
| 1
| Bit 0
| Poor figure of merit
| 1.15
| 0
| 0
| 0
| unknown
| No QC was performed.
|-
| 1
| good data
| All QC tests passed.
|-
|-
| 2
| 2
| probably good data
| Bit 1
| Data have failed one or more QC tests but detailed examination after processing (e.g. by visual examination) suggests data is good.
| Large fraction of data with spikes
|-
| 5%
| 3
| 0
| potentially correctable bad data
| 0
| These data are not to be used without scientific correction or re-calibration (e.g. uncertain shear sensor sensitivity).
|-
|-
| 4
| 4
| bad data
| Bit 2
| Data have failed one or more tests.
| Anomalously large disagreement between dissipation estimates from probes
| 2.772
| 1
| 4
|-
|-
| 5
| 8
| -
| Bit 3
| Not used
| Too many iterations of despiking routine
| 8
| 0
| 0
|-
|-
| 6
| 16
| -
| Bit 4
| Not used
| Insufficient variance resolved
| 50%
| 1
| 16
|-
|-
| 7
| 32
| nominal value
| Bit 5
| Data were not observed but reported (e.g. instrument target depth.).
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|-
| 8
| 64
| interpolated value
| Bit 6
| Missing data may be interpolated from neighboring data in space or time.
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 128
| Bit 7
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|-
| 9
|
| missing value
|
| This is a fill value
|
|
|
| Final Q = 20
|}
|}


<br />
<br />
[Rolf left this here because I do not know what to do with it.]


The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.


Climate and Forecast Metadata Convention (CF) requires that QC flags carry attributes. In netCDF (Network Common Data Form) data files, the following information for quality control flagging should be provided for each data variable <PARAM>. <br />
-----------------------------
<br />
return to [[Flow chart for shear probes]]
<PARAM>_QC <br />
<PARAM>_QC:long_name = “quality flag of <PARAM>”; <br />
<PARAM>_QC:conventions = “OceanSITES QC Flags”; <br />
<PARAM>_QC:flag_values = 0, 1, 2, 3, 4, 7, 8, 9;  <br />
<PARAM>_QC:flag_meanings = “0:unknown 1:good_data 2:probably_good_data 3:potentially_correctable_bad_data 4:bad_data 7:nominal_value 8:interpolated_value 9:missing_value” <br/>
 


[[Category: Shear probes]]
[[Category: Shear probes]]

Latest revision as of 09:18, 28 August 2023

[In progress]


Shear-probe quality-control flags

The Q (quality control) flags associated with shear-probe measurements are bitwise flags with boolean values which are CF compliant. More information available here: https://mplnet.gsfc.nasa.gov/about-flags

Every dissipation estimate from every probe must have Q flag. The numerical values of the Q flags are as follows:

Flag Mask Bit Flag Meaning Example threshold value Ex: True =1 / False =0 Ex: Q value
1 Bit 0 Poor figure of merit 1.15 0 0
2 Bit 1 Large fraction of data with spikes 5% 0 0
4 Bit 2 Anomalously large disagreement between dissipation estimates from probes 2.772 1 4
8 Bit 3 Too many iterations of despiking routine 8 0 0
16 Bit 4 Insufficient variance resolved 50% 1 16
32 Bit 5 manual flag to be defined by user N/A 0 0
64 Bit 6 manual flag to be defined by user N/A 0 0
128 Bit 7 manual flag to be defined by user N/A 0 0
Final Q = 20


The Q flags are combined by their addition. For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test. A value of 255 means that all tests failed. The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.

return to Flow chart for shear probes

Retrieved from "http://atomix.app.uib.no/index.php?title=Quality_control_coding&oldid=4527"