Quality control coding: Difference between revisions

From Atomix
← Older edit
VisualWikitext
No edit summary
No edit summary
 
(19 intermediate revisions by 2 users not shown)
Line 1: Line 1:
'''In progress
'''[In progress]
'''
'''


Line 5: Line 5:
'''Shear-probe quality-control flags'''
'''Shear-probe quality-control flags'''


The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.
The Q (quality control) flags associated with shear-probe measurements are bitwise flags with boolean values which are CF compliant. More information available here: https://mplnet.gsfc.nasa.gov/about-flags


Every dissipation estimate from every probe must have Q flag.
Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:
The numerical values of the Q flags are as follows:
Q =
      0, if all checks pass
      1, if FOM > FOM_limit
      2, if despike_fraction > despike_fraction_limit
      4, if |log(e_max)-log(e_min)|> diss_ratio_limit X \sigma_{\ln\varepsilon}
      8, if despike_iterations > despike_iterations_limit
      16, manual flag by user
The Q flags are combined by their addition.
For example a Q value of 3 means that the dissipation estimated failed both FOM_limit test and the despike_fraction test.
A value of 15 means that all tests failed.
A failure of any one test (<math>Q\ne0</math>) means that a dissipation test should not be trusted.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.
'''Ocean Sites'''
Providing quality-control flags according to Ocean Sites is encouraged.
These are described at [http://www.oceansites.org/ Ocean Sites] for quailty control (QC) coding.
This flagging scheme is mostly compatible with the primary level flagging recommended by [http://www.ioccp.org/images/D4standards/IOC-OceanDataStandards54-3-2013.pdf Intergovernmental Oceanographic Commission of UNESCO (2013)].
However, only the flags of 0, 1, and 4 make sense for dissipation estimates derived from shear-probe data.


{| class="wikitable"  
{| class="wikitable"  
|-
|-
! Flag
! Flag Mask
! Meaning
! Bit
! Comment
! Flag Meaning
! Example threshold value
| Ex: True =1  / False =0
| Ex: Q value
|-
|-
| 1
| Bit 0
| Poor figure of merit
| 1.15
| 0
| 0
| 0
| unknown
| No QC was performed.
|-
| 1
| good data
| All QC tests passed.
|-
|-
| 2
| 2
| probably good data
| Bit 1
| Data have failed one or more QC tests but detailed examination after processing (e.g. by visual examination) suggests data is good.
| Large fraction of data with spikes
| 5%
| 0
| 0
|-
|-
| 3
| 4
| potentially correctable bad data
| Bit 2
| These data are not to be used without scientific correction or re-calibration (e.g. uncertain shear sensor sensitivity).
| Anomalously large disagreement between dissipation estimates from probes
| 2.772
| 1
| 4
|-
|-
| 4
| 8
| bad data
| Bit 3
| Data have failed one or more tests.
| Too many iterations of despiking routine
| 8
| 0
| 0
|-
|-
| 5
| 16
| -
| Bit 4
| Not used
| Insufficient variance resolved
| 50%
| 1
| 16
|-
|-
| 6
| 32
| -
| Bit 5
| Not used
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|-
| 7
| 64
| nominal value
| Bit 6
| Data were not observed but reported (e.g. instrument target depth.).
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|-
| 8
| 128
| interpolated value
| Bit 7
| Missing data may be interpolated from neighboring data in space or time.
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|-
| 9
|
| missing value
|
| This is a fill value
|
|
|
| Final Q = 20
|}
|}


<br />
<br />
[Rolf left the stuff below in place because I do not know what to do with it.]


The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.


Climate and Forecast Metadata Convention (CF) requires that QC flags carry attributes. In netCDF (Network Common Data Form) data files, the following information for quality control flagging should be provided for each data variable <PARAM>. <br />
-----------------------------
<br />
return to [[Flow chart for shear probes]]
<PARAM>_QC <br />
<PARAM>_QC:long_name = “quality flag of <PARAM>”; <br />
<PARAM>_QC:conventions = “OceanSITES QC Flags”; <br />
<PARAM>_QC:flag_values = 0, 1, 2, 3, 4, 7, 8, 9;  <br />
<PARAM>_QC:flag_meanings = “0:unknown 1:good_data 2:probably_good_data 3:potentially_correctable_bad_data 4:bad_data 7:nominal_value 8:interpolated_value 9:missing_value” <br/>
 


[[Category: Shear probes]]
[[Category: Shear probes]]

Latest revision as of 09:18, 28 August 2023

[In progress]


Shear-probe quality-control flags

The Q (quality control) flags associated with shear-probe measurements are bitwise flags with boolean values which are CF compliant. More information available here: https://mplnet.gsfc.nasa.gov/about-flags

Every dissipation estimate from every probe must have Q flag. The numerical values of the Q flags are as follows:

Flag Mask Bit Flag Meaning Example threshold value Ex: True =1 / False =0 Ex: Q value
1 Bit 0 Poor figure of merit 1.15 0 0
2 Bit 1 Large fraction of data with spikes 5% 0 0
4 Bit 2 Anomalously large disagreement between dissipation estimates from probes 2.772 1 4
8 Bit 3 Too many iterations of despiking routine 8 0 0
16 Bit 4 Insufficient variance resolved 50% 1 16
32 Bit 5 manual flag to be defined by user N/A 0 0
64 Bit 6 manual flag to be defined by user N/A 0 0
128 Bit 7 manual flag to be defined by user N/A 0 0
Final Q = 20


The Q flags are combined by their addition. For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test. A value of 255 means that all tests failed. The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.

return to Flow chart for shear probes

Retrieved from "http://atomix.app.uib.no/index.php?title=Quality_control_coding&oldid=4527"