Quality control coding: Difference between revisions

From Atomix
← Older edit
VisualWikitext
No edit summary
No edit summary
 
(32 intermediate revisions by 3 users not shown)
Line 1: Line 1:
'''Example: DEFINE A NAME'''
'''[In progress]
copied from code--- to be tidied up:
'''
Provide a Q flag for every dissipation estimate
Q =
      0, if all checks pass
      1, if FOM > FOM_limit
      2, if despike_fraction > despike_fraction_limit
      4, if |log(e_max)-log(e_min)|> 5 sigma


This allows one to identify the uniqe....


'''Shear-probe quality-control flags'''


'''Example: Ocean Sites'''
The Q (quality control) flags associated with shear-probe measurements are bitwise flags with boolean values which are CF compliant. More information available here: https://mplnet.gsfc.nasa.gov/about-flags
 
One option is to follow [http://www.oceansites.org/ Ocean Sites] for quailty control (QC) coding. The flagging scheme is mostly compatible with the primary level flagging recommended by [http://www.ioccp.org/images/D4standards/IOC-OceanDataStandards54-3-2013.pdf Intergovernmental Oceanographic Commission of UNESCO (2013)].


Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:


{| class="wikitable"  
{| class="wikitable"  
|-
|-
! Flag
! Flag Mask
! Meaning
! Bit
! Comment
! Flag Meaning
! Example threshold value
| Ex: True =1  / False =0
| Ex: Q value
|-
|-
| 1
| Bit 0
| Poor figure of merit
| 1.15
| 0
| 0
| 0
| unknown
| No QC was performed.
|-
| 1
| good data
| All QC tests passed.
|-
|-
| 2
| 2
| probably good data
| Bit 1
| Data have failed one or more QC tests but detailed examination after processing (e.g. by visual examination) suggests data is good.
| Large fraction of data with spikes
| 5%
| 0
| 0
|-
|-
| 3
| 4
| potentially correctable bad data
| Bit 2
| These data are not to be used without scientific correction or re-calibration (e.g. uncertain shear sensor sensitivity).
| Anomalously large disagreement between dissipation estimates from probes
| 2.772
| 1
| 4
|-
|-
| 4
| 8
| bad data
| Bit 3
| Data have failed one or more tests.
| Too many iterations of despiking routine
| 8
| 0
| 0
|-
|-
| 5
| 16
| -
| Bit 4
| Not used
| Insufficient variance resolved
| 50%
| 1
| 16
|-
|-
| 6
| 32
| -
| Bit 5
| Not used
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|-
| 7
| 64
| nominal value
| Bit 6
| Data were not observed but reported (e.g. instrument target depth.).
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|-
| 8
| 128
| interpolated value
| Bit 7
| Missing data may be interpolated from neighboring data in space or time.
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|-
| 9
|
| missing value
|
| This is a fill value
|
|
|
| Final Q = 20
|}
|}


<br />
<br />
Climate and Forecast Metadata Convention (CF) requires that QC flags carry attributes. In netCDF (Network Common Data Form) data files, the following information for quality control flagging should be provided for each data variable <PARAM>. <br />
<br />
<PARAM>_QC <br />
<PARAM>_QC:long_name = “quality flag of <PARAM>”; <br />
<PARAM>_QC:conventions = “OceanSITES QC Flags”; <br />
<PARAM>_QC:flag_values = 0, 1, 2, 3, 4, 7, 8, 9;  <br />
<PARAM>_QC:flag_meanings = “0:unknown 1:good_data 2:probably_good_data 3:potentially_correctable_bad_data 4:bad_data 7:nominal_value 8:interpolated_value 9:missing_value” <br/>


The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.
-----------------------------
return to [[Flow chart for shear probes]]


[[Category: Shear probes]]
[[Category: Shear probes]]

Latest revision as of 09:18, 28 August 2023

[In progress]


Shear-probe quality-control flags

The Q (quality control) flags associated with shear-probe measurements are bitwise flags with boolean values which are CF compliant. More information available here: https://mplnet.gsfc.nasa.gov/about-flags

Every dissipation estimate from every probe must have Q flag. The numerical values of the Q flags are as follows:

Flag Mask Bit Flag Meaning Example threshold value Ex: True =1 / False =0 Ex: Q value
1 Bit 0 Poor figure of merit 1.15 0 0
2 Bit 1 Large fraction of data with spikes 5% 0 0
4 Bit 2 Anomalously large disagreement between dissipation estimates from probes 2.772 1 4
8 Bit 3 Too many iterations of despiking routine 8 0 0
16 Bit 4 Insufficient variance resolved 50% 1 16
32 Bit 5 manual flag to be defined by user N/A 0 0
64 Bit 6 manual flag to be defined by user N/A 0 0
128 Bit 7 manual flag to be defined by user N/A 0 0
Final Q = 20


The Q flags are combined by their addition. For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test. A value of 255 means that all tests failed. The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.

return to Flow chart for shear probes

Retrieved from "http://atomix.app.uib.no/index.php?title=Quality_control_coding&oldid=4527"