Quality control of ε estimates (QA2): Difference between revisions

From Atomix
mNo edit summary
Yuengdjern (talk | contribs)
No edit summary
Line 1: Line 1:
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings.
Quality control measures for each beam:
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Ensure sufficient <math> D_{ll} </math> samples were used in the regression.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. If noise is too high, <math> \epsilon </math> are rejected.
# Data segments for which the regression coefficient a<sub>0</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
# Data segments for which the regression coefficient a<sub>0</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
# Examine the consistency of <math>\varepsilon</math> between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
# Examine the consistency of <math>\varepsilon</math> between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
Line 10: Line 12:


------  
------  
'''[In progress]
'''
'''How ADCP structure function quality-control flags are applied'''
The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.
Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:
{| class="wikitable"
|-
! Flag Mask
! Bit
! Flag Meaning
! Example threshold value
| Ex: True =1  / False =0
| Ex: Q value
|-
| 1
| Bit 0
| if FOM > FOM_limit
| 2
| 0
| 0
|-
| 2
| Bit 1
| if despike_fraction > despike_fraction_limit
| 40%
| 0
| 0
|-
| 4
| Bit 2
| if |log(e_max)-log(e_min)|> diss_ratio_limit X \sigma_{\ln\varepsilon}
| N/A
| 1
| 4
|-
| 8
| Bit 3
| if despike_iterations > despike_iterations_limit
| To be confirmed
| 0
| 0
|-
| 16
| Bit 4
| if variance resolved less than a threshold
| 50%
| 1
| 16
|-
| 32
| Bit 5
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 64
| Bit 6
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 128
| Bit 7
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|
|
|
|
|
| Final Q = 20
|}
<br />
The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.
[[Category: Shear probes]]
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]


[[Category:Velocity profilers]]
[[Category:Velocity profilers]]

Revision as of 20:41, 3 June 2022

Quality control measures for each beam:

  1. Data segments for which the regression coefficient a1 (see previous step) is negative yield an imaginary ε value, which should be rejected
  2. Ensure sufficient Dll samples were used in the regression.
  3. Use the coefficient a0 (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. If noise is too high, ϵ are rejected.
  4. Data segments for which the regression coefficient a0 (see previous step) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
  5. Examine the consistency of ε between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
  6. Evaluate the impact of varying rmax values (within the anticipated inertial range) on ε; an increase in ε with increasing rmax is likely to indicate that v’ retains a non-turbulent contribution to the velocity difference between bins
  7. The goodness of fit (R2) for the regression provides a basic indication of the quality of the fit
  8. A better indication of the quality of the fit is usually provided by looking at the ratio of the estimated ε value to that based on the 95%-ile confidence interval estimate of the a1 regression coefficient e.g. reject values where the ratio exceeds a specified threshold
  9. Examine the distribution of ε estimates - in most situations, this would be expected to be log-normal
  10. Comparison of observed values with nominal values based on established boundary-forced scalings may also be informative and help to identify observation or processing issues

[In progress]


How ADCP structure function quality-control flags are applied

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites Ocean Sites for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag. The numerical values of the Q flags are as follows:

Flag Mask Bit Flag Meaning Example threshold value Ex: True =1 / False =0 Ex: Q value
1 Bit 0 if FOM > FOM_limit 2 0 0
2 Bit 1 if despike_fraction > despike_fraction_limit 40% 0 0
4 Bit 2 log(e_max)-log(e_min)|> diss_ratio_limit X \sigma_{\ln\varepsilon} N/A 1 4
8 Bit 3 if despike_iterations > despike_iterations_limit To be confirmed 0 0
16 Bit 4 if variance resolved less than a threshold 50% 1 16
32 Bit 5 manual flag to be defined by user N/A 0 0
64 Bit 6 manual flag to be defined by user N/A 0 0
128 Bit 7 manual flag to be defined by user N/A 0 0
Final Q = 20


The Q flags are combined by their addition. For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test. A value of 255 means that all tests failed. The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.


Return to ADCP Flow Chart front page