Quality control of ε estimates (QA2): Difference between revisions

From Atomix
mNo edit summary
Yuengdjern (talk | contribs)
No edit summary
Line 1: Line 1:
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings.
Quality control measures for each beam:
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Ensure sufficient <math> D_{ll} </math> samples were used in the regression.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. If noise is too high, <math> \epsilon </math> are rejected.
# Data segments for which the regression coefficient a<sub>0</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
# Data segments for which the regression coefficient a<sub>0</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
# Examine the consistency of <math>\varepsilon</math> between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
# Examine the consistency of <math>\varepsilon</math> between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
Line 10: Line 12:


------  
------  
'''[In progress]
'''
'''How ADCP structure function quality-control flags are applied'''
The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.
Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:
{| class="wikitable"
|-
! Flag Mask
! Bit
! Flag Meaning
! Example threshold value
| Ex: True =1  / False =0
| Ex: Q value
|-
| 1
| Bit 0
| if FOM > FOM_limit
| 2
| 0
| 0
|-
| 2
| Bit 1
| if despike_fraction > despike_fraction_limit
| 40%
| 0
| 0
|-
| 4
| Bit 2
| if |log(e_max)-log(e_min)|> diss_ratio_limit X \sigma_{\ln\varepsilon}
| N/A
| 1
| 4
|-
| 8
| Bit 3
| if despike_iterations > despike_iterations_limit
| To be confirmed
| 0
| 0
|-
| 16
| Bit 4
| if variance resolved less than a threshold
| 50%
| 1
| 16
|-
| 32
| Bit 5
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 64
| Bit 6
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 128
| Bit 7
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|
|
|
|
|
| Final Q = 20
|}
<br />
The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.
[[Category: Shear probes]]
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]


[[Category:Velocity profilers]]
[[Category:Velocity profilers]]

Revision as of 20:41, 3 June 2022

Quality control measures for each beam:

  1. Data segments for which the regression coefficient a1 (see previous step) is negative yield an imaginary [math]\displaystyle{ \varepsilon }[/math] value, which should be rejected
  2. Ensure sufficient [math]\displaystyle{ D_{ll} }[/math] samples were used in the regression.
  3. Use the coefficient [math]\displaystyle{ a_0 }[/math] (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. If noise is too high, [math]\displaystyle{ \epsilon }[/math] are rejected.
  4. Data segments for which the regression coefficient a0 (see previous step) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
  5. Examine the consistency of [math]\displaystyle{ \varepsilon }[/math] between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
  6. Evaluate the impact of varying rmax values (within the anticipated inertial range) on [math]\displaystyle{ \varepsilon }[/math]; an increase in [math]\displaystyle{ \varepsilon }[/math] with increasing rmax is likely to indicate that v’ retains a non-turbulent contribution to the velocity difference between bins
  7. The goodness of fit (R2) for the regression provides a basic indication of the quality of the fit
  8. A better indication of the quality of the fit is usually provided by looking at the ratio of the estimated [math]\displaystyle{ \varepsilon }[/math] value to that based on the 95%-ile confidence interval estimate of the a1 regression coefficient e.g. reject values where the ratio exceeds a specified threshold
  9. Examine the distribution of [math]\displaystyle{ \varepsilon }[/math] estimates - in most situations, this would be expected to be log-normal
  10. Comparison of observed values with nominal values based on established boundary-forced scalings may also be informative and help to identify observation or processing issues

[In progress]


How ADCP structure function quality-control flags are applied

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites Ocean Sites for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag. The numerical values of the Q flags are as follows:

Flag Mask Bit Flag Meaning Example threshold value Ex: True =1 / False =0 Ex: Q value
1 Bit 0 if FOM > FOM_limit 2 0 0
2 Bit 1 if despike_fraction > despike_fraction_limit 40% 0 0
4 Bit 2 log(e_max)-log(e_min)|> diss_ratio_limit X \sigma_{\ln\varepsilon} N/A 1 4
8 Bit 3 if despike_iterations > despike_iterations_limit To be confirmed 0 0
16 Bit 4 if variance resolved less than a threshold 50% 1 16
32 Bit 5 manual flag to be defined by user N/A 0 0
64 Bit 6 manual flag to be defined by user N/A 0 0
128 Bit 7 manual flag to be defined by user N/A 0 0
Final Q = 20


The Q flags are combined by their addition. For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test. A value of 255 means that all tests failed. The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.


Return to ADCP Flow Chart front page