Quality control of ε estimates (QA2): Difference between revisions

From Atomix
← Older edit
VisualWikitext
Yuengdjern (talk | contribs)
215 edits
No edit summary
Yuengdjern (talk | contribs)
215 edits
No edit summary
 
Line 1: Line 1:
Quality control measures for each beam:
Quality control measures for each beam (flagged in benchmark datasets):
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Ensure sufficient <math> D_{ll} </math> samples were used in the regression.  
# Ensure sufficient <math> D_{ll} </math> samples were used in the regression.  
Line 16: Line 16:
------  
------  
To see how the data flags are applied, go to [[Velocity Profiler data flags| Velocity Profiler Data Flags]]
To see how the data flags are applied, go to [[Velocity Profiler data flags| Velocity Profiler Data Flags]]
'''[In progress]
'''
'''How ADCP structure function quality-control flags are applied'''
The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.
Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:
{| class="wikitable"
|-
! Flag Mask
! Bit
! Flag Meaning
! Example threshold value
| Ex: True =1  / False =0
| Ex: Q value
|-
| 1
| Bit 0
| if FOM > FOM_limit
| 2
| 0
| 0
|-
| 2
| Bit 1
| if despike_fraction > despike_fraction_limit
| 40%
| 0
| 0
|-
| 4
| Bit 2
| if |log(e_max)-log(e_min)|> diss_ratio_limit X \sigma_{\ln\varepsilon}
| N/A
| 1
| 4
|-
| 8
| Bit 3
| if despike_iterations > despike_iterations_limit
| To be confirmed
| 0
| 0
|-
| 16
| Bit 4
| if variance resolved less than a threshold
| 50%
| 1
| 16
|-
| 32
| Bit 5
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 64
| Bit 6
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 128
| Bit 7
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|
|
|
|
|
| Final Q = 20
|}
<br />
The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.




Latest revision as of 20:58, 3 June 2022

Quality control measures for each beam (flagged in benchmark datasets):

  1. Data segments for which the regression coefficient a1 (see previous step) is negative yield an imaginary [math]\displaystyle{ \varepsilon }[/math] value, which should be rejected
  2. Ensure sufficient [math]\displaystyle{ D_{ll} }[/math] samples were used in the regression.
  3. Use the coefficient [math]\displaystyle{ a_0 }[/math] (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. If noise is too high, [math]\displaystyle{ \epsilon }[/math] are rejected.
  4. Data segments for which the regression coefficient a0 (see previous step) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
  5. In the case of [math]\displaystyle{ \epsilon }[/math] estimated using the modified regression method that accounts for oscillatory motion, reject data for invalid values of [math]\displaystyle{ a_3 }[/math].
  6. A better indication of the quality of the fit is usually provided by looking at the ratio of the estimated [math]\displaystyle{ \varepsilon }[/math] value to that based on the 95%-ile confidence interval estimate of the a1 regression coefficient e.g. reject values where the ratio exceeds a specified threshold
  7. The goodness of fit (R2) for the regression provides a basic indication of the quality of the fit, data with low R2 are typically rejected.

Other measures (not flagged):

  1. Examine the distribution of [math]\displaystyle{ \varepsilon }[/math] estimates - in most situations, this would be expected to be log-normal
  2. Comparison of observed values with nominal values based on established boundary-forced scalings may also be informative and help to identify observation or processing issues

Quality control measures for final [math]\displaystyle{ \epsilon }[/math] estimate:

  1. Examine the consistency of [math]\displaystyle{ \varepsilon }[/math] between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value

To see how the data flags are applied, go to Velocity Profiler Data Flags


Return to ADCP Flow Chart front page

Retrieved from "http://atomix.app.uib.no/index.php?title=Quality_control_of_ε_estimates_(QA2)&oldid=4136"