Raw data review (QA1): Difference between revisions

From Atomix
Changing bullets
Trying to make items more specific to help new ADCP users.
Line 1: Line 1:
== Raw data review ==
== Raw data review ==


The raw data review seeks to identify the Level 1 data that satisfies a range of quality control criteria. These may involve comparison of data between beams, between bins and over time and be informed by data from other sensors e.g. meteorological, wave or CTD sensors. Typical criteria used to identify possible bad data include:
The objective of the raw data review is to ensure that the velocity data used for the calculation of the structure functions are of good quality before proceeding with the turbulence analysis. Bad data are typically identified from the velocity data themselves and other ancillary data (e.g. correlations). Recommendations for quality control based on each data type include:  


# Correlation  
# Correlation  
#* minimum threshold
#* flag data below a minimum threshold
# Echo intensity  
# Echo intensity  
#* Identify false target / “fish” detection  
#* flag data where there are anomalies due to false targets (e.g “fish” detection, surface reflection, mooring line reflection)
# Percent good
# Percent good
#* applies to measurement modes with in-instrument averaging across multiple pings per ensemble
#* flag data with low values (applies to measurement modes with in-instrument averaging across multiple pings per ensemble)
# Orientation (heading, pitch, roll) and depth (if sensor installed)  
# Orientation (heading, pitch, roll) and depth (if sensor installed)  
#* resolve location of observations
#* flag periods of high variability 
#* identify factors affecting location and any periodic motion
#* flag periods where instrument orientation deviates significantly from the expected behaviour of the mooring or instrument frame [(JMM) REPLACEMENT FOR 'resolve location of observations', WHICH I FOUND CONFUSING. IS THIS ANY BETTER?]
# Along beam velocity
# Along beam velocity
#* variations in data return rate
#* flag values where there are variations in data return rate
#* values outside the nominal measurement range for the instrument configuration
#* flag values outside the nominal measurement range for the instrument configuration
#* evidence of phase wrapping (ambiguity velocity) in pulse-pulse coherent observations  
#* flag values with evidence of phase wrapping (in pulse-pulse coherent observations)
#* periodicity indicating waves or oscillatory motion  
#* flag values where there is periodicity indicating waves or oscillatory motion  
#* distribution outliers
#* flag values that are outside of the expected distribution  
#* burst variance spatial and temporal trends  
#* flag values where the burst variance shows evidence of spatial and temporal trends
# Earth coordinate velocities
#* flag data where the error velocity is large (from ADCPs with at least 4 beams)
#* flag data where the earth velocities are comparable to the ambiguity velocity to check for possible phase wrapping
#* flag data where the burst variance shows evidence of spatial and temporal trends  
#* flag data where there is a large horizontal shear over the observation range
#* flag data where the ADCP orientation deviates from the vertical and bin mapping may be necessary [(JMM) ALREADY INCLUDED ABOVE?]
# Temperature and salinity (if sensors installed)  
# Temperature and salinity (if sensors installed)  
#* indication of changes in local stratification and/or internal wave activity  
#* flag values where there is evidence of changes in local stratification and/or internal wave activity  
# Earth coordinate velocity 
 
#* may need to be derived from along-beam velocity
If possible, it is also recommended that you compare your data between beams, between bins and over time. It can also be useful to compare data to other sensors (e.g. meteorological, wave or CTD sensors).
#* bin mapping if ADCP orientation isn’t vertical
 
#* error velocity from 4-beam instruments
#* comparison with ambiguity velocity to check for possible phase wrapping
#* burst variance spatial and temporal trends
#* shear over observation range


Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

Revision as of 00:18, 11 November 2021

Raw data review

The objective of the raw data review is to ensure that the velocity data used for the calculation of the structure functions are of good quality before proceeding with the turbulence analysis. Bad data are typically identified from the velocity data themselves and other ancillary data (e.g. correlations). Recommendations for quality control based on each data type include:

  1. Correlation
    • flag data below a minimum threshold
  2. Echo intensity
    • flag data where there are anomalies due to false targets (e.g “fish” detection, surface reflection, mooring line reflection)
  3. Percent good
    • flag data with low values (applies to measurement modes with in-instrument averaging across multiple pings per ensemble)
  4. Orientation (heading, pitch, roll) and depth (if sensor installed)
    • flag periods of high variability
    • flag periods where instrument orientation deviates significantly from the expected behaviour of the mooring or instrument frame [(JMM) REPLACEMENT FOR 'resolve location of observations', WHICH I FOUND CONFUSING. IS THIS ANY BETTER?]
  5. Along beam velocity
    • flag values where there are variations in data return rate
    • flag values outside the nominal measurement range for the instrument configuration
    • flag values with evidence of phase wrapping (in pulse-pulse coherent observations)
    • flag values where there is periodicity indicating waves or oscillatory motion
    • flag values that are outside of the expected distribution
    • flag values where the burst variance shows evidence of spatial and temporal trends
  6. Earth coordinate velocities
    • flag data where the error velocity is large (from ADCPs with at least 4 beams)
    • flag data where the earth velocities are comparable to the ambiguity velocity to check for possible phase wrapping
    • flag data where the burst variance shows evidence of spatial and temporal trends
    • flag data where there is a large horizontal shear over the observation range
    • flag data where the ADCP orientation deviates from the vertical and bin mapping may be necessary [(JMM) ALREADY INCLUDED ABOVE?]
  7. Temperature and salinity (if sensors installed)
    • flag values where there is evidence of changes in local stratification and/or internal wave activity

If possible, it is also recommended that you compare your data between beams, between bins and over time. It can also be useful to compare data to other sensors (e.g. meteorological, wave or CTD sensors).


Return to ADCP Flow Chart front page