Atomix - User contributions [en]

Velocity Profiler data flags

2023-01-10T13:57:30Z

Brian scannell:

'''How ADCP structure function quality-control flags are applied'''

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:

'''Level 2:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| segment_outlier
| Flag data points <math> > 3 \sigma</math> away from segment mean for bin.
| <math>3 \sigma</math>
| 1
| 2
|-
| 4
| Bit 2
| profile_outlier
| Flag data points <math> > 3 \sigma</math> away from beam-profile-mean.
| <math>3 \sigma</math>
| 0
| 0
|-
|
|
|
|
|
|
| Final Q = 2
|}

'''Level 3:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| insufficient_number_velocity_samples
| Flag segments where the ratio of the number of viable values in the velocity difference to the total number of data points is less than the threshold
| 0.5
| 0
| 0
|-
| 4
| Bit 2
| too_close_to_range_limits
| Flag bins that are too close to the first or last range bin (I.e. not enough bins to do differencing)
| 3
| 1
| 4
|-
|
|
|
|
|
|
| Final Q = 4
|}

'''Level 4:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| Rsquared_too_low
| Flag values where <math> R^2 < </math> threshold
| 0.6
| 0
| 0
|-
| 4
| Bit 2
| delta_epsi_too_large
| Flag values where <math> \delta \epsilon / \epsilon > </math> threshold
| 0.6
| 1
| 4
|-
| 8
| Bit 3
| A3_coeff_invalid
| Flag values where <math> a_3 </math> < threshold
| 0
| 0
| 0
|-
| 16
| Bit 4
| dll_intercept_too_low
| Flag values where intercept (<math> a_0 </math>) < threshold
| 0
| 1
| 16
|-
| 32
| Bit 5
| dll_intercept_too_high
| Flag values where intercept > threshold (threshhold <math> = 2*yint\_{expected} = 2*(2*\sigma_v^2) </math>, where <math> \sigma_v </math> is expected accuracy in along-beam vel.
| Instrument dependent
| 0
| 0
|-
| 64
| Bit 6
| regression_poorly_conditioned
| Flag values where there are fewer than a certain number of points for the regression
| 3
| 0
| 0
|-
| 128
| Bit 7
| dll_slope_out_of_range
| Flag values where slope <math> a_1 < 0 </math> as this would lead to negative epsilon.
| 0
| 0
| 0
|-
|
|
|
|
|
|
| Final Q = 20
|}

<br />

The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.
Note that importance (severity of the failure) is ranked by the numeric value of the flag mask.

Level 4 data (velocity profilers)

2022-07-11T14:04:29Z

Brian scannell:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]).
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profiler data flags | Velocity Profiler data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_DEL_RATIO
| specific_turbulent_kinetic_energy_dissipation_in_sea_water_error_ratio
| TIME, Z_DIST, N_BEAM
| Ratio of the uncertainty, <math>\delta\varepsilon</math> to the value of <math>\varepsilon</math>. Computed from the confidence intervals as <math>\frac{\delta\varepsilon}{\epsilon} = \frac{3\ (\text{SLOPE_CI_HIGH}\ -\ \text{SLOPE_CI_LOW})}{4\ \text{REGRESSION_COEFF_A1}}</math> {{FontColor|fg=red|text=Link to derivation}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]])
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| regression_model
| model regressed against the data to determine <math>\varepsilon</math>
| Examples include:
standard model, i.e. <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}</math> <br /><br />
modified model, i.e. <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}+a_3((\delta r)^{2/3})^3 </math>
|-
| regression_method
| method used to regress the model against the data
| Examples include:
Least-squares using evaluated separation distances to r_max.
|-
| EPSI_FINAL_method
| method used to calculate EPSI_FINAL from EPSI
| Examples include:
arithmetic mean across beams of resolved EPSI after all EPSI_FLAGS applied.
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-07-11T14:02:31Z

Brian scannell:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]).
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profiler data flags | Velocity Profiler data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_DEL_RATIO
| specific_turbulent_kinetic_energy_dissipation_in_sea_water_error_ratio
| TIME, Z_DIST, N_BEAM
| Ratio of the uncertainty, <math>\delta\varepsilon</math> to the value of <math>\varepsilon</math>. Computed from the confidence intervals as <math>\frac{\delta\varepsilon}{\epsilon} = \frac{3\ (\text{EPSI_CI_HIGH}\ -\ \text{EPSI_CI_LOW})}{4\ \text{REGRESSION_COEFF_A1}}</math> {{FontColor|fg=red|text=Link to derivation}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]])
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| regression_model
| model regressed against the data to determine <math>\varepsilon</math>
| Examples include:
standard model, i.e. <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}</math> <br /><br />
modified model, i.e. <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}+a_3((\delta r)^{2/3})^3 </math>
|-
| regression_method
| method used to regress the model against the data
| Examples include:
Least-squares using evaluated separation distances to r_max.
|-
| EPSI_FINAL_method
| method used to calculate EPSI_FINAL from EPSI
| Examples include:
arithmetic mean across beams of resolved EPSI after all EPSI_FLAGS applied.
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 2 data (velocity profilers)

2022-07-11T09:29:04Z

Brian scannell: /* Variables \ddagger */

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Array of 1 to number of segments into which the data has been separated (Note TIME variable)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}. Corresponds to <math> b^\prime </math> in [[Nomenclature]]
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables''' <math>\ddagger</math>
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile and is a proxy for TIME
|-
| R_VEL_DETRENDED_FLAGS
| water_radial_velocity_of_scatterers_towards_
instrument_status_flag
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| This variable provides flags for outliers in each segment <math> \geq 2 </math> standard deviations.
|-
| colspan="4" |
<math>\ddagger</math> One could also re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after restructuring into segments.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Level 4 data (velocity profilers)

2022-06-13T11:55:34Z

Brian scannell: Corrected units for REGRESSION_COEFF_A3

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]).
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profiler data flags | Velocity Profiler data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]])
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 2 data (velocity profilers)

2022-04-22T10:41:02Z

Brian scannell:

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Array of 1 to number of segments into which the data has been separated (Note TIME variable)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)
|}
</div>

=Variables <math>\ddagger</math>=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| Along beam velocity [m/s] for each of the beams.
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after restructuring into segments.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Level 1 data (velocity profilers)

2022-04-22T10:31:09Z

Brian scannell: Format tweaks

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
The required dimensions and variables for the first processing level within NetCDF ATOMIX format for ADCP velocity measurements are described below. This processing level contains the raw measurements recorded by the instrument. If sampling is in [[Burst sampling|burst mode]], the measurements from individual [[Burst sampling|bursts]] are appended together.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
==Required variables==
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comments and units
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| TIME, Z_DIST, N_BEAM
| Units in m/s
|-
| R_VEL_FLAGS
| water_radial_velocity_of_scatterers_
towards_instrument_status_flag
| TIME, Z_DIST, N_BEAM
| Boolean flags (8 bit, 0-255) to represent one of 8 possible reasons for rejection. {{FontColor|fg=white|bg=red|text= Link to own flagging page}}
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| vertical size of the ADCP bins. Usually the same for all 4 beams, but 5th beam can vary.
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| TIME
| This variable identifies each record (velocity) profile
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Quality-control variables recorded by acoustic-Doppler instruments''' <math>\ddagger</math>
|-
| ABSIC
| backscatter_intensity
| TIME, Z_DIST, N_BEAM
| Units of counts
|-
| CORR
| correlation_magnitude_from_each_acoustic_beam
| TIME, Z_DIST, N_BEAM
| Unit of counts
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Platform motion variables''' <math>\dagger</math>
|-
| HEADING
| platform_yaw_angle
| TIME
| degrees, clockwise from true North
|-
| PITCH
| platform_pitch_angle
| TIME
| degrees
|-
| ROLL
| platform_roll_angle
| TIME
| degrees
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables''' <math>\dagger\dagger</math>
|-
| BURST_NUMBER
| unique_identifier_for_each_burst
| TIME
| Integers of 1, 2, etc to designate which [[Burst sampling|burst]] the velocities are associated with. Only required when measuring in burst-mode.
|-
| colspan="4" | <math>\ddagger</math> Quality-control variable names are consistent for those onboard RDI Teleydyne ADCPs. Nortek has CORRN (noise correlation as a % instead of counts) detailed in the [[Level_1_data_(velocity_point-measurements)#Variables|ADV Level 1 variables]].

<math>\dagger</math> Motion variables are necessary for structure function analysis for situating the bins relative the vertical (gravity). These variables do enable converting measurements into geographical.

<math>\dagger\dagger</math> Only required when measuring in bursts e.g., 18 min of continuous measurements every hour. Can omit if continuously sampled.
|}

==Optional ancillary variables==

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| PRES
| sea_water_pressure
| TIME
| dbar, equals 0 at the sea surface and positive down.
|-
| TEMP*
| sea_water_temperature
| TIME
| degrees_Celsius, in-situ temperature ITS-90 scale
|-
| colspan="4" | *Data from concurrent sensors may be optionally included e.g., salinity, dissolved oxygen. Kinematic viscosity of seawater needs to be calculated during processing,
|}

</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">
<br>
This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.
{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the ATOMIX Level 1 content.
| <blockquote>''This group includes the raw measurements from the recorder and ancillary measurements required for quality-controlling them using the manufacturer's recommendations.''</blockquote>
|-
| comment (optional)
| Information is pertinent to problems in the raw data files during collection.
| Examples: Stitching of files, corruption of binary files that were recovered by the manufacturer, etc.
|}
</div>

----
Return to [[Dataset requirements for ADCP structure function]]

Go to [[Level 2 data (velocity profilers)| Level 2 segmented]]

Level 3 data (velocity profilers)

2022-04-22T10:21:35Z

Brian scannell: Removed reference to BURST_NUMBER as no longer relevant

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
[[File:SF atomix ADCP.png|300px|thumb|Schematic of ADCP processing nomenclature]]
The required dimensions and variables for the structure-function processing level within NetCDF ATOMIX format for velocity ADCP measurements are described below. This NetCDF group contains the structure function (DLL) calculated as a function of the along-beam separation for the available/usable ADCP bins.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Segment midpoint time. Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_DEL
| along-beam_separation_distance_over_
which_DLL_is_evaluated_in_number_of_bins
| N_DEL
| Number of bins separating two velocity measurements used to calculate DLL
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| DLL
| second_order_structure_function
| TIME, Z_DIST, N_BEAM, N_DEL
| Differences in velocities squared have been time-averaged (units of m2/s2).
|-
| DLL_FLAGS
| second_order_structure_function_
status_flag
| TIME, Z_DIST, N_BEAM, N_DEL
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| R_DEL
| along-beam_separation_distance_at_
which_structure_function_is_evaluated
| TIME, Z_DIST, N_BEAM, N_DEL
| Estimated quantity (in meters) from N_DEL (Level 3), BIN_SIZE (Level 2) and THETA (Level 2).
|-
| R_DIST
| distance_from_sensor_along_beams
| Z_DIST, N_BEAM
| Along-beam bin centre distance (in meters) from the transducer
|-
| DLL_N
| second_order_structure_function_number_
of_observations
| TIME, Z_DIST, N_BEAM, N_DEL
| The number of available measurements in each segment i.e., data quality.
|-
| N_SEGMENT
| unique_identifier_for_each_segment_
in_the_entire_available_timeseries
| TIME
| Enables backtracking to [[Level 2 segmented (velocity profilers)|previous processing level]]
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the structure function Dll as a function of the separation distance. Any ancillary information required for estimating the dissipation of turbulent kinetic energy may also be stored here. ''</blockquote>
|-
| dll_calculation_type
| Specify differencing technique used to estimate DLL
| Examples include:
<blockquote>Central-differencing</blockquote>
<blockquote>Forward-differencing</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes <math>\ddagger</math>
|-
| stationarity_testing
| Any testing done on the segment to verify stationarity?
| {{FontColor|fg=white|bg=red|text=To be revisited once testing begins}}. Tentatively refer to [https://bitbucket.org/efm_cb/netcdf/src/master/TestData/adcp_atomix_metada.yml demo yaml] file.
|-
| noise_testing
| Details of testing the noise levels, or if the signal comprises mostly of noise?
|
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 2 data (velocity profilers)| Level 2 segmented velocities]]

Go to [[Level 4 data (velocity profilers)| Level 4 dissipation estimates]]

Level 3 data (velocity profilers)

2022-04-22T10:19:06Z

Brian scannell: Update order of dimensions as per TEAMS discussion

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
[[File:SF atomix ADCP.png|300px|thumb|Schematic of ADCP processing nomenclature]]
The required dimensions and variables for the structure-function processing level within NetCDF ATOMIX format for velocity ADCP measurements are described below. This NetCDF group contains the structure function (DLL) calculated as a function of the along-beam separation for the available/usable ADCP bins.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Segment midpoint time. Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_DEL
| along-beam_separation_distance_over_
which_DLL_is_evaluated_in_number_of_bins
| N_DEL
| Number of bins separating two velocity measurements used to calculate DLL
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| DLL
| second_order_structure_function
| TIME, Z_DIST, N_BEAM, N_DEL
| Differences in velocities squared have been time-averaged (units of m2/s2).
|-
| DLL_FLAGS
| second_order_structure_function_
status_flag
| TIME, Z_DIST, N_BEAM, N_DEL
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| R_DEL
| along-beam_separation_distance_at_
which_structure_function_is_evaluated
| TIME, Z_DIST, N_BEAM, N_DEL
| Estimated quantity (in meters) from N_DEL (Level 3), BIN_SIZE (Level 2) and THETA (Level 2).
|-
| R_DIST
| distance_from_sensor_along_beams
| Z_DIST, N_BEAM
| Along-beam bin centre distance (in meters) from the transducer
|-
| DLL_N
| second_order_structure_function_number_
of_observations
| TIME, Z_DIST, N_BEAM, N_DEL
| The number of available measurements in each segment i.e., data quality.
|-
| N_SEGMENT
| unique_identifier_for_each_segment_
in_the_entire_available_timeseries
| TIME
| Enables backtracking to [[Level 2 segmented (velocity profilers)|previous processing level]]
|-
| BURST_NUMBER <math>\dagger\dagger</math>
| unique_identifier_for_each_burst
| TIME
| Only required when measuring in burst-mode. Integers of 1, 2, etc to designate which burst the velocities are associated with.
|-
| colspan="4" |
<math>\dagger\dagger</math> Optional variable required only when measuring in bursts e.g., 18 min of continuous measurements every hour.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the structure function Dll as a function of the separation distance. Any ancillary information required for estimating the dissipation of turbulent kinetic energy may also be stored here. ''</blockquote>
|-
| dll_calculation_type
| Specify differencing technique used to estimate DLL
| Examples include:
<blockquote>Central-differencing</blockquote>
<blockquote>Forward-differencing</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes <math>\ddagger</math>
|-
| stationarity_testing
| Any testing done on the segment to verify stationarity?
| {{FontColor|fg=white|bg=red|text=To be revisited once testing begins}}. Tentatively refer to [https://bitbucket.org/efm_cb/netcdf/src/master/TestData/adcp_atomix_metada.yml demo yaml] file.
|-
| noise_testing
| Details of testing the noise levels, or if the signal comprises mostly of noise?
|
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 2 data (velocity profilers)| Level 2 segmented velocities]]

Go to [[Level 4 data (velocity profilers)| Level 4 dissipation estimates]]

Level 2 data (velocity profilers)

2022-04-22T10:09:40Z

Brian scannell: Modified description of dimensions N_SEGMENT and N_SAMPLE

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Array of 1 to number of segments into which the data has been separated (Note TIME variable)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| Along beam velocity [m/s] for each of the beams.
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after segmenting them into smaller chunks.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Level 2 data (velocity profilers)

2022-04-22T10:02:08Z

Brian scannell: Modified order of dimensions to match latest proposal in TEAMS discussion

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Replacement for using TIME centred in the segment as dimension
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Max sample number is the segment length (seconds) multiplied by the sampling rate
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| Along beam velocity [m/s] for each of the beams.
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after segmenting them into smaller chunks.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Level 1 data (velocity profilers)

2022-04-22T09:49:00Z

Brian scannell: Changed dimension for PROFILE_NUMBER to TIME (was listed as N_SEGMENT, N_SAMPLE which are only relevant at L2)

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
The required dimensions and variables for the first processing level within NetCDF ATOMIX format for ADCP velocity measurements are described below. This processing level contains the raw measurements recorded by the instrument. If sampling is in [[Burst sampling|burst mode]], the measurements from individual [[Burst sampling|bursts]] are appended together.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
==Required variables==
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comments and units
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| TIME, Z_DIST, N_BEAM
| Units in m/s
|-
| R_VEL_FLAGS
| water_radial_velocity_of_scatterers_
towards_instrument_status_flag
| TIME, Z_DIST, N_BEAM
| Boolean flags (8 bit, 0-255) to represent one of 8 possible reasons for rejection. {{FontColor|fg=white|bg=red|text= Link to own flagging page}}
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| vertical size of the ADCP bins. Usually the same for all 4 beams, but 5th beam can vary.
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| TIME
| This variable identifies each record (velocity) profile
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Quality-control variables recorded by acoustic-Doppler instruments''' <math>\ddagger</math>
|-
| ABSIC
| backscatter_intensity
| TIME, Z_DIST, N_BEAM
| Units of counts
|-
| CORR
| correlation_magnitude_from_each_acoustic_beam
| TIME, Z_DIST, N_BEAM
| Unit of counts
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Platform motion variables'''<math>\dagger</math>
|-
| HEADING
| platform_yaw_angle
| TIME
| degrees, clockwise from true North
|-
| PITCH
| platform_pitch_angle
| TIME
| degrees
|-
| ROLL
| platform_roll_angle
| TIME
| degrees
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables'''
|-
| BURST_NUMBER
| unique_identifier_for_each_burst
| TIME
| Integers of 1, 2, etc to designate which [[Burst sampling|burst]] the velocities are associated with. Can omit if continuously sampled.
|-
| colspan="4" | <math>\ddagger</math> Quality-control variable names are consistent for those onboard RDI Teleydyne ADCPs. Nortek has CORRN (noise correlation as a % instead of counts) detailed in the [[Level_1_data_(velocity_point-measurements)#Variables|ADV Level 1 variables]].

<math>\dagger</math> Motion variables are necessary for structure function analysis for situating the bins relative the vertical (gravity). These variables do enable converting measurements into geographical.
|}

==Optional ancillary variables==

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| PRES
| sea_water_pressure
| TIME
| dbar, equals 0 at the sea surface and positive down.
|-
| TEMP*
| sea_water_temperature
| TIME
| degrees_Celsius, in-situ temperature ITS-90 scale
|-
| colspan="4" | *Data from concurrent sensors may be optionally included e.g., salinity, dissolved oxygen. Kinematic viscosity of seawater needs to be calculated during processing,
|}

</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">
<br>
This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.
{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the ATOMIX Level 1 content.
| <blockquote>''This group includes the raw measurements from the recorder and ancillary measurements required for quality-controlling them using the manufacturer's recommendations.''</blockquote>
|-
| comment (optional)
| Information is pertinent to problems in the raw data files during collection.
| Examples: Stitching of files, corruption of binary files that were recovered by the manufacturer, etc.
|}
</div>

----
Return to [[Dataset requirements for ADCP structure function]]

Go to [[Level 2 data (velocity profilers)| Level 2 segmented]]

Level 1 data (velocity profilers)

2022-04-22T09:46:07Z

Brian scannell: changed reference to R_DIST to Z_DIST

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
The required dimensions and variables for the first processing level within NetCDF ATOMIX format for ADCP velocity measurements are described below. This processing level contains the raw measurements recorded by the instrument. If sampling is in [[Burst sampling|burst mode]], the measurements from individual [[Burst sampling|bursts]] are appended together.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
==Required variables==
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comments and units
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| TIME, Z_DIST, N_BEAM
| Units in m/s
|-
| R_VEL_FLAGS
| water_radial_velocity_of_scatterers_
towards_instrument_status_flag
| TIME, Z_DIST, N_BEAM
| Boolean flags (8 bit, 0-255) to represent one of 8 possible reasons for rejection. {{FontColor|fg=white|bg=red|text= Link to own flagging page}}
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| vertical size of the ADCP bins. Usually the same for all 4 beams, but 5th beam can vary.
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Quality-control variables recorded by acoustic-Doppler instruments''' <math>\ddagger</math>
|-
| ABSIC
| backscatter_intensity
| TIME, Z_DIST, N_BEAM
| Units of counts
|-
| CORR
| correlation_magnitude_from_each_acoustic_beam
| TIME, Z_DIST, N_BEAM
| Unit of counts
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Platform motion variables'''<math>\dagger</math>
|-
| HEADING
| platform_yaw_angle
| TIME
| degrees, clockwise from true North
|-
| PITCH
| platform_pitch_angle
| TIME
| degrees
|-
| ROLL
| platform_roll_angle
| TIME
| degrees
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables'''
|-
| BURST_NUMBER
| unique_identifier_for_each_burst
| TIME
| Integers of 1, 2, etc to designate which [[Burst sampling|burst]] the velocities are associated with. Can omit if continuously sampled.
|-
| colspan="4" | <math>\ddagger</math> Quality-control variable names are consistent for those onboard RDI Teleydyne ADCPs. Nortek has CORRN (noise correlation as a % instead of counts) detailed in the [[Level_1_data_(velocity_point-measurements)#Variables|ADV Level 1 variables]].

<math>\dagger</math> Motion variables are necessary for structure function analysis for situating the bins relative the vertical (gravity). These variables do enable converting measurements into geographical.
|}

==Optional ancillary variables==

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| PRES
| sea_water_pressure
| TIME
| dbar, equals 0 at the sea surface and positive down.
|-
| TEMP*
| sea_water_temperature
| TIME
| degrees_Celsius, in-situ temperature ITS-90 scale
|-
| colspan="4" | *Data from concurrent sensors may be optionally included e.g., salinity, dissolved oxygen. Kinematic viscosity of seawater needs to be calculated during processing,
|}

</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">
<br>
This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.
{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the ATOMIX Level 1 content.
| <blockquote>''This group includes the raw measurements from the recorder and ancillary measurements required for quality-controlling them using the manufacturer's recommendations.''</blockquote>
|-
| comment (optional)
| Information is pertinent to problems in the raw data files during collection.
| Examples: Stitching of files, corruption of binary files that were recovered by the manufacturer, etc.
|}
</div>

----
Return to [[Dataset requirements for ADCP structure function]]

Go to [[Level 2 data (velocity profilers)| Level 2 segmented]]

Talk:Level 4 data (velocity profilers)

2022-02-03T15:27:02Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:07, 7 January 2022 (CET) suggested Comments:<br>
R_MAX - maximum R_DEL separation distance used for the regression, dimensions should be N_BEAM and units is meters<br>
REGRESSION_COEFFICIENT_A0 - intercept from the regression of DLL against R_DEL, indicative of noise, units is meter**2 / second**2<br>
REGRESSION_COEFFICIENT_A1 - regression coefficient from the regression of DLL against R_DEL used in the calculation of <math>\varepsilon</math>, units is meter**(4/3) / second**2<br>
REGRESSION_R2 - goodness of fit parameter for the regression (user to include method description in metadata)<br>
REGRESSION_N - number of data points used in the regression

Level 1 data (velocity profilers)

2022-01-24T09:34:37Z

Brian scannell:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
The required dimensions and variables for the first processing level within NetCDF ATOMIX format for ADCP velocity measurements are described below. This processing level contains the raw measurements recorded by the instrument. If sampling is in burst mode, the measurements from individual bursts are appended together.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z
|-
| Z_DIST
| distance_from_sensor_along_z_instrument_axis
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
==Required variables==
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comments and units
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| TIME, Z_DIST, N_BEAM
| Units in m/s
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30o)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| vertical size of the ADCP bins. Usually the same for all 4 beams, but 5th beam can vary.
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Quality-control variables recorded by acoustic-Doppler instruments''' <math>\ddagger</math>
|-
| ABSIC
| backscatter_intensity
| TIME, Z_DIST, N_BEAM
| Units of counts
|-
| CORR
| correlation_magnitude
| TIME, Z_DIST, N_BEAM
| Unit of counts
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Platform motion variables'''<math>\dagger</math>
|-
| HEADING
| platform_yaw_angle
| TIME
| degrees, clockwise from true North
|-
| PITCH
| platform_pitch_angle
| TIME
| degrees
|-
| ROLL
| platform_roll_angle
| TIME
| degrees
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables'''
|-
| BURST_NUMBER
| unique_identifier_for_each_burst
| TIME
| Integers of 1, 2, etc to designate which burst the velocities are associated with. Can omit if continuously sampled.
|-
| colspan="4" | <math>\ddagger</math> Quality-control variable names are consistent for those onboard RDI Teleydyne ADCPs. Nortek has CORRN (noise correlation as a % instead of counts) detailed in the [[Level_1_data_(velocity_point-measurements)#Variables|ADV Level 1 variables]].

<math>\dagger</math> Motion variables are necessary for structure function analysis for situating the bins relative the vertical (gravity). These variables do enable converting measurements into geographical.
|}

==Optional ancillary variables==

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| PRES
| sea_water_pressure
| TIME
| dbar, equals 0 at the sea surface and positive down.
|-
| TEMP*
| sea_water_temperature
| TIME
| degrees_Celsius, in-situ temperature ITS-90 scale
|-
| colspan="4" | *Data from concurrent sensors may be optionally included e.g., salinity, dissolved oxygen. Kinematic viscosity of seawater needs to be calculated during processing,
|}

</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">
<br>
This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.
{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the ATOMIX Level 1 content.
| <blockquote>''This group includes the raw measurements from the recorder and ancillary measurements required for quality-controlling them using the manufacturer's recommendations.''</blockquote>
|-
| comment (optional)
| Information is pertinent to problems in the raw data files during collection.
| Examples: Stitching of files, corruption of binary files that were recovered by the manufacturer, etc.
|}
</div>

----
Return to [[Dataset requirements for ADCP structure function]]

Go to [[Level 2 data (velocity profilers)| Level 2 segmented]]

Talk:Level 3 data (velocity profilers)

2022-01-18T09:26:43Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:11, 30 December 2021 (CET) re. DLL_FLAGS - we have both the calculated DLL and the associated qaqc flags at the same level. It makes me wonder whether it would be better to have the R_VEL_FLAGS at level 1. They are currently at level 2 and therefore require either that the level 1 data is duplicated at level 2 or means they sit separate from the associated data. Why not have level 1 being the raw data with qaqc flags and level 2 being the data rearranged into segments (which may be the original bursts) with appropriate pre-processing (detrending) ready for the DLL calculation? It would seem cleaner and more consistent.

This would also allow for the possibility of separate qaqc flags to be defined at level 2 e.g. outlier detection based on the segmented data.

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:39, 17 January 2022 (CET) The flags were changed.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 16:47, 29 December 2021 (CET)
re. TIME dimension comments - the requirement to define time bounds for each segment looks rather complex and I’m not sure that it adds anything. Presumably the requirement to specify bounds will not be mandatory?

Having introduced N_SEGMENT as a dimension at level 2, with TIME as a variable, we are now reverting to TIME as the dimension with N_SEGMENT as the variable. Given that TIME is now derived as the mean time for the observations in the segment, wouldn’t it be more appropriate to keep it as the variable?

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:39, 17 January 2022 (CET) Time bnds is based on CF-compliant dataset. Personally, TIME should be always a dimension as per CF standards. Not a dimension. It was changed at LEvl 2 because people preferred not calculating TIME at Level 2 (violating CF-standards), but NETCDF guis can handle quite nicely time (centered) variables.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:01, 30 December 2021 (CET)
re. R_DEL / R_DEL5 dimension comments - R_DEL should be calculated as a function of R_DIST, which itself is a function of bin size and theta, but having defined R_DIST, it should now be the basis on which R_DEL is calculated. So for example (assuming Matlab indexing), for a central difference scheme evaluated at bin 10 i.e. R_DIST(10), the two-bin separation R_DEL(2) = R_DIST(11) - R_DIST(9), whereas for a forward difference scheme evaluated at bin 10, R_DEL(2) would be R_DIST(12) - R_DIST(10). The R_DEL(2) values will be identical, but the principle is that R_DEL is the separation distance distance the velocity observations being compared.

Also note that R_DEL units should be specified as (in meters).

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:00, 30 December 2021 (CET) re. DLL_N comment - suggest reword as “number of instances when the velocity difference is evaluated, maximum is [number of profiles in segment - either max(N_SAMPLE) or possibly segment_length if redefined as number of profiles rather than time duration]"

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:45, 17 January 2022 (CET) The comments in these tables are for the wiki (teaching purposes). The highlights text in pink brings users to the up-to-date attribute tables. The use "number_of_observations" is a standard modifier for CF-standards and should not be changed. It means the number of usable (good) samples in that "chunk" of data i.e., segment.
----

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:23, 17 January 2022 (CET) For levels 1 and 2 we have moved from R_DIST being a dimension to Z_DIST so that data for both angled and vertical beams can be combined in the R_VEL array. This requires that the profile times are the same for all beams, but leaves open the possibility of the vertical bin size being different between the angled and vertical beams - hence the dimension of BIN_SIZE at level 1 is N_BEAM. However, if we allow for that possibility, then we run into problems using R_DEL as a dimension at level 3 - since the values will differ between the beams and, as I understand it, we can’t have dimension R_DEL with dimensions N_BEAM and R_DEL.

If we constrain the flexibility such that the vertical beam data can only be combined with the angled beam data if both the sampling times and the vertical bin size are the same for both, then at level 3 we can define a dimension Z_DEL which the vertical separation distance between bins at which the mean of the squared velocity difference is calculated - this would be the same for both vertical and angled beams. It would have dimension Z_DEL. Then R_DEL would be a derived variable with dimensions N_BEAM and Z_DEL calculated as a function of THETA and Z_DEL.

If either the sampling times or the vertical bin size differed between the angled and vertical beams, the user would have to prepare separate data files for the two.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 10:26, 18 January 2022 (CET) Alternatively, we could specify the “R_DEL" dimension in “bin separations” i.e. integer values - although this might be better named as N_SEPARATION or something. R_DEL would then be a derived variable calculated from BIN_SIZE (dimension N_BEAM), THETA (dimension N_BEAM) and N_SEPARATION (dimension N_SEPARATION). Whilst this would allow for differences in BIN_SIZE, it would still be problematic if there were significant differences in the number of bins between the beams and sampling times still need to be the same for all beams.

Talk:Level 3 data (velocity profilers)

2022-01-17T14:23:16Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:11, 30 December 2021 (CET) re. DLL_FLAGS - we have both the calculated DLL and the associated qaqc flags at the same level. It makes me wonder whether it would be better to have the R_VEL_FLAGS at level 1. They are currently at level 2 and therefore require either that the level 1 data is duplicated at level 2 or means they sit separate from the associated data. Why not have level 1 being the raw data with qaqc flags and level 2 being the data rearranged into segments (which may be the original bursts) with appropriate pre-processing (detrending) ready for the DLL calculation? It would seem cleaner and more consistent.

This would also allow for the possibility of separate qaqc flags to be defined at level 2 e.g. outlier detection based on the segmented data.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 16:47, 29 December 2021 (CET)
re. TIME dimension comments - the requirement to define time bounds for each segment looks rather complex and I’m not sure that it adds anything. Presumably the requirement to specify bounds will not be mandatory?

Having introduced N_SEGMENT as a dimension at level 2, with TIME as a variable, we are now reverting to TIME as the dimension with N_SEGMENT as the variable. Given that TIME is now derived as the mean time for the observations in the segment, wouldn’t it be more appropriate to keep it as the variable?

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:01, 30 December 2021 (CET)
re. R_DEL / R_DEL5 dimension comments - R_DEL should be calculated as a function of R_DIST, which itself is a function of bin size and theta, but having defined R_DIST, it should now be the basis on which R_DEL is calculated. So for example (assuming Matlab indexing), for a central difference scheme evaluated at bin 10 i.e. R_DIST(10), the two-bin separation R_DEL(2) = R_DIST(11) - R_DIST(9), whereas for a forward difference scheme evaluated at bin 10, R_DEL(2) would be R_DIST(12) - R_DIST(10). The R_DEL(2) values will be identical, but the principle is that R_DEL is the separation distance distance the velocity observations being compared.

Also note that R_DEL units should be specified as (in meters).

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:00, 30 December 2021 (CET) re. DLL_N comment - suggest reword as “number of instances when the velocity difference is evaluated, maximum is [number of profiles in segment - either max(N_SAMPLE) or possibly segment_length if redefined as number of profiles rather than time duration]"

----

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:23, 17 January 2022 (CET) For levels 1 and 2 we have moved from R_DIST being a dimension to Z_DIST so that data for both angled and vertical beams can be combined in the R_VEL array. This requires that the profile times are the same for all beams, but leaves open the possibility of the vertical bin size being different between the angled and vertical beams - hence the dimension of BIN_SIZE at level 1 is N_BEAM. However, if we allow for that possibility, then we run into problems using R_DEL as a dimension at level 3 - since the values will differ between the beams and, as I understand it, we can’t have dimension R_DEL with dimensions N_BEAM and R_DEL.

If we constrain the flexibility such that the vertical beam data can only be combined with the angled beam data if both the sampling times and the vertical bin size are the same for both, then at level 3 we can define a dimension Z_DEL which the vertical separation distance between bins at which the mean of the squared velocity difference is calculated - this would be the same for both vertical and angled beams. It would have dimension Z_DEL. Then R_DEL would be a derived variable with dimensions N_BEAM and Z_DEL calculated as a function of THETA and Z_DEL.

If either the sampling times or the vertical bin size differed between the angled and vertical beams, the user would have to prepare separate data files for the two.

Talk:Level 2 data (velocity profilers)

2022-01-17T13:59:59Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:57, 29 December 2021 (CET) I don’t think the definitions of the two level 2 levels work. With the current definitions, the detrended velocity is a variable under the optional segmented data level. But in practice the data is always segmented and (almost always) detrended, it is just that the “segment" is often the “burst”, which is why I think having both complicates matters.

I think we want to achieve two things. Firstly, we want to have a record of the qaqc of the level 1 velocity data. Secondly, we want to see the data structured and pre-processed as the input to the DLL calculation.

At level 1 we have dimensions TIME, R_DIST and N_BEAM with the key variable R_VEL and I’m proposing the index N_PROF (dimension TIME). The qaqc data is then R_VEL_FLAGS (dimensions TIME, R_DIST, N_BEAM). I’m not sure it is necessary to replicate the level 1 R_VEL variable itself at level 2 - surely that just makes the large data files even larger?

The dimensions for the segmented data are now N_SEGMENT (integer count of segment number), N_SAMPLE (integer count of the profiles used in each segment), plus the existing R_DIST and N_BEAM. I would suggest that we use the variable N_PROF with dimension (N_SEGMENT, N_SAMPLE) to show the unique number of the profiles as used in the segmented data. I’m not sure whether we should be using the same variable name, but the principle is that you could select any segment and read the profile numbers used for each sample and be able to relate this directly to the level 1 R_VEL with the level 2 R_VEL_FLAGS qaqc criteria applied.

If the data was originally collected in bursts of 300 profiles, then N_PROF would simply increment 1 to 300 for segment 1, 301 to 600 for segment 2 etc. If instead, the data was collected continuously and the chosen segment length was 300 with a 50% overlap, N_PROF would be 1 to 300 for segment 1, 151 to 450 for segment 2, 301 to 600 for segment 3 etc.

If we weren’t duplicating the level 1 R_VEL, we could define the level 2 R_VEL as having dimensions (N_SEGMENT, N_SAMPLE, R_DIST, N_BEAM) with the qaqc flags and any detrending / preprocessing applied.

TIME would now be a variable either with dimensions (N_SEGMENT, N_SAMPLE) containing the individual profile timestamps (effectively replicating N_PROF) as suggested or it could be the segment mean time with dimension (N_SEGMENT).

I think this approach simplifies things as well as providing an audit trail

----
Responses by [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:24, 9 January 2022 (CET)

* Lots to ponder but I agree that Level2_qaqc could be removed entirely. Level1 should store velocities (R_VEL in its rawest form i.e., no data removed), along with the R_VEL_FLAGS. A user can then grab R_VEL and R_VEL_FLAGS, find all R_VEL_FLAGS>0 (since 0=good) and set those indices to NaN. The R_VEL_FLAGS isn't velocities but numbers b/w 0-255 denoting an 8-bit boolean flag. It;s very tedious to re-write the data 2x, and people will want to compare the flags (e.g., low corr) with perhaps plotting the correlation. Forcing people to load data b/w groups should be avoided.

* Dimensions
** I would remove R_DIST from Level 1-2 and replace it with Z_DIST for Level1-Level2. R_DIST should appear as a calculated variable using THETA=[NBEAMS array] and Z_DIST at Level3 where it's used (R_DIST would be a matrix of N_beams* Z_dist). This change enables storing all 5 beams in the same matrix and avoids the very complex (very tedious/difficult) of writing a NETCDF file where each variable will need to be manually assigned the dimensions since there will be CORRV, ABSCV, R_VELV, R_VEL_FLAGSV that will have ever so slightly different dimensions than CORR, ABSC etc when all beams have the same Z reference (z being the instrument's z). Dimension variables for CF-compliant dictate that time, lat, long and z (or geographical) should be given priority.

* N_PROF - I would call it N_PROFILE but yes agree with what you've proposed.

----
[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 11:57, 17 January 2022 (CET) Alternative text for comments field:

* N_SEGMENT array of 1 to number of data segments for analysis
* N_SAMPLE array of 1 to number of samples in each segment

Also:

* all instances of R_DIST dimension should now be Z_DIST
* BIN_SIZE should have dimension N_BEAM as per level 1
* not sure we need R_VEL repeated at this level - surely the R_VEL_DETRENDED is sufficient?
* shouldn’t we have R_VEL_DETRENDED_FLAGS for any qaqc checks applied at this stage?
* suggest TIME has dimension N_SEGMENT and is the mean of the level 1 TIME values for the individual profiles - the information detailing which profiles are included in each segment being provided by PROFILE_NUMBER (dimensions N_SEGMENT and N_SAMPLE) - this will then be the TIME dimension for levels 3 and 4 (although I’m unsure whether the bounds attribute can be set for a variable).

Talk:Level 2 data (velocity profilers)

2022-01-17T11:04:31Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:57, 29 December 2021 (CET) I don’t think the definitions of the two level 2 levels work. With the current definitions, the detrended velocity is a variable under the optional segmented data level. But in practice the data is always segmented and (almost always) detrended, it is just that the “segment" is often the “burst”, which is why I think having both complicates matters.

I think we want to achieve two things. Firstly, we want to have a record of the qaqc of the level 1 velocity data. Secondly, we want to see the data structured and pre-processed as the input to the DLL calculation.

At level 1 we have dimensions TIME, R_DIST and N_BEAM with the key variable R_VEL and I’m proposing the index N_PROF (dimension TIME). The qaqc data is then R_VEL_FLAGS (dimensions TIME, R_DIST, N_BEAM). I’m not sure it is necessary to replicate the level 1 R_VEL variable itself at level 2 - surely that just makes the large data files even larger?

The dimensions for the segmented data are now N_SEGMENT (integer count of segment number), N_SAMPLE (integer count of the profiles used in each segment), plus the existing R_DIST and N_BEAM. I would suggest that we use the variable N_PROF with dimension (N_SEGMENT, N_SAMPLE) to show the unique number of the profiles as used in the segmented data. I’m not sure whether we should be using the same variable name, but the principle is that you could select any segment and read the profile numbers used for each sample and be able to relate this directly to the level 1 R_VEL with the level 2 R_VEL_FLAGS qaqc criteria applied.

If the data was originally collected in bursts of 300 profiles, then N_PROF would simply increment 1 to 300 for segment 1, 301 to 600 for segment 2 etc. If instead, the data was collected continuously and the chosen segment length was 300 with a 50% overlap, N_PROF would be 1 to 300 for segment 1, 151 to 450 for segment 2, 301 to 600 for segment 3 etc.

If we weren’t duplicating the level 1 R_VEL, we could define the level 2 R_VEL as having dimensions (N_SEGMENT, N_SAMPLE, R_DIST, N_BEAM) with the qaqc flags and any detrending / preprocessing applied.

TIME would now be a variable either with dimensions (N_SEGMENT, N_SAMPLE) containing the individual profile timestamps (effectively replicating N_PROF) as suggested or it could be the segment mean time with dimension (N_SEGMENT).

I think this approach simplifies things as well as providing an audit trail

----
Responses by [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:24, 9 January 2022 (CET)

* Lots to ponder but I agree that Level2_qaqc could be removed entirely. Level1 should store velocities (R_VEL in its rawest form i.e., no data removed), along with the R_VEL_FLAGS. A user can then grab R_VEL and R_VEL_FLAGS, find all R_VEL_FLAGS>0 (since 0=good) and set those indices to NaN. The R_VEL_FLAGS isn't velocities but numbers b/w 0-255 denoting an 8-bit boolean flag. It;s very tedious to re-write the data 2x, and people will want to compare the flags (e.g., low corr) with perhaps plotting the correlation. Forcing people to load data b/w groups should be avoided.

* Dimensions
** I would remove R_DIST from Level 1-2 and replace it with Z_DIST for Level1-Level2. R_DIST should appear as a calculated variable using THETA=[NBEAMS array] and Z_DIST at Level3 where it's used (R_DIST would be a matrix of N_beams* Z_dist). This change enables storing all 5 beams in the same matrix and avoids the very complex (very tedious/difficult) of writing a NETCDF file where each variable will need to be manually assigned the dimensions since there will be CORRV, ABSCV, R_VELV, R_VEL_FLAGSV that will have ever so slightly different dimensions than CORR, ABSC etc when all beams have the same Z reference (z being the instrument's z). Dimension variables for CF-compliant dictate that time, lat, long and z (or geographical) should be given priority.

* N_PROF - I would call it N_PROFILE but yes agree with what you've proposed.

----
[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 11:57, 17 January 2022 (CET) Alternative text for comments field:

* N_SEGMENT array of 1 to number of data segments for analysis
* N_SAMPLE array of 1 to number of samples in each segment

Also:

* all instances of R_DIST dimension should now be Z_DIST
* not sure we need R_VEL repeated at this level - surely the R_VEL_DETRENDED is sufficient?
* shouldn’t we have R_VEL_DETRENDED_FLAGS for any qaqc checks applied at this stage?
* suggest TIME has dimension N_SEGMENT and is the mean of the level 1 TIME values for the individual profiles - the information detailing which profiles are included in each segment being provided by PROFILE_NUMBER (dimensions N_SEGMENT and N_SAMPLE)

Talk:Level 2 data (velocity profilers)

2022-01-17T10:57:09Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:57, 29 December 2021 (CET) I don’t think the definitions of the two level 2 levels work. With the current definitions, the detrended velocity is a variable under the optional segmented data level. But in practice the data is always segmented and (almost always) detrended, it is just that the “segment" is often the “burst”, which is why I think having both complicates matters.

I think we want to achieve two things. Firstly, we want to have a record of the qaqc of the level 1 velocity data. Secondly, we want to see the data structured and pre-processed as the input to the DLL calculation.

At level 1 we have dimensions TIME, R_DIST and N_BEAM with the key variable R_VEL and I’m proposing the index N_PROF (dimension TIME). The qaqc data is then R_VEL_FLAGS (dimensions TIME, R_DIST, N_BEAM). I’m not sure it is necessary to replicate the level 1 R_VEL variable itself at level 2 - surely that just makes the large data files even larger?

The dimensions for the segmented data are now N_SEGMENT (integer count of segment number), N_SAMPLE (integer count of the profiles used in each segment), plus the existing R_DIST and N_BEAM. I would suggest that we use the variable N_PROF with dimension (N_SEGMENT, N_SAMPLE) to show the unique number of the profiles as used in the segmented data. I’m not sure whether we should be using the same variable name, but the principle is that you could select any segment and read the profile numbers used for each sample and be able to relate this directly to the level 1 R_VEL with the level 2 R_VEL_FLAGS qaqc criteria applied.

If the data was originally collected in bursts of 300 profiles, then N_PROF would simply increment 1 to 300 for segment 1, 301 to 600 for segment 2 etc. If instead, the data was collected continuously and the chosen segment length was 300 with a 50% overlap, N_PROF would be 1 to 300 for segment 1, 151 to 450 for segment 2, 301 to 600 for segment 3 etc.

If we weren’t duplicating the level 1 R_VEL, we could define the level 2 R_VEL as having dimensions (N_SEGMENT, N_SAMPLE, R_DIST, N_BEAM) with the qaqc flags and any detrending / preprocessing applied.

TIME would now be a variable either with dimensions (N_SEGMENT, N_SAMPLE) containing the individual profile timestamps (effectively replicating N_PROF) as suggested or it could be the segment mean time with dimension (N_SEGMENT).

I think this approach simplifies things as well as providing an audit trail

----
Responses by [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:24, 9 January 2022 (CET)

* Lots to ponder but I agree that Level2_qaqc could be removed entirely. Level1 should store velocities (R_VEL in its rawest form i.e., no data removed), along with the R_VEL_FLAGS. A user can then grab R_VEL and R_VEL_FLAGS, find all R_VEL_FLAGS>0 (since 0=good) and set those indices to NaN. The R_VEL_FLAGS isn't velocities but numbers b/w 0-255 denoting an 8-bit boolean flag. It;s very tedious to re-write the data 2x, and people will want to compare the flags (e.g., low corr) with perhaps plotting the correlation. Forcing people to load data b/w groups should be avoided.

* Dimensions
** I would remove R_DIST from Level 1-2 and replace it with Z_DIST for Level1-Level2. R_DIST should appear as a calculated variable using THETA=[NBEAMS array] and Z_DIST at Level3 where it's used (R_DIST would be a matrix of N_beams* Z_dist). This change enables storing all 5 beams in the same matrix and avoids the very complex (very tedious/difficult) of writing a NETCDF file where each variable will need to be manually assigned the dimensions since there will be CORRV, ABSCV, R_VELV, R_VEL_FLAGSV that will have ever so slightly different dimensions than CORR, ABSC etc when all beams have the same Z reference (z being the instrument's z). Dimension variables for CF-compliant dictate that time, lat, long and z (or geographical) should be given priority.

* N_PROF - I would call it N_PROFILE but yes agree with what you've proposed.

----
[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 11:57, 17 January 2022 (CET) Alternative text for comments field:

* N_SEGMENT array of 1 to number of data segments for analysis
* N_SAMPLE array of 1 to number of samples in each segment

Also all instances of R_DIST dimension should now be Z_DIST

Not sure we need R_VEL repeated at this level - surely the R_VEL_DETRENDED is sufficient?

And shouldn’t we have R_VEL_DETRENDED_FLAGS for any qaqc checks applied at this stage?

Talk:Level 1 data (velocity profilers)

2022-01-17T10:34:05Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:16, 28 December 2021 (CET) I don’t think the definitions work properly for instruments with a vertical beam. We are inconsistent in separating the “regular” and “vertical” beams e.g. dimension N_BEAM is defined as including the vertical beam, but is used as a dimension for R_VEL which doesn’t include the vertical beam velocity data.

There is also the issue identified in the notes that some instruments allow the vertical beam to be defined with a different sampling rate / operating mode / ensemble averaging / burst interval etc and have a different bin size, requiring a suggested TIME5 dimension as well as separate variables wherever TIME is a dimension. I suggest it would be cleaner to always treat the angled and vertical beams separately.

Should we also allow for the possibility of instruments with three angled beams and a vertical beam? If so, the use of the tag “5" on the variable names becomes somewhat inappropriate. I suggest we use the tag “V" for vertical (which is also Roman 5).

The mandatory dimensions would remain as TIME, R_DIST and N_BEAM but these only relate to the angled beams (so N_BEAM would be either [1 2 3] or [1 2 3 4]). The variables R_VEL, ABSI and CORR use all of these dimensions, whilst BURST_NUMBER uses TIME. THETA and BIN_SIZE are then constants (not sure why THETA is currently shown with dimension TIME - I think that’s an error).

We then have optional dimensions TIMEV and R_DISTV and optional variables R_VELV, ABSIV and CORRV with both of these dimensions; BURST_NUMBERV with dimension TIMEV; and BIN_SIZEV is a constant. If the sampling for the vertical beam is the same as for the angled beams, then TIMEV and TIME are simply the same - although I suspect this would be the exception.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 12:45, 29 December 2021 (CET) The more I think about it, the less convinced I become that BURST_NUMBER is actually helpful - I’m certainly not convinced it should be a mandatory variable. It isn’t part of the raw data output from TRDI instruments (not sure about Nortek) and it serves no purpose if the data is collected on a continuous basis. Plus it implicit in the TIME dimension values anyway.

More fundamentally, BURST_NUMBER replicates what N_SEGMENT does and I don’t think we need the added complexity of having both.

What I think would be helpful would be to have a variable that is an integer count simply identifying each velocity profile instance (consisting of number of beams x number of bins velocity values) at level 1, this variable (PROFILE_NUMBER or N_PROF or something - I’m unclear on the naming convention) would have the dimension TIME and would simply increment 1, 2, 3 etc. but (with the N_BEAM and R_DIST) provides a unique identifier for each velocity value and as an integer is much easier to work with (e.g. for indexing) than the float TIME. With an equivalent for the vertical beam where appropriate.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:49, 28 December 2021 (CET) Suggested units of ABSI are db. For the TRDI instruments the WA configuration setting for the fish rejection algorithm (which is likely to be familiar to any user configuring an instrument for deployment) is based on the echo intensity “count” returned by the instrument not db. So if users are implementing the fish rejection algorithm as part of their QC, I would expect they will use a count difference threshold rather than a db difference. The conversion between counts and db is very simple (1 count = 0.5 db), unless users want to get into distance compensation, but requiring the conversion to db may be unhelpful. Perhaps we should allow the user to specify either count or db?

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 14:00, 28 December 2021 (CET) Optional ancillary variables - what is dimension N_VEL_INSTRUMENT shown for variable PRES? Re note on TEMP - Kinematic viscosity is not required for the SF calculation.

[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 14:17, 7 January 2022 (CET) I agree that BURST_NUMBER shouldn't be required. It is useful for continuous data in the sense that it distinguishes the ensemble that the data are part of, but I think this could be better captured at level 2 in the segmented data. I also agree that we shouldn't limit ourselves to calling the vertical beam #5, and like Brian's suggestion of using a "V" instead.

----
[[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 02:03, 15 January 2022 (CET) Responses:
* The units for ABSI and CORR are different from ABSIC (counts) and CORRN (%). They should be in the native unit, not converted.
* RV was changed according TEAMS discussion.
* Burst_number was is not the same as N_SEGMENT. Nortek instrument spits data file into many "bursts" and it's useful to be able to refer back to the rawest dat files. I will let the user decide to add if desired.

----
[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 11:17, 17 January 2022 (CET) Dimensions for required variable PROFILE_NUMBER should just be TIME whilst the R_DIST dimension for R_VEL, ABSIC and CORR should now be Z_DIST.

I think we also agreed that the level 1 qaqc R_VEL_FLAGS should also be included as a required variable?

Talk:Level 1 data (velocity profilers)

2022-01-17T10:30:50Z

Brian scannell:

Talk:Level 1 data (velocity profilers)

2022-01-17T10:17:21Z

Brian scannell:

Talk:Level 4 data (velocity profilers)

2022-01-07T14:12:06Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:07, 7 January 2022 (CET) suggested Comments:<br>
R_MAX - maximum R_DEL separation distance used for the regression, units is meters<br>
REGRESSION_COEFFICIENT_A0 - intercept from the regression of DLL against R_DEL, indicative of noise, units is meter**2 / second**2<br>
REGRESSION_COEFFICIENT_A1 - regression coefficient from the regression of DLL against R_DEL used in the calculation of <math>\varepsilon</math>, units is meter**(4/3) / second**2<br>
REGRESSION_R2 - goodness of fit parameter for the regression (user to include method description in metadata)<br>
REGRESSION_N - number of data points used in the regression

Talk:Level 4 data (velocity profilers)

2022-01-07T14:07:22Z

Brian scannell: Created page with "~~~~ suggested Comments:<br> R_MAX - maximum R_DEL separation distance used for the regression, units is meters<br> REGRESSION_COEFFICIENT_A0 - intercept from the regression o..."

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:07, 7 January 2022 (CET) suggested Comments:<br>
R_MAX - maximum R_DEL separation distance used for the regression, units is meters<br>
REGRESSION_COEFFICIENT_A0 - intercept from the regression of DLL against R_DEL, indicative of noise, units is meter**2 / second**2<br>
REGRESSION_COEFFICIENT_A1 - regression coefficient from the regression of DLL against R_DEL used in the calculation of <math>\varepsilon</math>, units is meter**(4/3) / second**2

Velocity profilers

2022-01-07T10:26:17Z

Brian scannell: Undo revision 3532 by Brian scannell (talk)

== Welcome to the ADCP structure function group ==

The [[Acoustic-Doppler Current Profilers]] (ADCP) structure function group addresses best practices in obtaining [[turbulent kinetic energy dissipation]] rate estimates from ADCPs using a platform independent approach. Our recommendations are applicable for measurements from bed-mounted ADCPs or ADCPs attached to mooring lines.

== Scope ==
We will provide
# [[ADCP structure function ε estimates flow chart]]
# Quality control measures for both pre- and post-processed data.
# [[Dataset requirements for ADCP structure function]]
# [[Benchmark datasets for ADCP structure function]]

Structure function-derived estimates of turbulence parameters from lowered ADCPs mounted on CTD rosettes and other profiling or drifting platforms are beyond the scope of this working group.

Our recommendations are intended to be applicable to measurements irrespective of the manufacturer provided the data is of sufficient quality for resolving the turbulence inertial subrange of the velocity spectra.

The benchmark datasets are intended to be a resource that can be used by the community to evaluate routines in any programming language.

[[Category:Velocity profilers]]

Velocity profilers

2021-12-31T14:17:33Z

Brian scannell:

== Welcome to the ADCP structure function group ==

The [[Acoustic-Doppler Current Profilers]] (ADCP) structure function group addresses best practices in obtaining [[turbulent kinetic energy dissipation]] rate estimates from ADCPs using a platform independent approach. Our recommendations are applicable for measurements from bed-mounted ADCPs or ADCPs attached to mooring lines.

== Scope ==
We will provide
# [[ADCP structure function <math>\varepsilon</math> estimates flow chart]]
# Quality control measures for both pre- and post-processed data.
# [[Dataset requirements for ADCP structure function]]
# [[Benchmark datasets for ADCP structure function]]

Structure function-derived estimates of turbulence parameters from lowered ADCPs mounted on CTD rosettes and other profiling or drifting platforms are beyond the scope of this working group.

Our recommendations are intended to be applicable to measurements irrespective of the manufacturer provided the data is of sufficient quality for resolving the turbulence inertial subrange of the velocity spectra.

The benchmark datasets are intended to be a resource that can be used by the community to evaluate routines in any programming language.

[[Category:Velocity profilers]]

Talk:Level 3 data (velocity profilers)

2021-12-30T14:11:20Z

Brian scannell:

Talk:Level 3 data (velocity profilers)

2021-12-30T14:00:41Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 16:47, 29 December 2021 (CET)
re. TIME dimension comments - the requirement to define time bounds for each segment looks rather complex and I’m not sure that it adds anything. Presumably the requirement to specify bounds will not be mandatory?

Having introduced N_SEGMENT as a dimension at level 2, with TIME as a variable, we are now reverting to TIME as the dimension with N_SEGMENT as the variable. Given that TIME is now derived as the mean time for the observations in the segment, wouldn’t it be more appropriate to keep it as the variable?

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:01, 30 December 2021 (CET)
re. R_DEL / R_DEL5 dimension comments - R_DEL should be calculated as a function of R_DIST, which itself is a function of bin size and theta, but having defined R_DIST, it should now be the basis on which R_DEL is calculated. So for example (assuming Matlab indexing), for a central difference scheme evaluated at bin 10 i.e. R_DIST(10), the two-bin separation R_DEL(2) = R_DIST(11) - R_DIST(9), whereas for a forward difference scheme evaluated at bin 10, R_DEL(2) would be R_DIST(12) - R_DIST(10). The R_DEL(2) values will be identical, but the principle is that R_DEL is the separation distance distance the velocity observations being compared.

Also note that R_DEL units should be specified as (in meters).

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:00, 30 December 2021 (CET) re. DLL_N comment - suggest reword as “number of instances when the velocity difference is evaluated, maximum is [number of profiles in segment - either max(N_SAMPLE) or possibly segment_length if redefined as number of profiles rather than time duration]"

Talk:Level 3 data (velocity profilers)

2021-12-30T13:51:14Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 16:47, 29 December 2021 (CET) re. TIME dimension comments - the requirement to define time bounds for each segment looks rather complex and I’m not sure that it adds anything. Presumably the requirement to specify bounds will not be mandatory?

Having introduced N_SEGMENT as a dimension at level 2, with TIME as a variable, we are now reverting to TIME as the dimension with N_SEGMENT as the variable. Given that TIME is now derived as the mean time for the observations in the segment, wouldn’t it be more appropriate to keep it as the variable?

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:01, 30 December 2021 (CET) re. R_DEL / R_DEL5 dimension comments - R_DEL should be calculated as a function of R_DIST, which itself is a function of bin size and theta, but having defined R_DIST, it should now be the basis on which R_DEL is calculated. So for example (assuming Matlab indexing), for a central difference scheme evaluated at bin 10 i.e. R_DIST(10), the two-bin separation R_DEL(2) = R_DIST(11) - R_DIST(9), whereas for a forward difference scheme evaluated at bin 10, R_DEL(2) would be R_DIST(12) - R_DIST(10). The R_DEL(2) values will be identical, but the principle is that R_DEL is the separation distance distance the velocity observations being compared.

Also note that R_DEL units should be specified as (in meters).

Talk:Level 3 data (velocity profilers)

2021-12-30T12:04:08Z

Brian scannell:

Talk:Level 3 data (velocity profilers)

2021-12-30T12:01:57Z

Brian scannell:

Talk:Level 3 data (velocity profilers)

2021-12-29T15:47:46Z

Brian scannell: Created page with "~~~~ re the TIME dimension - the requirement to define time bounds for each segment looks rather complex and I’m not sure that it adds anything. Presumably the requirement..."

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 16:47, 29 December 2021 (CET) re the TIME dimension - the requirement to define time bounds for each segment looks rather complex and I’m not sure that it adds anything. Presumably the requirement to specify bounds will not be mandatory?

Talk:Level 2 segmented (velocity profilers)

2021-12-29T13:38:28Z

Brian scannell: Created page with "~~~~ see discussion of level 2 variables at https://wiki.uib.no/atomix/index.php/Talk:Level_2_data_(velocity_profilers)"

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 14:38, 29 December 2021 (CET) see discussion of level 2 variables at https://wiki.uib.no/atomix/index.php/Talk:Level_2_data_(velocity_profilers)

Talk:Level 2 qaqc velocities (velocity profilers)

2021-12-29T13:37:41Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 14:37, 29 December 2021 (CET) see discussion of level 2 variables at https://wiki.uib.no/atomix/index.php/Talk:Level_2_data_(velocity_profilers)

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:58, 28 December 2021 (CET) As per discussion re level 1 - suggest dimensions are TIME, R_DIST and N_BEAM for the standard angled beams with the option of TIMEV and R_DISTV for the vertical beam if required.

THETA and BIN_SIZE are then constants, being the same for all the angled beams, with no TIME dependence for THETA.

R_VEL_FLAGS comments - suggest modify "Boolean flags (8 bit, 0-255) to represent one of 8 possible reasons for rejection” to "Boolean flags (8 bit, 0-255) to represent any combination of up to 8 possible reasons for rejection”.

Talk:Level 2 data (velocity profilers)

2021-12-29T12:57:01Z

Brian scannell:

Talk:Level 1 data (velocity profilers)

2021-12-29T11:45:13Z

Brian scannell:

Talk:Level 2 data (velocity profilers)

2021-12-28T15:01:18Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:51, 28 December 2021 (CET) The description of the segmented data as being detrended implies a distinction from the standard level 2 data. The “standard” level 2 data should also be detrended - but across the burst rather than a segment. Makes me wonder whether having both burst and segment adds anything other than possible confusion? Perhaps omit BURST_NUMBER at level 1 and introduce SEGMENT_NUMBER as a required variable at level 2? Having SAMPLE_NUMBER (or possibly PROFILE_NUMBER) as a level 1 variable (dimension TIME) replicates TIME as a unique identifier for each set of observations (dimensions R_DIST, N_BEAM) but as an integer variable it would be easier to work with when defining bursts / segments.

Talk:Level 2 qaqc velocities (velocity profilers)

2021-12-28T14:58:36Z

Brian scannell: Created page with "~~~~ As per discussion re level 1 - suggest dimensions are TIME, R_DIST and N_BEAM for the standard angled beams with the option of TIMEV and R_DISTV for the vertical beam if..."

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:58, 28 December 2021 (CET) As per discussion re level 1 - suggest dimensions are TIME, R_DIST and N_BEAM for the standard angled beams with the option of TIMEV and R_DISTV for the vertical beam if required.

THETA and BIN_SIZE are then constants, being the same for all the angled beams, with no TIME dependence for THETA.

R_VEL_FLAGS comments - suggest modify "Boolean flags (8 bit, 0-255) to represent one of 8 possible reasons for rejection” to "Boolean flags (8 bit, 0-255) to represent any combination of up to 8 possible reasons for rejection”.

Talk:Level 2 data (velocity profilers)

2021-12-28T14:51:55Z

Brian scannell: Created page with "~~~~ The description of the segmented data as being detrended implies a distinction from the standard level 2 data. The “standard” level 2 data should also be detrended -..."

Talk:Level 1 data (velocity profilers)

2021-12-28T14:39:48Z

Brian scannell:

Talk:Level 1 data (velocity profilers)

2021-12-28T13:00:19Z

Brian scannell:

Talk:Level 1 data (velocity profilers)

2021-12-28T12:56:58Z

Brian scannell:

Talk:Level 1 data (velocity profilers)

2021-12-28T12:49:25Z

Brian scannell:

Talk:Level 1 data (velocity profilers)

2021-12-28T12:16:40Z

Brian scannell:

Talk:Level 1 data (velocity profilers)

2021-12-28T11:32:42Z

Brian scannell:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:36, 22 December 2021 (CET)
I think dimension N_BEAM has to be the beam numbers for the beams with the standard beam angle - so will typically be [1 2 3]or [1 2 3 4]. If we allow it to include beam 5, we run into a problem with the variables such as R_VEL for which N_BEAM is a dimension.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:38, 22 December 2021 (CET)
Are there any designs out there with three angled beams plus a vertical beam? If so, perhaps rather than R_DIST5 etc we should use R_DISTV to indicate vertical - although it also has the advantage of being the Roman 5...

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 12:32, 28 December 2021 (CET) Plus the vertical beam is often configurable with different sampling rates / burst periods etc. which suggests there is a need for a separate variable TIME5 (or following the suggestion above TIMEV).

Talk:Level 1 data (velocity profilers)

2021-12-22T14:38:14Z

Brian scannell:

Talk:Level 1 data (velocity profilers)

2021-12-22T14:36:26Z

Brian scannell: Created page with "~~~~ I think dimension N_BEAM has to be the beam numbers for the beams with the standard beam angle - so typically 3 or 4. If we allow it to include beam 5, we run into a pro..."

Deployment

2021-12-10T11:25:11Z

Brian scannell:

In order to collect useful measurements that '''actually resolve''' the turbulence statistics consistent with the application of the Kolmogorov hypotheses of [[isotropic turbulence]], it is important to configure and deploy your instrument using best practices. In setting up your instrument, please consider the following reccomendations:

==Environmental Conditions ==
<div class="mw-collapsible mw-collapsed" id="env" data-collapsetext="Collapse" data-expandtext="Expand conditions">
<br>
* Ensure measurement velocity range is sufficient for anticipated background flow, tides, surface waves and internal waves
* For pulse-pulse coherent measurements, minimize potential issues due to phase wrapping by setting the ambiguity velocity to be larger than the maximum flow speed that is expected
* Ensure that spatial parameters (number of bins and bin size) are selected so that several bins are within the expected [[velocity inertial subrange model | inertial subrange]] that extends from the Kolmogorov scale [[Nomenclature| <math>L_K</math>]] to the Ozmidov scale [[Nomenclature| <math>L_o</math>]] . Use anticipated stratification and turbulence levels to determine <math>L_K</math> and <math>L_o</math> for the deployment location.
</div>

== Velocity Measurements ==
<div class="mw-collapsible mw-collapsed" id="vel" data-collapsetext="Collapse" data-expandtext="Expand measurements">
<br>
* Record raw data in along-beam coordinates
* Maximise velocity accuracy whilst minimising averaging (pings per ensemble)
* If using a duty cycle, ensure that each burst is long enough to obtain stationary statistics necessary for <math>\varepsilon</math> estimates
* Maximise the number of profiles (ensembles) per <math>\varepsilon</math> estimate observation period to improve statistics
* Avoid/reduce interference with nearby instruments to reduce/avoid interference by sampling at different intervals.
* For instruments with an extra (vertical) beam, select the desired configuration of the angled beams
</div>

== Motion control during deployment ==
<div class="mw-collapsible mw-collapsed" id="motioncontrol" data-collapsetext="Collapse" data-expandtext="Expand motion control">
<br>
* For bottom-mounted instruments, minimise motion by ensuring that the frame is sufficiently heavy and streamlined to withstand the flow conditions at the deployment location
* For moored instruments, minimise motion by ensuring there is sufficient buoyancy on frame to hold position well. Also ensure that the buoyancy components do not obstruct beam path
* For moored instruments, consider impact of knock-down on location of observations in the water column when designing the mooring
* For moored instruments, collect depth and orientation data (heading, pitch and roll) at the same frequency as the velocity profiles data. Depth measurements could be made by the pressure sensor on ADCP or by adjacent instrument. Consider high resolution add-ons such as Altitude and Heading Reference Sensor (AHRS).
</div>

== Power and Storage for self-contained deployments ==
<div class="mw-collapsible mw-collapsed" id="power-storage" data-collapsetext="Collapse" data-expandtext="Expand power and storage">
<br>
* Ensure the battery and memory capacity are sufficient for the planned deployment duration
* Factor in the expected water temperature when estimating the battery capacity and energy consumption
* Factor in the available memory and the manufacturer’s expected memory required per recorded profile (ensemble) when estimating the memory capacity
* For long deployments, extend the interval between observation periods (burst mode). This allows for longer duration deployments but reduces the temporal resolution of <math>\varepsilon</math> estimates
</div>

<br></br>
Next Step: [[ Raw data review (QA1) ]] <br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Talk:Deployment

2021-12-10T11:18:03Z

Brian scannell:

== Suggestions ==

* I think the environmental conditions, i.e. the scales of turbulence and their time/length scales are important. The epsilon, and the inertial subrange dictate how big your r can be, and also the sampling rate/duration of bursts. [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 01:15, 11 November 2021 (CET)

* I agree with Cynthia. I'm transferring Brian's comments here for now, we can build out those pages between now and Warnemund. I also addressed JMM's comment about defining AHRS [[User:Yuengdjern|Yuengdjern]] ([[User talk:Yuengdjern|talk]]) 14:54, 15 November 2021 (CET)

* To do list for Ilker - from Brian: SUGGEST REMOVING BOTH OF THE FOLLOWING PAGES'''
# [[ADCP Hardware]]
# [[ADCP Environment]]
[[User:Yuengdjern|Yuengdjern]] ([[User talk:Yuengdjern|talk]]) 15:04, 15 November 2021 (CET)

* Other unresolved Comments:
# (JMM) IS THERE A TYPICAL PINGS PER ENSEMBLE WE CAN RECOMMEND? (i.e. 1 or 2 are all I used at a high flow site)
# [(JMM) I REWORDED THIS FROM '<math>\varepsilon</math> estimate observation period', but I don't know if it is any less confisuing]
# Re: number of profiles: IS THIS ONLY DETERMINED BY THE SAMPLE RATE?]
# Re bin numbers and sizes: [SHOULD WE QUANTIFY THIS]
# Re nomenclature table: [IS IT POSSIBLE TO LINK TO APPROPRIATE TABLE?]
# Re beam configuration: [(JMM) CAN WE BE MORE SPECIFIC HERE? ]]
[[User:Yuengdjern|Yuengdjern]] ([[User talk:Yuengdjern|talk]]) 15:21, 15 November 2021 (CET)

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 12:18, 10 December 2021 (CET) Under motion control, I suggest including a comment about mounting the ADCP in a streamlined buoy or frame designed to maintain steady orientation to the flow. Open frames can oscillate a lot, spherical buoys are better, but a streamlined buoy should be best. Oscillations over the period of the burst result in the background flow seen by a beam varying over the burst, potentially introducing a bias if the background flow is sheared.