Atomix - User contributions [en]

Level 4 data (velocity profilers)

2022-07-08T22:29:30Z

Jmmcmillan: Added EPSI_DEL_RATIO as a variable since it should be included in datafiles at this stage in the testing

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]).
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profiler data flags | Velocity Profiler data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_DEL_RATIO
| specific_turbulent_kinetic_energy_dissipation_in_sea_water_error_ratio
| TIME, Z_DIST, N_BEAM
| Ratio of the uncertainty, <math>\delta\varepsilon</math> to the value of <math>\varepsilon</math>. Computed from the confidence intervals as <math>\frac{\delta\varepsilon}{\epsilon} = \frac{3\ (\text{EPSI_CI_HIGH}\ -\ \text{EPSI_CI_LOW})}{4\ \text{REGRESSION_COEFF_A0}}</math> {{FontColor|fg=red|text=Link to derivation}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]])
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| regression_model
| model regressed against the data to determine <math>\varepsilon</math>
| Examples include:
standard model, i.e. <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}</math> <br /><br />
modified model, i.e. <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}+a_3((\delta r)^{2/3})^3 </math>
|-
| regression_method
| method used to regress the model against the data
| Examples include:
Least-squares using evaluated separation distances to r_max.
|-
| EPSI_FINAL_method
| method used to calculate EPSI_FINAL from EPSI
| Examples include:
arithmetic mean across beams of resolved EPSI after all EPSI_FLAGS applied.
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-07-08T14:19:25Z

Jmmcmillan: Added EPSI_FINAL_method attribute

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]).
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profiler data flags | Velocity Profiler data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]])
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| regression_model
| model regressed against the data to determine <math>\varepsilon</math>
| Examples include:
standard model, i.e. <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}</math> <br /><br />
modified model, i.e. <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}+a_3((\delta r)^{2/3})^3 </math>
|-
| regression_method
| method used to regress the model against the data
| Examples include:
Least-squares using evaluated separation distances to r_max.
|-
| EPSI_FINAL_method
| method used to calculate EPSI_FINAL from EPSI
| Examples include:
arithmetic mean across beams of resolved EPSI after all EPSI_FLAGS applied.
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-07-08T14:08:25Z

Jmmcmillan: Updated attributes to represent those that are being used in the test datasets

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]).
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profiler data flags | Velocity Profiler data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]])
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| regression_model
| model regressed against the data to determine <math>\varepsilon</math>
| Examples include:
standard model, i.e. <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}</math> <br /><br />
modified model, i.e. <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}+a_3((\delta r)^{2/3})^3 </math>
|-
| regression_method
| method used to regress the model against the data
| Examples include:
Least-squares using evaluated separation distances to r_max.
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 3 data (velocity profilers)

2022-07-08T13:53:52Z

Jmmcmillan: Change dll_calculation type to dll_method for consistency with Brian's data

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
[[File:SF atomix ADCP.png|300px|thumb|Schematic of ADCP processing nomenclature]]
The required dimensions and variables for the structure-function processing level within NetCDF ATOMIX format for velocity ADCP measurements are described below. This NetCDF group contains the structure function (DLL) calculated as a function of the along-beam separation for the available/usable ADCP bins.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Segment midpoint time. Units in Days since reference time specified in variable attribute.
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_DEL
| along-beam_separation_distance_over_
which_DLL_is_evaluated_in_number_of_bins
| N_DEL
| Number of bins separating two velocity measurements used to calculate DLL
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| DLL
| second_order_structure_function
| TIME, Z_DIST, N_BEAM, N_DEL
| Differences in velocities squared have been time-averaged (units of m2/s2).
|-
| DLL_FLAGS
| second_order_structure_function_
status_flag
| TIME, Z_DIST, N_BEAM, N_DEL
| For details see [[Velocity Profiler data flags| Velocity profiler data flags]]
|-
| R_DEL
| along-beam_separation_distance_at_
which_structure_function_is_evaluated
| N_BEAM, N_DEL
| Estimated quantity (in meters) from N_DEL (Level 3), BIN_SIZE (Level 2) and THETA (Level 2).
|-
| R_DIST
| distance_from_sensor_along_beams
| Z_DIST, N_BEAM
| Along-beam bin centre distance (in meters) from the transducer
|-
| DLL_N
| second_order_structure_function_number_
of_observations
| TIME, Z_DIST, N_BEAM, N_DEL
| The number of available measurements in each segment i.e., data quality.
|-
| N_SEGMENT
| unique_identifier_for_each_segment_
in_the_entire_available_timeseries
| TIME
| Enables backtracking to [[Level 2 segmented (velocity profilers)|previous processing level]]
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the structure function Dll as a function of the separation distance. Any ancillary information required for estimating the dissipation of turbulent kinetic energy may also be stored here. ''</blockquote>
|-
| dll_method
| Specify differencing technique used to estimate DLL
| Examples include:
<blockquote>Central-differencing</blockquote>
<blockquote>Forward-differencing</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes <math>\ddagger</math>
|-
| stationarity_testing
| Any testing done on the segment to verify stationarity?
| {{FontColor|fg=white|bg=red|text=To be revisited once testing begins}}. Tentatively refer to [https://bitbucket.org/efm_cb/netcdf/src/master/TestData/adcp_atomix_metada.yml demo yaml] file.
|-
| noise_testing
| Details of testing the noise levels, or if the signal comprises mostly of noise?
|
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 2 data (velocity profilers)| Level 2 segmented velocities]]

Go to [[Level 4 data (velocity profilers)| Level 4 dissipation estimates]]

Level 1 data (velocity profilers)

2022-04-26T19:24:27Z

Jmmcmillan: Add reminder to agree on variable name

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
The required dimensions and variables for the first processing level within NetCDF ATOMIX format for ADCP velocity measurements are described below. This processing level contains the raw measurements recorded by the instrument. If sampling is in [[Burst sampling|burst mode]], the measurements from individual [[Burst sampling|bursts]] are appended together.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
==Required variables==
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comments and units
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| TIME, Z_DIST, N_BEAM
| Units in m/s
|-
| R_VEL_FLAGS
| water_radial_velocity_of_scatterers_
towards_instrument_status_flag
| TIME, Z_DIST, N_BEAM
| Boolean flags (8 bit, 0-255) to represent one of 8 possible reasons for rejection. {{FontColor|fg=white|bg=red|text= Link to own flagging page}}
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| vertical size of the ADCP bins. Usually the same for all 4 beams, but 5th beam can vary.
|-
| PROFILE_NUMBER {{FontColor|fg=white|bg=red|text= or N_PROFILE (agree as group)}}
| unique_identifier_for_each_profile
| TIME
| This variable identifies each record (velocity) profile
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Quality-control variables recorded by acoustic-Doppler instruments''' <math>\ddagger</math>
|-
| ABSIC
| backscatter_intensity
| TIME, Z_DIST, N_BEAM
| Units of counts
|-
| CORR
| correlation_magnitude_from_each_acoustic_beam
| TIME, Z_DIST, N_BEAM
| Unit of counts
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Platform motion variables''' <math>\dagger</math>
|-
| HEADING
| platform_yaw_angle
| TIME
| degrees, clockwise from true North
|-
| PITCH
| platform_pitch_angle
| TIME
| degrees
|-
| ROLL
| platform_roll_angle
| TIME
| degrees
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables''' <math>\dagger\dagger</math>
|-
| BURST_NUMBER
| unique_identifier_for_each_burst
| TIME
| Integers of 1, 2, etc to designate which [[Burst sampling|burst]] the velocities are associated with. Only required when measuring in burst-mode.
|-
| colspan="4" | <math>\ddagger</math> Quality-control variable names are consistent for those onboard RDI Teleydyne ADCPs. Nortek has CORRN (noise correlation as a % instead of counts) detailed in the [[Level_1_data_(velocity_point-measurements)#Variables|ADV Level 1 variables]].

<math>\dagger</math> Motion variables are necessary for structure function analysis for situating the bins relative the vertical (gravity). These variables do enable converting measurements into geographical.

<math>\dagger\dagger</math> Only required when measuring in bursts e.g., 18 min of continuous measurements every hour. Can omit if continuously sampled.
|}

==Optional ancillary variables==

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| PRES
| sea_water_pressure
| TIME
| dbar, equals 0 at the sea surface and positive down.
|-
| TEMP*
| sea_water_temperature
| TIME
| degrees_Celsius, in-situ temperature ITS-90 scale
|-
| colspan="4" | *Data from concurrent sensors may be optionally included e.g., salinity, dissolved oxygen. Kinematic viscosity of seawater needs to be calculated during processing,
|}

</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">
<br>
This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.
{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the ATOMIX Level 1 content.
| <blockquote>''This group includes the raw measurements from the recorder and ancillary measurements required for quality-controlling them using the manufacturer's recommendations.''</blockquote>
|-
| comment (optional)
| Information is pertinent to problems in the raw data files during collection.
| Examples: Stitching of files, corruption of binary files that were recovered by the manufacturer, etc.
|}
</div>

----
Return to [[Dataset requirements for ADCP structure function]]

Go to [[Level 2 data (velocity profilers)| Level 2 segmented]]

Level 2 data (velocity profilers)

2022-04-26T19:23:39Z

Jmmcmillan: Add reminder to agree on variable name

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Array of 1 to number of segments into which the data has been separated (Note TIME variable)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)
|}
</div>

=Variables <math>\ddagger</math>=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| Along beam velocity [m/s] for each of the beams.
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| PROFILE_NUMBER {{FontColor|fg=white|bg=red|text= or N_PROFILE (agree as group)}}
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after restructuring into segments.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Level 4 data (velocity profilers)

2022-04-20T13:58:32Z

Jmmcmillan: Update time description

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. {{FontColor|fg=red|text=Provide link.}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>

|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
???
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 3 data (velocity profilers)

2022-04-20T13:58:01Z

Jmmcmillan:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
[[File:SF atomix ADCP.png|300px|thumb|Schematic of ADCP processing nomenclature]]
The required dimensions and variables for the structure-function processing level within NetCDF ATOMIX format for velocity ADCP measurements are described below. This NetCDF group contains the structure function (DLL) calculated as a function of the along-beam separation for the available/usable ADCP bins.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_DEL
| along-beam_separation_distance_
over_which_DLL_is_evaluated_in_number_of_bins
| N_DEL
| Number of bins separating two velocity measurements used to calculate DLL
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| DLL
| second_order_structure_function
| TIME, Z_DIST, N_DEL, N_BEAM
| Differences in velocities squared have been time-averaged (units of m2/s2).
|-
| DLL_FLAGS
| second_order_structure_function_status_flag
| TIME, Z_DIST, N_DEL, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| R_DEL
| along-beam_separation_distance_
at_which_structure_function_is_evaluated
| N_DEL, Z_DIST, N_BEAM, TIME
| Estimated quantity (in meters) from N_DEL (Level 3), BIN_SIZE (Level 2) and THETA (Level 2).
|-
| R_DIST
| distance_from_sensor_along_beams
| Z_DIST, N_BEAM
| Along-beam bin centre distance (in meters) from the transducer
|-
| DLL_N
| second_order_structure_function_number_of_observations
| TIME, Z_DIST, N_DEL, N_BEAM
| The number of available measurements in each segment i.e., data quality.
|-
| N_SEGMENT
| unique_identifier_for_each_segment_
in_the_entire_available_timeseries
| TIME
| Enables backtracking to [[Level 2 segmented (velocity profilers)|previous processing level]]
|-
| BURST_NUMBER <math>\dagger\dagger</math>
| unique_identifier_for_each_burst
| TIME
| Only required when measuring in burst-mode. Integers of 1, 2, etc to designate which burst the velocities are associated with.
|-
| colspan="4" |
<math>\dagger\dagger</math> Optional variable required only when measuring in bursts e.g., 18 min of continuous measurements every hour.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the structure function Dll as a function of the separation distance. Any ancillary information required for estimating the dissipation of turbulent kinetic energy may also be stored here. ''</blockquote>
|-
| dll_calculation_type
| Specify differencing technique used to estimate DLL
| Examples include:
<blockquote>Central-differencing</blockquote>
<blockquote>Forward-differencing</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes <math>\ddagger</math>
|-
| stationarity_testing
| Any testing done on the segment to verify stationarity?
| {{FontColor|fg=white|bg=red|text=To be revisited once testing begins}}. Tentatively refer to [https://bitbucket.org/efm_cb/netcdf/src/master/TestData/adcp_atomix_metada.yml demo yaml] file.
|-
| noise_testing
| Details of testing the noise levels, or if the signal comprises mostly of noise?
|
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 2 data (velocity profilers)| Level 2 segmented velocities]]

Go to [[Level 4 data (velocity profilers)| Level 4 dissipation estimates]]

Level 2 data (velocity profilers)

2022-04-20T13:56:59Z

Jmmcmillan: Update time description

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_
segment_in_the_entire_available_timeseries
| N_SEGMENT
| Replacement for using TIME centred in the segment as dimension
|-
| N_SAMPLE
| unique_identifier_for_each_
sample_within_the_segment
| N_SAMPLE
| Max sample number is the segment length (seconds) multiplied by the sampling rate
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| N_SEGMENT, Z_DIST,N_SAMPLE, N_BEAM
| Along beam velocity [m/s] for each of the beams.
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_SAMPLE, N_BEAM
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| constant
| Vertical size of the ADCP bins [m]. Usually the same for all 4 diverging beams.
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after segmenting them into smaller chunks.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Level 1 data (velocity profilers)

2022-04-20T13:56:04Z

Jmmcmillan: Update time description

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
The required dimensions and variables for the first processing level within NetCDF ATOMIX format for ADCP velocity measurements are described below. This processing level contains the raw measurements recorded by the instrument. If sampling is in [[Burst sampling|burst mode]], the measurements from individual [[Burst sampling|bursts]] are appended together.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
==Required variables==
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comments and units
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| TIME, Z_DIST, N_BEAM
| Units in m/s
|-
| R_VEL_FLAGS
| water_radial_velocity_of_scatterers_
towards_instrument_status_flag
| TIME, R_DIST, N_BEAM
| Boolean flags (8 bit, 0-255) to represent one of 8 possible reasons for rejection. {{FontColor|fg=white|bg=red|text= Link to own flagging page}}
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| vertical size of the ADCP bins. Usually the same for all 4 beams, but 5th beam can vary.
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Quality-control variables recorded by acoustic-Doppler instruments''' <math>\ddagger</math>
|-
| ABSIC
| backscatter_intensity
| TIME, Z_DIST, N_BEAM
| Units of counts
|-
| CORR
| correlation_magnitude_from_each_acoustic_beam
| TIME, Z_DIST, N_BEAM
| Unit of counts
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Platform motion variables'''<math>\dagger</math>
|-
| HEADING
| platform_yaw_angle
| TIME
| degrees, clockwise from true North
|-
| PITCH
| platform_pitch_angle
| TIME
| degrees
|-
| ROLL
| platform_roll_angle
| TIME
| degrees
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables'''
|-
| BURST_NUMBER
| unique_identifier_for_each_burst
| TIME
| Integers of 1, 2, etc to designate which [[Burst sampling|burst]] the velocities are associated with. Can omit if continuously sampled.
|-
| colspan="4" | <math>\ddagger</math> Quality-control variable names are consistent for those onboard RDI Teleydyne ADCPs. Nortek has CORRN (noise correlation as a % instead of counts) detailed in the [[Level_1_data_(velocity_point-measurements)#Variables|ADV Level 1 variables]].

<math>\dagger</math> Motion variables are necessary for structure function analysis for situating the bins relative the vertical (gravity). These variables do enable converting measurements into geographical.
|}

==Optional ancillary variables==

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| PRES
| sea_water_pressure
| TIME
| dbar, equals 0 at the sea surface and positive down.
|-
| TEMP*
| sea_water_temperature
| TIME
| degrees_Celsius, in-situ temperature ITS-90 scale
|-
| colspan="4" | *Data from concurrent sensors may be optionally included e.g., salinity, dissolved oxygen. Kinematic viscosity of seawater needs to be calculated during processing,
|}

</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">
<br>
This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.
{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the ATOMIX Level 1 content.
| <blockquote>''This group includes the raw measurements from the recorder and ancillary measurements required for quality-controlling them using the manufacturer's recommendations.''</blockquote>
|-
| comment (optional)
| Information is pertinent to problems in the raw data files during collection.
| Examples: Stitching of files, corruption of binary files that were recovered by the manufacturer, etc.
|}
</div>

----
Return to [[Dataset requirements for ADCP structure function]]

Go to [[Level 2 data (velocity profilers)| Level 2 segmented]]

Talk:Level 4 data (velocity profilers)

2022-03-21T21:26:16Z

Jmmcmillan: Update and clarification question

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:07, 7 January 2022 (CET) suggested Comments:<br>
R_MAX - maximum R_DEL separation distance used for the regression, dimensions should be N_BEAM and units is meters<br>
REGRESSION_COEFFICIENT_A0 - intercept from the regression of DLL against R_DEL, indicative of noise, units is meter**2 / second**2<br>
REGRESSION_COEFFICIENT_A1 - regression coefficient from the regression of DLL against R_DEL used in the calculation of <math>\varepsilon</math>, units is meter**(4/3) / second**2<br>
REGRESSION_R2 - goodness of fit parameter for the regression (user to include method description in metadata)<br>
REGRESSION_N - number of data points used in the regression

----
[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 22:26, 21 March 2022 (CET) Tables updated, which included the addition of Brian's desciptions above

----
[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 22:26, 21 March 2022 (CET) What is meant by "adjusted_for_number_of_terms" in the rsquared variable name?

Level 4 data (velocity profilers)

2022-03-21T21:23:56Z

Jmmcmillan: Rewrite units for better readability

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. {{FontColor|fg=red|text=Provide link.}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>

|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
???
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-03-21T21:22:43Z

Jmmcmillan: Updated descriptions based on Brian's comments in discussion page.

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>/s<math>^2</math> and value is proportional to instrument noise. {{FontColor|fg=red|text=Provide link.}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>/s<math>^2</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>

|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
???
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-03-21T21:16:43Z

Jmmcmillan: Removed separate info for 5-beam since now included through use of N_BEAM

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>

|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-03-21T21:15:05Z

Jmmcmillan: Filled in missing info

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional dimensions for 5-beam ADCP
|-
| Enter variables here or create a footnote (note that N_BEAM disappears as a dimension for the 5th beam)
|
| TIME, R_DIST5
|
|-
| colspan="4" |

<math>\dagger\dagger</math> Variable required only when measuring in [[Burst sampling|bursts]] e.g., 18 min of continuous measurements every hour.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-03-21T20:56:29Z

Jmmcmillan: Updated dimension to be Z_DIST instead of R_DIST

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=red|text=Specify how error was computed.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=red|text=Specify how error was computed.}}
|-
| R_MAX
| max-mum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=help!}}
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=help!}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=help!}}
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=help!}}
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=Note the CF-compliant standard name modifier "number_of_observations"}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional dimensions for 5-beam ADCP
|-
| Enter variables here or create a footnote (note that N_BEAM disappears as a dimension for the 5th beam)
|
| TIME, R_DIST5
|
|-
| colspan="4" |

<math>\dagger\dagger</math> Variable required only when measuring in [[Burst sampling|bursts]] e.g., 18 min of continuous measurements every hour.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-03-21T20:55:12Z

Jmmcmillan: Updated names for consistency with Cynthia'a netcdf tools

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, R_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, R_DIST, N_BEAM
| {{FontColor|fg=red|text=Specify how error was computed.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, R_DIST, N_BEAM
| {{FontColor|fg=red|text=Specify how error was computed.}}
|-
| R_MAX
| max-mum_separation_distance_for_DLL_regression
| TIME, R_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=help!}}
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, R_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=help!}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, R_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=help!}}
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, R_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=help!}}
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, R_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=Note the CF-compliant standard name modifier "number_of_observations"}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional dimensions for 5-beam ADCP
|-
| Enter variables here or create a footnote (note that N_BEAM disappears as a dimension for the 5th beam)
|
| TIME, R_DIST5
|
|-
| colspan="4" |

<math>\dagger\dagger</math> Variable required only when measuring in [[Burst sampling|bursts]] e.g., 18 min of continuous measurements every hour.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 1 data (velocity profilers)

2022-03-21T20:43:02Z

Jmmcmillan: Updated names to ensure consistency with Cynthia's netcdf tools

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
The required dimensions and variables for the first processing level within NetCDF ATOMIX format for ADCP velocity measurements are described below. This processing level contains the raw measurements recorded by the instrument. If sampling is in [[Burst sampling|burst mode]], the measurements from individual [[Burst sampling|bursts]] are appended together.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
==Required variables==
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comments and units
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| TIME, Z_DIST, N_BEAM
| Units in m/s
|-
| R_VEL_FLAGS
| water_radial_velocity_of_scatterers_
towards_instrument_status_flag
| TIME, R_DIST, N_BEAM
| Boolean flags (8 bit, 0-255) to represent one of 8 possible reasons for rejection. {{FontColor|fg=white|bg=red|text= Link to own flagging page}}
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| vertical size of the ADCP bins. Usually the same for all 4 beams, but 5th beam can vary.
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Quality-control variables recorded by acoustic-Doppler instruments''' <math>\ddagger</math>
|-
| ABSIC
| backscatter_intensity
| TIME, Z_DIST, N_BEAM
| Units of counts
|-
| CORR
| correlation_magnitude_from_each_acoustic_beam
| TIME, Z_DIST, N_BEAM
| Unit of counts
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Platform motion variables'''<math>\dagger</math>
|-
| HEADING
| platform_yaw_angle
| TIME
| degrees, clockwise from true North
|-
| PITCH
| platform_pitch_angle
| TIME
| degrees
|-
| ROLL
| platform_roll_angle
| TIME
| degrees
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables'''
|-
| BURST_NUMBER
| unique_identifier_for_each_burst
| TIME
| Integers of 1, 2, etc to designate which [[Burst sampling|burst]] the velocities are associated with. Can omit if continuously sampled.
|-
| colspan="4" | <math>\ddagger</math> Quality-control variable names are consistent for those onboard RDI Teleydyne ADCPs. Nortek has CORRN (noise correlation as a % instead of counts) detailed in the [[Level_1_data_(velocity_point-measurements)#Variables|ADV Level 1 variables]].

<math>\dagger</math> Motion variables are necessary for structure function analysis for situating the bins relative the vertical (gravity). These variables do enable converting measurements into geographical.
|}

==Optional ancillary variables==

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| PRES
| sea_water_pressure
| TIME
| dbar, equals 0 at the sea surface and positive down.
|-
| TEMP*
| sea_water_temperature
| TIME
| degrees_Celsius, in-situ temperature ITS-90 scale
|-
| colspan="4" | *Data from concurrent sensors may be optionally included e.g., salinity, dissolved oxygen. Kinematic viscosity of seawater needs to be calculated during processing,
|}

</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">
<br>
This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.
{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the ATOMIX Level 1 content.
| <blockquote>''This group includes the raw measurements from the recorder and ancillary measurements required for quality-controlling them using the manufacturer's recommendations.''</blockquote>
|-
| comment (optional)
| Information is pertinent to problems in the raw data files during collection.
| Examples: Stitching of files, corruption of binary files that were recovered by the manufacturer, etc.
|}
</div>

----
Return to [[Dataset requirements for ADCP structure function]]

Go to [[Level 2 data (velocity profilers)| Level 2 segmented]]

Level 2 data (velocity profilers)

2022-03-21T20:39:57Z

Jmmcmillan: Updated names for consistency with Cynthia'a netcdf tools

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_
segment_in_the_entire_available_timeseries
| N_SEGMENT
| Replacement for using TIME centred in the segment as dimension
|-
| N_SAMPLE
| unique_identifier_for_each_
sample_within_the_segment
| N_SAMPLE
| Max sample number is the segment length (seconds) multiplied by the sampling rate
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| N_SEGMENT, Z_DIST,N_SAMPLE, N_BEAM
| Along beam velocity [m/s] for each of the beams.
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_SAMPLE, N_BEAM
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since 1950-01-01T00:00:00Z
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| constant
| Vertical size of the ADCP bins [m]. Usually the same for all 4 diverging beams.
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after segmenting them into smaller chunks.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Talk:Level 3 data (velocity profilers)

2022-03-21T20:36:00Z

Jmmcmillan:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:11, 30 December 2021 (CET) re. DLL_FLAGS - we have both the calculated DLL and the associated qaqc flags at the same level. It makes me wonder whether it would be better to have the R_VEL_FLAGS at level 1. They are currently at level 2 and therefore require either that the level 1 data is duplicated at level 2 or means they sit separate from the associated data. Why not have level 1 being the raw data with qaqc flags and level 2 being the data rearranged into segments (which may be the original bursts) with appropriate pre-processing (detrending) ready for the DLL calculation? It would seem cleaner and more consistent.

This would also allow for the possibility of separate qaqc flags to be defined at level 2 e.g. outlier detection based on the segmented data.

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:39, 17 January 2022 (CET) The flags were changed.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 16:47, 29 December 2021 (CET)
re. TIME dimension comments - the requirement to define time bounds for each segment looks rather complex and I’m not sure that it adds anything. Presumably the requirement to specify bounds will not be mandatory?

Having introduced N_SEGMENT as a dimension at level 2, with TIME as a variable, we are now reverting to TIME as the dimension with N_SEGMENT as the variable. Given that TIME is now derived as the mean time for the observations in the segment, wouldn’t it be more appropriate to keep it as the variable?

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:39, 17 January 2022 (CET) Time bnds is based on CF-compliant dataset. Personally, TIME should be always a dimension as per CF standards. Not a dimension. It was changed at LEvl 2 because people preferred not calculating TIME at Level 2 (violating CF-standards), but NETCDF guis can handle quite nicely time (centered) variables.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:01, 30 December 2021 (CET)
re. R_DEL / R_DEL5 dimension comments - R_DEL should be calculated as a function of R_DIST, which itself is a function of bin size and theta, but having defined R_DIST, it should now be the basis on which R_DEL is calculated. So for example (assuming Matlab indexing), for a central difference scheme evaluated at bin 10 i.e. R_DIST(10), the two-bin separation R_DEL(2) = R_DIST(11) - R_DIST(9), whereas for a forward difference scheme evaluated at bin 10, R_DEL(2) would be R_DIST(12) - R_DIST(10). The R_DEL(2) values will be identical, but the principle is that R_DEL is the separation distance distance the velocity observations being compared.

Also note that R_DEL units should be specified as (in meters).

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:00, 30 December 2021 (CET) re. DLL_N comment - suggest reword as “number of instances when the velocity difference is evaluated, maximum is [number of profiles in segment - either max(N_SAMPLE) or possibly segment_length if redefined as number of profiles rather than time duration]"

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:45, 17 January 2022 (CET) The comments in these tables are for the wiki (teaching purposes). The highlights text in pink brings users to the up-to-date attribute tables. The use "number_of_observations" is a standard modifier for CF-standards and should not be changed. It means the number of usable (good) samples in that "chunk" of data i.e., segment.
----

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:23, 17 January 2022 (CET) For levels 1 and 2 we have moved from R_DIST being a dimension to Z_DIST so that data for both angled and vertical beams can be combined in the R_VEL array. This requires that the profile times are the same for all beams, but leaves open the possibility of the vertical bin size being different between the angled and vertical beams - hence the dimension of BIN_SIZE at level 1 is N_BEAM. However, if we allow for that possibility, then we run into problems using R_DEL as a dimension at level 3 - since the values will differ between the beams and, as I understand it, we can’t have dimension R_DEL with dimensions N_BEAM and R_DEL.

If we constrain the flexibility such that the vertical beam data can only be combined with the angled beam data if both the sampling times and the vertical bin size are the same for both, then at level 3 we can define a dimension Z_DEL which the vertical separation distance between bins at which the mean of the squared velocity difference is calculated - this would be the same for both vertical and angled beams. It would have dimension Z_DEL. Then R_DEL would be a derived variable with dimensions N_BEAM and Z_DEL calculated as a function of THETA and Z_DEL.

If either the sampling times or the vertical bin size differed between the angled and vertical beams, the user would have to prepare separate data files for the two.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 10:26, 18 January 2022 (CET) Alternatively, we could specify the “R_DEL" dimension in “bin separations” i.e. integer values - although this might be better named as N_SEPARATION or something. R_DEL would then be a derived variable calculated from BIN_SIZE (dimension N_BEAM), THETA (dimension N_BEAM) and N_SEPARATION (dimension N_SEPARATION). Whilst this would allow for differences in BIN_SIZE, it would still be problematic if there were significant differences in the number of bins between the beams and sampling times still need to be the same for all beams.
[[File:SF Fit JMM.png|thumb|Example of a SF fit using JMM's method with all possible combinations of bin separations leading to non-unique R_DEL values (Note: r on the x-axis i= R_DEL)]]]

:: [[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 23:18, 14 February 2022 (CET) I agree with Brian that we need a different dimension at level 3. I prefer using something like N_DEL or N_SEPARATION to specify the number of bins of separation as opposed to Z_DEL because it make more logical sense to work along the beams for level 3. In my specific implementation of the SF, my dimension R_DEL is actually non-unique (I take all possible combinations of separation distances within my turbulence 'blob'). For now, I've chosen to use N_R_DEL as my dimension which is a vector of integers from 1 to 28, where 28 is the maximum number of points I can have in my regression based on my choice of R_MAX. Then R_DEL is defined as a variable instead of a dimension. But, if it is possible to have non-unique values as a dimension then N_DEL could also work for me. I will do some testing to confirm.

----

[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:23, 21 March 2022 (CET) I've updated the dimension to be N_DEL instead of R_DEL on the wiki page.

----
[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:35, 21 March 2022 (CET) In the interest of reducing the number of required variables, we could remove:

* R_DIST: can be calculated from Z_DIST and THETA, and isn't actually used for any future analysis.
* N_SEGMENT: will always be a series of integers from 1 to length(TIME). I don't think this dimension from Level 2 needs to be repeated here.
* BURST_NUMBER: Not required for continuous sampling data sets

Talk:Level 3 data (velocity profilers)

2022-03-21T20:35:28Z

Jmmcmillan:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:11, 30 December 2021 (CET) re. DLL_FLAGS - we have both the calculated DLL and the associated qaqc flags at the same level. It makes me wonder whether it would be better to have the R_VEL_FLAGS at level 1. They are currently at level 2 and therefore require either that the level 1 data is duplicated at level 2 or means they sit separate from the associated data. Why not have level 1 being the raw data with qaqc flags and level 2 being the data rearranged into segments (which may be the original bursts) with appropriate pre-processing (detrending) ready for the DLL calculation? It would seem cleaner and more consistent.

This would also allow for the possibility of separate qaqc flags to be defined at level 2 e.g. outlier detection based on the segmented data.

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:39, 17 January 2022 (CET) The flags were changed.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 16:47, 29 December 2021 (CET)
re. TIME dimension comments - the requirement to define time bounds for each segment looks rather complex and I’m not sure that it adds anything. Presumably the requirement to specify bounds will not be mandatory?

Having introduced N_SEGMENT as a dimension at level 2, with TIME as a variable, we are now reverting to TIME as the dimension with N_SEGMENT as the variable. Given that TIME is now derived as the mean time for the observations in the segment, wouldn’t it be more appropriate to keep it as the variable?

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:39, 17 January 2022 (CET) Time bnds is based on CF-compliant dataset. Personally, TIME should be always a dimension as per CF standards. Not a dimension. It was changed at LEvl 2 because people preferred not calculating TIME at Level 2 (violating CF-standards), but NETCDF guis can handle quite nicely time (centered) variables.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:01, 30 December 2021 (CET)
re. R_DEL / R_DEL5 dimension comments - R_DEL should be calculated as a function of R_DIST, which itself is a function of bin size and theta, but having defined R_DIST, it should now be the basis on which R_DEL is calculated. So for example (assuming Matlab indexing), for a central difference scheme evaluated at bin 10 i.e. R_DIST(10), the two-bin separation R_DEL(2) = R_DIST(11) - R_DIST(9), whereas for a forward difference scheme evaluated at bin 10, R_DEL(2) would be R_DIST(12) - R_DIST(10). The R_DEL(2) values will be identical, but the principle is that R_DEL is the separation distance distance the velocity observations being compared.

Also note that R_DEL units should be specified as (in meters).

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:00, 30 December 2021 (CET) re. DLL_N comment - suggest reword as “number of instances when the velocity difference is evaluated, maximum is [number of profiles in segment - either max(N_SAMPLE) or possibly segment_length if redefined as number of profiles rather than time duration]"

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:45, 17 January 2022 (CET) The comments in these tables are for the wiki (teaching purposes). The highlights text in pink brings users to the up-to-date attribute tables. The use "number_of_observations" is a standard modifier for CF-standards and should not be changed. It means the number of usable (good) samples in that "chunk" of data i.e., segment.
----

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:23, 17 January 2022 (CET) For levels 1 and 2 we have moved from R_DIST being a dimension to Z_DIST so that data for both angled and vertical beams can be combined in the R_VEL array. This requires that the profile times are the same for all beams, but leaves open the possibility of the vertical bin size being different between the angled and vertical beams - hence the dimension of BIN_SIZE at level 1 is N_BEAM. However, if we allow for that possibility, then we run into problems using R_DEL as a dimension at level 3 - since the values will differ between the beams and, as I understand it, we can’t have dimension R_DEL with dimensions N_BEAM and R_DEL.

If we constrain the flexibility such that the vertical beam data can only be combined with the angled beam data if both the sampling times and the vertical bin size are the same for both, then at level 3 we can define a dimension Z_DEL which the vertical separation distance between bins at which the mean of the squared velocity difference is calculated - this would be the same for both vertical and angled beams. It would have dimension Z_DEL. Then R_DEL would be a derived variable with dimensions N_BEAM and Z_DEL calculated as a function of THETA and Z_DEL.

If either the sampling times or the vertical bin size differed between the angled and vertical beams, the user would have to prepare separate data files for the two.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 10:26, 18 January 2022 (CET) Alternatively, we could specify the “R_DEL" dimension in “bin separations” i.e. integer values - although this might be better named as N_SEPARATION or something. R_DEL would then be a derived variable calculated from BIN_SIZE (dimension N_BEAM), THETA (dimension N_BEAM) and N_SEPARATION (dimension N_SEPARATION). Whilst this would allow for differences in BIN_SIZE, it would still be problematic if there were significant differences in the number of bins between the beams and sampling times still need to be the same for all beams.
[[File:SF Fit JMM.png|thumb|Example of a SF fit using JMM's method with all possible combinations of bin separations leading to non-unique R_DEL values (Note: r on the x-axis i= R_DEL)]]]

:: [[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 23:18, 14 February 2022 (CET) I agree with Brian that we need a different dimension at level 3. I prefer using something like N_DEL or N_SEPARATION to specify the number of bins of separation as opposed to Z_DEL because it make more logical sense to work along the beams for level 3. In my specific implementation of the SF, my dimension R_DEL is actually non-unique (I take all possible combinations of separation distances within my turbulence 'blob'). For now, I've chosen to use N_R_DEL as my dimension which is a vector of integers from 1 to 28, where 28 is the maximum number of points I can have in my regression based on my choice of R_MAX. Then R_DEL is defined as a variable instead of a dimension. But, if it is possible to have non-unique values as a dimension then N_DEL could also work for me. I will do some testing to confirm.

----

[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:23, 21 March 2022 (CET) I've updated the dimension to be N_DEL instead of R_DEL on the wiki page.

---
[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:35, 21 March 2022 (CET) In the interest of reducing the number of required variables, we could remove:

* R_DIST: can be calculated from Z_DIST and THETA, and isn't actually used for any future analysis.
* N_SEGMENT: will always be a series of integers from 1 to length(TIME). I don't think this dimension from Level 2 needs to be repeated here.
* BURST_NUMBER: Not required for continuous sampling data sets

Level 3 data (velocity profilers)

2022-03-21T20:31:09Z

Jmmcmillan: Corrected names for consistency with netcdf tools used to generate files

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
[[File:SF atomix ADCP.png|300px|thumb|Schematic of ADCP processing nomenclature]]
The required dimensions and variables for the structure-function processing level within NetCDF ATOMIX format for velocity ADCP measurements are described below. This NetCDF group contains the structure function (DLL) calculated as a function of the along-beam separation for the available/usable ADCP bins.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_DEL
| along-beam_separation_distance_
over_which_DLL_is_evaluated_in_number_of_bins
| N_DEL
| Number of bins separating two velocity measurements used to calculate DLL
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| DLL
| second_order_structure_function
| TIME, Z_DIST, N_DEL, N_BEAM
| Differences in velocities squared have been time-averaged (units of m2/s2).
|-
| DLL_FLAGS
| second_order_structure_function_status_flag
| TIME, Z_DIST, N_DEL, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| R_DEL
| along-beam_separation_distance_
at_which_structure_function_is_evaluated
| N_DEL, Z_DIST, N_BEAM, TIME
| Estimated quantity (in meters) from N_DEL (Level 3), BIN_SIZE (Level 2) and THETA (Level 2).
|-
| R_DIST
| distance_from_sensor_along_beams
| Z_DIST, N_BEAM
| Along-beam bin centre distance (in meters) from the transducer
|-
| DLL_N
| second_order_structure_function_number_of_observations
| TIME, Z_DIST, N_DEL, N_BEAM
| The number of available measurements in each segment i.e., data quality.
|-
| N_SEGMENT
| unique_identifier_for_each_segment_
in_the_entire_available_timeseries
| TIME
| Enables backtracking to [[Level 2 segmented (velocity profilers)|previous processing level]]
|-
| BURST_NUMBER <math>\dagger\dagger</math>
| unique_identifier_for_each_burst
| TIME
| Only required when measuring in burst-mode. Integers of 1, 2, etc to designate which burst the velocities are associated with.
|-
| colspan="4" |
<math>\dagger\dagger</math> Optional variable required only when measuring in bursts e.g., 18 min of continuous measurements every hour.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the structure function Dll as a function of the separation distance. Any ancillary information required for estimating the dissipation of turbulent kinetic energy may also be stored here. ''</blockquote>
|-
| dll_calculation_type
| Specify differencing technique used to estimate DLL
| Examples include:
<blockquote>Central-differencing</blockquote>
<blockquote>Forward-differencing</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes <math>\ddagger</math>
|-
| stationarity_testing
| Any testing done on the segment to verify stationarity?
| {{FontColor|fg=white|bg=red|text=To be revisited once testing begins}}. Tentatively refer to [https://bitbucket.org/efm_cb/netcdf/src/master/TestData/adcp_atomix_metada.yml demo yaml] file.
|-
| noise_testing
| Details of testing the noise levels, or if the signal comprises mostly of noise?
|
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 2 data (velocity profilers)| Level 2 segmented velocities]]

Go to [[Level 4 data (velocity profilers)| Level 4 dissipation estimates]]

Talk:Level 3 data (velocity profilers)

2022-03-21T20:23:03Z

Jmmcmillan:

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:11, 30 December 2021 (CET) re. DLL_FLAGS - we have both the calculated DLL and the associated qaqc flags at the same level. It makes me wonder whether it would be better to have the R_VEL_FLAGS at level 1. They are currently at level 2 and therefore require either that the level 1 data is duplicated at level 2 or means they sit separate from the associated data. Why not have level 1 being the raw data with qaqc flags and level 2 being the data rearranged into segments (which may be the original bursts) with appropriate pre-processing (detrending) ready for the DLL calculation? It would seem cleaner and more consistent.

This would also allow for the possibility of separate qaqc flags to be defined at level 2 e.g. outlier detection based on the segmented data.

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:39, 17 January 2022 (CET) The flags were changed.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 16:47, 29 December 2021 (CET)
re. TIME dimension comments - the requirement to define time bounds for each segment looks rather complex and I’m not sure that it adds anything. Presumably the requirement to specify bounds will not be mandatory?

Having introduced N_SEGMENT as a dimension at level 2, with TIME as a variable, we are now reverting to TIME as the dimension with N_SEGMENT as the variable. Given that TIME is now derived as the mean time for the observations in the segment, wouldn’t it be more appropriate to keep it as the variable?

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:39, 17 January 2022 (CET) Time bnds is based on CF-compliant dataset. Personally, TIME should be always a dimension as per CF standards. Not a dimension. It was changed at LEvl 2 because people preferred not calculating TIME at Level 2 (violating CF-standards), but NETCDF guis can handle quite nicely time (centered) variables.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:01, 30 December 2021 (CET)
re. R_DEL / R_DEL5 dimension comments - R_DEL should be calculated as a function of R_DIST, which itself is a function of bin size and theta, but having defined R_DIST, it should now be the basis on which R_DEL is calculated. So for example (assuming Matlab indexing), for a central difference scheme evaluated at bin 10 i.e. R_DIST(10), the two-bin separation R_DEL(2) = R_DIST(11) - R_DIST(9), whereas for a forward difference scheme evaluated at bin 10, R_DEL(2) would be R_DIST(12) - R_DIST(10). The R_DEL(2) values will be identical, but the principle is that R_DEL is the separation distance distance the velocity observations being compared.

Also note that R_DEL units should be specified as (in meters).

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:00, 30 December 2021 (CET) re. DLL_N comment - suggest reword as “number of instances when the velocity difference is evaluated, maximum is [number of profiles in segment - either max(N_SAMPLE) or possibly segment_length if redefined as number of profiles rather than time duration]"

:: [[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 16:45, 17 January 2022 (CET) The comments in these tables are for the wiki (teaching purposes). The highlights text in pink brings users to the up-to-date attribute tables. The use "number_of_observations" is a standard modifier for CF-standards and should not be changed. It means the number of usable (good) samples in that "chunk" of data i.e., segment.
----

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:23, 17 January 2022 (CET) For levels 1 and 2 we have moved from R_DIST being a dimension to Z_DIST so that data for both angled and vertical beams can be combined in the R_VEL array. This requires that the profile times are the same for all beams, but leaves open the possibility of the vertical bin size being different between the angled and vertical beams - hence the dimension of BIN_SIZE at level 1 is N_BEAM. However, if we allow for that possibility, then we run into problems using R_DEL as a dimension at level 3 - since the values will differ between the beams and, as I understand it, we can’t have dimension R_DEL with dimensions N_BEAM and R_DEL.

If we constrain the flexibility such that the vertical beam data can only be combined with the angled beam data if both the sampling times and the vertical bin size are the same for both, then at level 3 we can define a dimension Z_DEL which the vertical separation distance between bins at which the mean of the squared velocity difference is calculated - this would be the same for both vertical and angled beams. It would have dimension Z_DEL. Then R_DEL would be a derived variable with dimensions N_BEAM and Z_DEL calculated as a function of THETA and Z_DEL.

If either the sampling times or the vertical bin size differed between the angled and vertical beams, the user would have to prepare separate data files for the two.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 10:26, 18 January 2022 (CET) Alternatively, we could specify the “R_DEL" dimension in “bin separations” i.e. integer values - although this might be better named as N_SEPARATION or something. R_DEL would then be a derived variable calculated from BIN_SIZE (dimension N_BEAM), THETA (dimension N_BEAM) and N_SEPARATION (dimension N_SEPARATION). Whilst this would allow for differences in BIN_SIZE, it would still be problematic if there were significant differences in the number of bins between the beams and sampling times still need to be the same for all beams.
[[File:SF Fit JMM.png|thumb|Example of a SF fit using JMM's method with all possible combinations of bin separations leading to non-unique R_DEL values (Note: r on the x-axis i= R_DEL)]]]

:: [[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 23:18, 14 February 2022 (CET) I agree with Brian that we need a different dimension at level 3. I prefer using something like N_DEL or N_SEPARATION to specify the number of bins of separation as opposed to Z_DEL because it make more logical sense to work along the beams for level 3. In my specific implementation of the SF, my dimension R_DEL is actually non-unique (I take all possible combinations of separation distances within my turbulence 'blob'). For now, I've chosen to use N_R_DEL as my dimension which is a vector of integers from 1 to 28, where 28 is the maximum number of points I can have in my regression based on my choice of R_MAX. Then R_DEL is defined as a variable instead of a dimension. But, if it is possible to have non-unique values as a dimension then N_DEL could also work for me. I will do some testing to confirm.

----
[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:23, 21 March 2022 (CET) I've updated the dimension to be N_DEL instead of R_DEL on the wiki page.

Level 3 data (velocity profilers)

2022-03-21T20:22:02Z

Jmmcmillan: Updated dimension to be N_DEL instead of R_DEL

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
[[File:SF atomix ADCP.png|300px|thumb|Schematic of ADCP processing nomenclature]]
The required dimensions and variables for the structure-function processing level within NetCDF ATOMIX format for velocity ADCP measurements are described below. This NetCDF group contains the structure function (DLL) calculated as a function of the along-beam separation for the available/usable ADCP bins.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_z_instrument_axis
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_DEL
| along-beam_separation_distance_
over_which_DLL_is_evaluated_in_number_of_bins
| N_DEL
| Number of bins separating two velocity measurements used to calculate DLL
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| DLL
| second_order_structure_function
| TIME, Z_DIST, N_DEL, N_BEAM
| Differences in velocities squared have been time-averaged (units of m2/s2).
|-
| DLL_FLAGS
| second_order_structure_function_status_flags
| TIME, Z_DIST, N_DEL, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| R_DEL
| along-beam_separation_distance_
at_which_structure_function_is_evaluated
| N_DEL, Z_DIST, N_BEAM, TIME
| Estimated quantity (in meters) from bin sizes and theta in Level 2 qaqc variables
|-
| R_DIST
| distance_from_sensor_along_beams
| Z_DIST, N_BEAM
| Along-beam bin centre distance (in meters) from the transducer
|-
| DLL_N
| second_order_structure_function_number_of_observations
| TIME, Z_DIST, N_DEL, N_BEAM
| Measure the number of available measurements in each segment i.e., data quality.
|-
| N_SEGMENT
| unique_identifier_for_each_segment_
in_the_entire_available_timeseries
| TIME
| Enables backtracking to [[Level 2 segmented (velocity profilers)|previous processing level]]
|-
| BURST_NUMBER <math>\dagger\dagger</math>
| unique_identifier_for_each_burst
| TIME
| Only required when measuring in burst-mode. Integers of 1, 2, etc to designate which burst the velocities are associated with.
|-
| colspan="4" |
<math>\dagger\dagger</math> Optional variable required only when measuring in bursts e.g., 18 min of continuous measurements every hour.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the structure function Dll as a function of the separation distance. Any ancillary information required for estimating the dissipation of turbulent kinetic energy may also be stored here. ''</blockquote>
|-
| dll_calculation_type
| Specify differencing technique used to estimate DLL
| Examples include:
<blockquote>Central-differencing</blockquote>
<blockquote>Forward-differencing</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes <math>\ddagger</math>
|-
| stationarity_testing
| Any testing done on the segment to verify stationarity?
| {{FontColor|fg=white|bg=red|text=To be revisited once testing begins}}. Tentatively refer to [https://bitbucket.org/efm_cb/netcdf/src/master/TestData/adcp_atomix_metada.yml demo yaml] file.
|-
| noise_testing
| Details of testing the noise levels, or if the signal comprises mostly of noise?
|
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 2 data (velocity profilers)| Level 2 segmented velocities]]

Go to [[Level 4 data (velocity profilers)| Level 4 dissipation estimates]]

Level 2 data (velocity profilers)

2022-03-21T19:59:24Z

Jmmcmillan: Added units and other minor clarifications

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_
segment_in_the_entire_available_timeseries
| N_SEGMENT
| Replacement for using TIME centred in the segment as dimension
|-
| N_SAMPLE
| unique_identifier_for_each_
sample_within_the_segment
| N_SAMPLE
| Max sample number is the segment length (seconds) multiplied by the sampling rate
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| Z_DIST
| distance_from_sensor_along_z_instrument_axis
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| N_SEGMENT, Z_DIST,N_SAMPLE, N_BEAM
| Along beam velocity [m/s] for each of the beams.
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_SAMPLE, N_BEAM
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since 1950-01-01T00:00:00Z
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| constant
| Vertical size of the ADCP bins [m]. Usually the same for all 4 diverging beams.
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after segmenting them into smaller chunks.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Talk:Level 1 data (velocity profilers)

2022-03-21T19:42:57Z

Jmmcmillan: Clarification questions

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:16, 28 December 2021 (CET) I don’t think the definitions work properly for instruments with a vertical beam. We are inconsistent in separating the “regular” and “vertical” beams e.g. dimension N_BEAM is defined as including the vertical beam, but is used as a dimension for R_VEL which doesn’t include the vertical beam velocity data.

There is also the issue identified in the notes that some instruments allow the vertical beam to be defined with a different sampling rate / operating mode / ensemble averaging / burst interval etc and have a different bin size, requiring a suggested TIME5 dimension as well as separate variables wherever TIME is a dimension. I suggest it would be cleaner to always treat the angled and vertical beams separately.

Should we also allow for the possibility of instruments with three angled beams and a vertical beam? If so, the use of the tag “5" on the variable names becomes somewhat inappropriate. I suggest we use the tag “V" for vertical (which is also Roman 5).

The mandatory dimensions would remain as TIME, R_DIST and N_BEAM but these only relate to the angled beams (so N_BEAM would be either [1 2 3] or [1 2 3 4]). The variables R_VEL, ABSI and CORR use all of these dimensions, whilst BURST_NUMBER uses TIME. THETA and BIN_SIZE are then constants (not sure why THETA is currently shown with dimension TIME - I think that’s an error).

We then have optional dimensions TIMEV and R_DISTV and optional variables R_VELV, ABSIV and CORRV with both of these dimensions; BURST_NUMBERV with dimension TIMEV; and BIN_SIZEV is a constant. If the sampling for the vertical beam is the same as for the angled beams, then TIMEV and TIME are simply the same - although I suspect this would be the exception.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 12:45, 29 December 2021 (CET) The more I think about it, the less convinced I become that BURST_NUMBER is actually helpful - I’m certainly not convinced it should be a mandatory variable. It isn’t part of the raw data output from TRDI instruments (not sure about Nortek) and it serves no purpose if the data is collected on a continuous basis. Plus it implicit in the TIME dimension values anyway.

More fundamentally, BURST_NUMBER replicates what N_SEGMENT does and I don’t think we need the added complexity of having both.

What I think would be helpful would be to have a variable that is an integer count simply identifying each velocity profile instance (consisting of number of beams x number of bins velocity values) at level 1, this variable (PROFILE_NUMBER or N_PROF or something - I’m unclear on the naming convention) would have the dimension TIME and would simply increment 1, 2, 3 etc. but (with the N_BEAM and R_DIST) provides a unique identifier for each velocity value and as an integer is much easier to work with (e.g. for indexing) than the float TIME. With an equivalent for the vertical beam where appropriate.

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 13:49, 28 December 2021 (CET) Suggested units of ABSI are db. For the TRDI instruments the WA configuration setting for the fish rejection algorithm (which is likely to be familiar to any user configuring an instrument for deployment) is based on the echo intensity “count” returned by the instrument not db. So if users are implementing the fish rejection algorithm as part of their QC, I would expect they will use a count difference threshold rather than a db difference. The conversion between counts and db is very simple (1 count = 0.5 db), unless users want to get into distance compensation, but requiring the conversion to db may be unhelpful. Perhaps we should allow the user to specify either count or db?

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 14:00, 28 December 2021 (CET) Optional ancillary variables - what is dimension N_VEL_INSTRUMENT shown for variable PRES? Re note on TEMP - Kinematic viscosity is not required for the SF calculation.

[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 14:17, 7 January 2022 (CET) I agree that BURST_NUMBER shouldn't be required. It is useful for continuous data in the sense that it distinguishes the ensemble that the data are part of, but I think this could be better captured at level 2 in the segmented data. I also agree that we shouldn't limit ourselves to calling the vertical beam #5, and like Brian's suggestion of using a "V" instead.

----
[[User:CynthiaBluteau|CynthiaBluteau]] ([[User talk:CynthiaBluteau|talk]]) 02:03, 15 January 2022 (CET) Responses:
* The units for ABSI and CORR are different from ABSIC (counts) and CORRN (%). They should be in the native unit, not converted.
* RV was changed according TEAMS discussion.
* Burst_number was is not the same as N_SEGMENT. Nortek instrument spits data file into many "bursts" and it's useful to be able to refer back to the rawest dat files. I will let the user decide to add if desired.

----
[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 11:17, 17 January 2022 (CET) Dimensions for required variable PROFILE_NUMBER should just be TIME whilst the R_DIST dimension for R_VEL, ABSIC and CORR should now be Z_DIST.

I think we also agreed that the level 1 qaqc R_VEL_FLAGS should also be included as a required variable?

----
[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 20:42, 21 March 2022 (CET) Can someone clarify the following:

1. Should PROFILE_NUMBER be N_PROFILE (for consistency with other identifying integers like N_BEAM)? I noticed that in Cynthia's netcdf tools, the variable N_PROFILE exists, but PROFILE_NUMBER does not. The same comment applied to BURST_NUMBER.

2. ABSIC is in counts, and ABSI is in dB? Should we just require one or the other and not both?

Level 1 data (velocity profilers)

2022-03-21T19:26:38Z

Jmmcmillan: Added degree symbol

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
The required dimensions and variables for the first processing level within NetCDF ATOMIX format for ADCP velocity measurements are described below. This processing level contains the raw measurements recorded by the instrument. If sampling is in [[Burst sampling|burst mode]], the measurements from individual [[Burst sampling|bursts]] are appended together.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z
|-
| Z_DIST
| distance_from_sensor_along_z_instrument_axis
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
==Required variables==
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comments and units
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| TIME, Z_DIST, N_BEAM
| Units in m/s
|-
| R_VEL_FLAGS
| water_radial_velocity_of_scatterers_
towards_instrument_status_flags
| TIME, R_DIST, N_BEAM
| Boolean flags (8 bit, 0-255) to represent one of 8 possible reasons for rejection. {{FontColor|fg=white|bg=red|text= Link to own flagging page}}
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| vertical size of the ADCP bins. Usually the same for all 4 beams, but 5th beam can vary.
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Quality-control variables recorded by acoustic-Doppler instruments''' <math>\ddagger</math>
|-
| ABSIC
| backscatter_intensity
| TIME, Z_DIST, N_BEAM
| Units of counts
|-
| CORR
| correlation_magnitude
| TIME, Z_DIST, N_BEAM
| Unit of counts
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Platform motion variables'''<math>\dagger</math>
|-
| HEADING
| platform_yaw_angle
| TIME
| degrees, clockwise from true North
|-
| PITCH
| platform_pitch_angle
| TIME
| degrees
|-
| ROLL
| platform_roll_angle
| TIME
| degrees
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables'''
|-
| BURST_NUMBER
| unique_identifier_for_each_burst
| TIME
| Integers of 1, 2, etc to designate which [[Burst sampling|burst]] the velocities are associated with. Can omit if continuously sampled.
|-
| colspan="4" | <math>\ddagger</math> Quality-control variable names are consistent for those onboard RDI Teleydyne ADCPs. Nortek has CORRN (noise correlation as a % instead of counts) detailed in the [[Level_1_data_(velocity_point-measurements)#Variables|ADV Level 1 variables]].

<math>\dagger</math> Motion variables are necessary for structure function analysis for situating the bins relative the vertical (gravity). These variables do enable converting measurements into geographical.
|}

==Optional ancillary variables==

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| PRES
| sea_water_pressure
| TIME
| dbar, equals 0 at the sea surface and positive down.
|-
| TEMP*
| sea_water_temperature
| TIME
| degrees_Celsius, in-situ temperature ITS-90 scale
|-
| colspan="4" | *Data from concurrent sensors may be optionally included e.g., salinity, dissolved oxygen. Kinematic viscosity of seawater needs to be calculated during processing,
|}

</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">
<br>
This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.
{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the ATOMIX Level 1 content.
| <blockquote>''This group includes the raw measurements from the recorder and ancillary measurements required for quality-controlling them using the manufacturer's recommendations.''</blockquote>
|-
| comment (optional)
| Information is pertinent to problems in the raw data files during collection.
| Examples: Stitching of files, corruption of binary files that were recovered by the manufacturer, etc.
|}
</div>

----
Return to [[Dataset requirements for ADCP structure function]]

Go to [[Level 2 data (velocity profilers)| Level 2 segmented]]

Benchmark datasets for ADCP structure function

2022-03-11T23:37:10Z

Jmmcmillan: Added tidal channel data information

This page provides an overview of the benchmark dataset for instruments that measure velocity profiles e.g., [[Acoustic-Doppler Current Profilers|acoustic-Doppler current profilers]] from diverse suppliers and models.

== Datasets available ==
Selection and preparation of benchmark datasets are a work in progress! These benchmark datasets will cover a range of marine environments, background stratification and flow fields.

{| class="wikitable"
|+Summary of potential benchmark datasets for testing existing and future algorithms
|-
! style="width: 12%"| Dataset name
! Total depth
! Deployment height above bottom
! Background speed
! <math>\varepsilon</math> range
! Stratification/shear information
! style="width: 20%"| Comment
|-
!
! [m]
! [m]
! [m/s]
! [W/kg]
!
!
|-
| Tidal shelf
|
|
|
|
|
|
|-
| Lake
|
|
|
|
|
|
|
|-
| RDI4beam_TidalChannel_GP130620BPb
| 23
| 0.5
| 2.5
| 7e-6 to 2e-4
| unstratified
| high quality data
|}

-------------------------
return to [[Velocity profilers| ADCP structure function group Welcome Page]]
[[Category:Velocity profilers]]

Talk:Level 3 data (velocity profilers)

2022-02-14T22:18:41Z

Jmmcmillan: Issue with R_DEL as dimension.

File:SF Fit JMM.png

2022-02-14T22:18:16Z

Jmmcmillan:

Example of a SF fit using JMM's method with all possible combinations of bin separations leading to non-unique R_DEL values (Note: r on the x-axis i= R_DEL)]

Talk:Level 1 data (velocity profilers)

2022-01-07T13:17:14Z

Jmmcmillan: Response to Brian

Level 1 data (velocity profilers)

2022-01-07T12:16:14Z

Jmmcmillan:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
The required dimensions and variables for the first processing level within NetCDF ATOMIX format for ADCP velocity measurements are described below. This processing level contains the raw measurements recorded by the instrument. If sampling is in burst mode, the measurements from individual bursts are appended together.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since 1950-01-01T00:00:00Z
|-
| R_DIST
| distance_from_sensor_along_beams
| R_DIST
| along-beam bin centre distance (in meters) from the transducer
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional dimensions
|-
| R_DIST5
| distance_from_sensor_along_beam_5
| R_DIST5
| along-beam bin centre distance (in meters) from the 5th beam, which is usually aligned with vertical axis of the ADCP.
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
==Required variables==
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comments and units
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| TIME, R_DIST, N_BEAM
| Units in m/s
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30o)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| vertical size of the ADCP bins. Usually the same for all 4 beams, but 5th beam can vary.
|-
| BURST_NUMBER
| unique_identifier_for_each_burst
| TIME
| Integers of 1, 2, etc to designate which burst the velocities are associated with. Can set all to 1 if continuously sampled.
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Quality-control variables recorded by acoustic-Doppler instruments''' <math>\ddagger</math>
|-
| ABSI
| backscatter_intensity_from_each_acoustic_beam
| TIME, R_DIST, N_BEAM
| Units of db
|-
| CORR
| correlation_magnitude_from_each_acoustic_beam
| TIME, R_DIST, N_BEAM
| Unit of counts
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Platform motion variables'''<math>\dagger</math>
|-
| HEADING
| platform_yaw_angle
| TIME
| degrees, clockwise from true North
|-
| PITCH
| platform_pitch_angle
| TIME
| degrees
|-
| ROLL
| platform_roll_angle
| TIME
| degrees
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables for [[:File:SF atomix ADCP.png|5-beam ADCP]]'''<math>\dagger\dagger</math>
|-
| R_VEL5
| water_radial_velocity_of_scatterers_towards_instrument_of_beam_5
| TIME, R_DIST5
| Units in m/s
|-
| colspan="4" | <math>\ddagger</math> Quality-control variable names are consistent for those onboard RDI Teleydyne ADCPs. {{FontColor|fg=white|bg=red|text= Not sure for Nortek, perhaps they use the same as their Vectors}} detailed in the [[Level_1_data_(velocity_point-measurements)#Variables|ADV Level 1 variables]].

<math>\dagger</math> Motion variables are necessary for structure function analysis for situating the bins relative the vertical (gravity). These variables do enable converting measurements into geographical.

<math>\dagger\dagger</math> 5-beam ADCPs has the 5th beam usually pointing directly up with its own bin sampling height, and may also be sampled at a different rate. In these situations, the data must be stored in separate variables given the different dimensions assigned. Simply append the number 5 to the variable name (e.g., TIME5)
|}

==Optional ancillary variables==

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| PRES
| sea_water_pressure
| TIME, N_VEL_INSTRUMENT
| dbar, equals 0 at the sea surface and positive down.
|-
| TEMP*
| sea_water_temperature
| TIME
| degrees_Celsius, in-situ temperature ITS-90 scale
|-
| colspan="4" | *Data from concurrent sensors may be optionally included e.g., salinity, dissolved oxygen. Kinematic viscosity of seawater needs to be calculated during processing,
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">
<br>
This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.
{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the ATOMIX Level 1 content.
| <blockquote>''This group includes the raw measurements from the recorder and ancillary measurements required for quality-controlling them using the manufacturer's recommendations.''</blockquote>
|-
| comment (optional)
| Information is pertinent to problems in the raw data files during collection.
| Examples: Stitching of files, corruption of binary files that were recovered by the manufacturer, etc.
|}
</div>

----
Return to [[Dataset requirements for ADCP structure function]]

Go to [[Level 2 data (velocity profilers)| Level 2 quality-controlled and segmented velocities]]

Deployment

2021-11-15T22:24:59Z

Jmmcmillan: Formatting update to remove unnecessary numbers

In order to collect useful measurements that '''actually resolve''' the turbulence statistics consistent with the application of the Kolmogorov hypotheses of [[isotropic turbulence]], it is important to configure and deploy your instrument using best practices. In setting up your instrument, it is recommended that consider the following recommendations:

=Environmental Conditions =
<div class="mw-collapsible mw-collapsed" id="env" data-collapsetext="Collapse" data-expandtext="Expand conditions">
* Ensure measurement velocity range is sufficient for anticipated background flow, tides, surface waves and internal waves
* For pulse-pulse coherent measurements, minimize potential issues due to phase wrapping by setting the ambiguity velocity to be larger than the maximum flow speed that is expected
* Ensure that spatial parameters (number of bins and bin size) are selected so that several bins are within the expected [[velocity inertial subrange model | inertial subrange]] that extends from the Kolmogorov scale [[Nomenclature| <math>L_K</math>]] to the Ozmidov scale [[Nomenclature| <math>L_o</math>]] . Use anticipated stratification and turbulence levels to determine <math>L_K</math> and <math>L_o</math> for the deployment location.
</div>

= Velocity Measurements =
<div class="mw-collapsible mw-collapsed" id="vel" data-collapsetext="Collapse" data-expandtext="Expand measurements">
* Record raw data in along-beam coordinates
* Maximise velocity accuracy whilst minimising averaging (pings per ensemble)
* If using a duty cycle, ensure that each burst is long enough to obtain stationary statistics necessary for <math>\varepsilon</math> estimates
* Maximise the number of profiles (ensembles) per <math>\varepsilon</math> estimate observation period to improve statistics
* Avoid/reduce interference with nearby instruments to reduce/avoid interference by sampling at different intervals.
* For instruments with an extra (vertical) beam, select the desired configuration of the angled beams
</div>

= Motion control during deployment =
<div class="mw-collapsible mw-collapsed" id="motioncontrol" data-collapsetext="Collapse" data-expandtext="Expand motion control">
* For bottom mounted instruments, minimise motion by ensuring that the frame is sufficiently heavy and streamlined to withstand the flow conditions at the deployment location
* For moored instruments, minimise motion by ensuring there is sufficient buoyancy on frame to hold position well. Also ensure that the buoyancy components do not obstruct beam path
* For moored instruments, consider impact of knock-down on location of observations in the water column when designing the mooring
* For moored instruments, collect depth and orientation data (heading, pitch and roll) at the same frequency as the velocity profiles data. Depth measurements could be made by the pressure sensor on ADCP or by adjacent instrument. Consider high resolution add-ons such as Altitude and Heading Reference Sensor (AHRS).
</div>

= Power and Storage for self-contained deployments =
<div class="mw-collapsible mw-collapsed" id="power-storage" data-collapsetext="Collapse" data-expandtext="Expand power and storage">
* Ensure the battery and memory capacity are sufficient for the planned deployment duration
* Factor in the expected water temperature when estimating the battery capacity and energy consumption
* Factor in the available memory and the manufacturer’s expected memory required per recorded profile (ensemble) when estimating the memory capacity
* For long deployments, extend the interval between observation periods (burst mode). This allows for longer duration deployments but reduces the temporal resolution of <math>\varepsilon</math> estimates
</div>

Next Step: [[ Raw data review (QA1) ]] <br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

Structure function empirical constant

2021-11-12T21:46:27Z

Jmmcmillan:

{{DefineConcept
|parameter_name=<math>C_2</math>
|description=The empirical constant relating the longitudinal structure function <math>D_{LL}</math> to the dissipation rate (<math>\varepsilon</math>)
|article_type=Fundamentals
|instrument_type=Velocity profilers
}}
Dimensional analysis can be used to show that <math>D_{LL}</math> must satisfy the "two-thirds law", i.e.,
<math>D_{LL}(r,t) = C_2\varepsilon^{2/3}r^{2/3}</math>
where <math>C_2</math> is a universal constant.

The value of the constant is generally accepted to be <math>2.1\pm 0.1</math>, based on the following studies:
# Sauvageot (1992)<ref name="Sauvageot">{{Cite journal
|authors= H. Sauvageot
|journal_or_publisher= Artech House
|paper_or_booktitle= Radar Meteorology
|year= 1992
}}</ref>: Used Doppler radar measurements of turbulence in the atmosphere to obtain a value of <math>2.0\pm 0.1</math>
# Saddoughi and Veeravalli (1994)<ref name="Saddoughi">{{Cite journal
|authors= K. R. Sreenivasan
|journal_or_publisher= J. Fluid Mech.
|paper_or_booktitle= Local isotropy in turbulent boundary layers at high Reynolds number
|year= 1994
|doi= https://doi.org/10.1017/S0022112094001370
}}</ref>: Used measurements in a wind tunnel to obtain a value of <math>2.1\pm 0.1</math>
# Sreenivasan (1995) <ref name="Sreenivasan">{{Cite journal
|authors= K. R. Sreenivasan
|journal_or_publisher= Phys. Fluids
|paper_or_booktitle= On the universality of the Kolmogorov constant
|year= 1995
|doi= 10.1063/1.868656
}}</ref>: Compiled the results from experimental studies of both grid turbulence and shear flows to conclude that a value of 2.0 agreed best with the spectral inertial subrange equation

== Notes ==

Structure function empirical constant

2021-11-12T21:30:40Z

Jmmcmillan: Created page with "{{DefineConcept |parameter_name=<math>C_2</math> |description=The empirical constant relating the longitudinal structure function <math>D_{LL}</math> to the dissipation rate (..."

Processing your ADCP data using structure function techniques

2021-11-12T21:03:27Z

Jmmcmillan:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:
# Extract or compute the [[along-beam bin center separation]] [<math>r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin, [<math>v’(n, t)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file)
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / r_0</math>]
# Calculate the structure function <math>D</math> for all possible bin separations <math>\delta</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme. Some things to consider are: [JMM: SHOULD WE INCLUDE THESE HERE, OR MAKE A LINK TO A PAGE ABOUT QUALITY CONTROL METRICS TO COMPUTE?]
#* Including <math>D(n,\delta)</math> for <math>\delta=1</math> may be inappropriate since the velocity estimates from adjacent bins are not wholly independent, therefore the impact of its inclusion should be evaluated
#* Keep a record of the number of instances when the squared velocity difference is evaluated for each bin <math>n</math> and separation distance <math>\delta r_{0}</math> and their distribution because they are potential quality control metrics
#* The impact of additional quality criteria can also be tested e.g. valid data requirements for all intermediate separation distances, so for a forward-difference scheme with <math>n=2</math> and <math>\delta=5</math>, require all data in bins 2 to 7 to meet Level 1 QC requirements for the profile to be included when averaging to calculate <math>D(n,\delta)</math>
# Perform a regression of <math>D(n,\delta)</math> against <math>(\delta r_0)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r<sub>0</sub> separation distances. [JMM: THE FOLLOWING ITEMS ARE CONFUSING. SINCE THIS IS BEST PRACTICE, CAN WE JUST RECOMMEND ONE METHOD?]
## If <math>D(n,\delta)</math> was evaluated using a forward-difference scheme, the regression is done for the combined data from all bins in the selected range, hence the maximum number of <math>D(n, \delta)</math> values for each separation distance will be the number of bins in the range less 1 for <math>\delta</math> = 1, reducing by 1 for each increment in <math>\delta</math>, with the regression ultimately yielding a single <math>\varepsilon</math> value for the data segment
## If <math>D(n,\delta)</math> was evaluated using a bin-centred difference scheme, the regression can either be done:
##* for each bin individually, with a single <math>D(n, \delta)</math> for each separation distance, ultimately yielding an <math>\varepsilon</math> for each bin; or
##* by combining the data for all of the bins, with each separation distance having a <math>D(n, \delta)</math> value for each bin, with the regression again ultimately yielding a single <math>\varepsilon</math> value for the data segment
## The regression is typically done as a least-squares fit, either as: <br /><br /> <math>D = a_0 + a_1 (\delta r_0)^{2/3}</math>; or as <br /> <math>D = a_0 + a_1 (\delta r_0)^{2/3}+a_3((\delta r_0)^{2/3})^3 </math> <br /><br /> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
# Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <br /><br /> <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> <br /><br /> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. [JMM: MOVE TO QA2 STEPS?]

PERHAPS WE CAN INCLUDE A FIGURE LIKE THIS TO HELP DEFINE VARIABLES.
[[File:ADCPschematic SF.png]]

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

Talk:Processing your ADCP data using structure function techniques

2021-11-12T20:54:17Z

Jmmcmillan:

[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:50, 12 November 2021 (CET) Can we change the title of this page to be more specific? Like "Computing the structure functions and dissipation rates"

[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:54, 12 November 2021 (CET) I find the use of <math>D(n,\delta)</math> a little confusing. In reality, for each range bin and each time ensemble, D only depends on <math>\delta</math>. Can we drop the n for simplicity?

Talk:Processing your ADCP data using structure function techniques

2021-11-12T20:50:27Z

Jmmcmillan: Created page with "~~~~ Can we change the title of this page to be more specific? Like "Computing the structure functions and dissipation rates""

Raw data review (QA1)

2021-11-12T20:43:15Z

Jmmcmillan:

== Raw data review ==

The objective of the raw data review is to ensure that the velocity data used for the calculation of the structure functions are of good quality before proceeding with the turbulence analysis. Bad data are typically identified from the velocity data themselves and other ancillary data (e.g. correlations). It is recommended that velocity data should be flagged if the following are observed:
# Data quality is poor
#* Low correlation values
#* Echo intensity anomalies
#* Low percent good values
#* Data return rate varies
# Unrealistic velocity values
#* Evidence of phase wrapping
#* Velocities outside nominal measurement range
#* Velocities outside expected distribution
# Significant instrument motion and orientation
#* High variability in pitch, heading and roll
#* Orientation deviates from expected values
# There is wave or periodic motion
#* Periodic motion at wave frequencies observed in velocity data
# Velocity shear is too large
#* earth velocities indicate significant horizontal shear
# Stationary assumption may be violated
#* variance of velocity bursts shows spatial or temporal trends [ALTHOUGH HARD TO DETERMINE WHAT IS REAL VARIABILITY]
# Stratification is too large
#* Temperature and salinity (if available) indicate local stratification
-----
Next step: [[ ]]<br></br>
Previous step: [[ Deployment ]] <br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Raw data review (QA1)

2021-11-12T20:41:43Z

Jmmcmillan:

== Raw data review ==

The objective of the raw data review is to ensure that the velocity data used for the calculation of the structure functions are of good quality before proceeding with the turbulence analysis. Bad data are typically identified from the velocity data themselves and other ancillary data (e.g. correlations). It is recommended that velocity data should be flagged if the following are observed:
# Data quality is poor
#* Low correlation values
#* Echo intensity anomalies
#* Low percent good values
#* Data return rate varies
# Unrealistic velocity values
#* Evidence of phase wrapping
#* Velocities outside nominal measurement range
#* Velocities outside expected distribution
# Significant instrument motion and orientation
#* High variability in pitch, heading and roll
#* Orientation deviates from expected values
# There is wave or periodic motion
#* Periodic motion at wave frequencies observed in velocity data
# Velocity shear is too large
#* earth velocities indicate significant horizontal shear
# Stationary assumption may be violated
#* variance of velocity bursts shows spatial or temporal trends [ALTHOUGH HARD TO DETERMINE WHAT IS REAL VARIABILITY]
# Stratification is too large
#* Temperature and salinity (if available) indicate local stratification
-----
Next step: [[ ]]
Previous step: [[ Deployment ]]
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Talk:Set structure function processing parameters

2021-11-12T20:40:41Z

Jmmcmillan:

[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:40, 12 November 2021 (CET) I think we should remove this section and incorporate into the next step.

Deployment

2021-11-12T20:37:44Z

Jmmcmillan:

In order to collect useful measurements that '''actually resolve''' the turbulence statistics consistent with the application of the Kolmogorov hypotheses of [[isotropic turbulence]], it is important to configure and deploy your instrument using best practices. In setting up your instrument, it is recommended that consider the following recommendations:

# Environmental conditions
#* Ensure measurement velocity range is sufficient for anticipated background flow, tides, surface waves and internal waves
#* For pulse-pulse coherent measurements, minimise potential issues due to phase wrapping by setting the ambiguity velocity to be larger than the maximum flow speed that is expected
#* Ensure that spatial parameters (number of bins and bin size) are selected so that several bins [SHOULD WE QUANTIFY THIS] are within the expected [[velocity inertial subrange model | inertial subrange]] that extends from the Kolmogorov scale [[Nomenclature| <math>L_K</math>]] to the Ozmidov scale [[Nomenclature| <math>L_o</math>]] [IS IT POSSIBLE TO LINK TO APPROPRIATE TABLE?]. Use anticipated stratification and turbulence levels to determine <math>L_K</math> and <math>L_o</math> for the deployment location.
# Velocity measurements
#* Record raw data in along-beam coordinates
#* Maximise velocity accuracy whilst minimising averaging (pings per ensemble) [(JMM): IS THERE A TYPICAL PINGS PER ENSEMBLE WE CAN RECOMMEND? (i.e. 1 or 2 are all I used at a high flow site)
#* If using a duty cycle, ensure that each burst is long enough to obtain stationary statistics necessary for <math>\varepsilon</math> estimates [(JMM) I REWORDED THIS FROM '<math>\varepsilon</math> estimate observation period', but I don't know if it is any less confisuing]
#* Maximise the number of profiles (ensembles) per <math>\varepsilon</math> estimate observation period to improve statistics [(JMM) IS THIS ONLY DETERMINED BY THE SAMPLE RATE?]
#* Avoid/reduce interference with nearby instruments to reduce/avoid interference by sampling at different intervals.
#* For instruments with an extra (vertical) beam, select the desired configuration of the angled beams [(JMM) CAN WE BE MORE SPECIFIC HERE? ]]
# Motion control during deployment
#* For bottom mounted instruments, minimise motion by ensuring that the frame is sufficiently heavy and streamlined to withstand the flow conditions at the deployment location
#* For moored instruments, minimise motion by ensuring there is sufficient buoyancy on frame to hold position well. Also ensure that the buoyancy components do not obstruct beam path
#* For moored instruments, consider impact of knock-down on location of observations in the water column when designing the mooring
#* For moored instruments, collect depth and orientation data (heading, pitch and roll) at the same frequency as the velocity profiles data. Depth measurements could be made by the pressure sensor on ADCP or by adjacent instrument. Consider high resolution add-on’s such as AHRS. [(JMM) WHAT IS THIS ACRONYM]
# Power and Storage for self-contained deployments
#* Ensure the battery and memory capacity are sufficient for the planned deployment duration
#* Factor in the expected water temperature when estimating the battery capacity and energy consumption
#* Factor in the available memory and the manufacturer’s expected memory required per recorded profile (ensemble) when estimating the memory capacity
#* For long deployments, extend the interval between observation periods (burst mode). This allows for longer duration deployments but reduces the temporal resolution of <math>\varepsilon</math> estimates

'''SUGGEST REMOVING BOTH OF THE FOLLOWING PAGES'''

# [[ADCP Hardware]]
# [[ADCP Environment]]

Next Step: [[ Raw data review (QA1) ]] <br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

Deployment

2021-11-12T20:36:21Z

Jmmcmillan:

In order to collect useful measurements that '''actually resolve''' the turbulence statistics consistent with the application of the Kolmogorov hypotheses of [[isotropic turbulence]], it is important to configure and deploy your instrument using best practices. In setting up your instrument, it is recommended that consider the following recommendations:

# Environmental conditions
#* Ensure measurement velocity range is sufficient for anticipated background flow, tides, surface waves and internal waves
#* For pulse-pulse coherent measurements, minimise potential issues due to phase wrapping by setting the ambiguity velocity to be larger than the maximum flow speed that is expected
#* Ensure that spatial parameters (number of bins and bin size) are selected so that several bins [SHOULD WE QUANTIFY THIS] are within the expected [[velocity inertial subrange model | inertial subrange]] that extends from the Kolmogorov scale [[Nomenclature| <math>L_K</math>]] to the Ozmidov scale [[Nomenclature| <math>L_o</math>]] [IS IT POSSIBLE TO LINK TO APPROPRIATE TABLE?]. Use anticipated stratification and turbulence levels to determine <math>L_K</math> and <math>L_o</math> for the deployment location.
# Velocity measurements
#* Record raw data in along-beam coordinates
#* Maximise velocity accuracy whilst minimising averaging (pings per ensemble) [(JMM): IS THERE A TYPICAL PINGS PER ENSEMBLE WE CAN RECOMMEND? (i.e. 1 or 2 are all I used at a high flow site)
#* If using a duty cycle, ensure that each burst is long enough to obtain stationary statistics necessary for <math>\varepsilon</math> estimates [(JMM) I REWORDED THIS FROM '<math>\varepsilon</math> estimate observation period', but I don't know if it is any less confisuing]
#* Maximise the number of profiles (ensembles) per <math>\varepsilon</math> estimate observation period to improve statistics [(JMM) IS THIS ONLY DETERMINED BY THE SAMPLE RATE?]
#* Avoid/reduce interference with nearby instruments to reduce/avoid interference by sampling at different intervals.
#* For instruments with an extra (vertical) beam, select the desired configuration of the angled beams [(JMM) CAN WE BE MORE SPECIFIC HERE? ]]
# Motion control during deployment
#* For bottom mounted instruments, minimise motion by ensuring that the frame is sufficiently heavy and streamlined to withstand the flow conditions at the deployment location
#* For moored instruments, minimise motion by ensuring there is sufficient buoyancy on frame to hold position well. Also ensure that the buoyancy components do not obstruct beam path
#* For moored instruments, consider impact of knock-down on location of observations in the water column when designing the mooring
#* For moored instruments, collect depth and orientation data (heading, pitch and roll) at the same frequency as the velocity profiles data. Depth measurements could be made by the pressure sensor on ADCP or by adjacent instrument. Consider high resolution add-on’s such as AHRS. [(JMM) WHAT IS THIS ACRONYM]
# Power and Storage for self-contained deployments
#* Ensure the battery and memory capacity are sufficient for the planned deployment duration
#* Factor in the expected water temperature when estimating the battery capacity and energy consumption
#* Factor in the available memory and the manufacturer’s expected memory required per recorded profile (ensemble) when estimating the memory capacity
#* For long deployments, extend the interval between observation periods (burst mode). This allows for longer duration deployments but reduces the temporal resolution of <math>\varepsilon</math> estimates

'''SUGGEST REMOVING BOTH OF THE FOLLOWING PAGES'''

# [[ADCP Hardware]]
# [[ADCP Environment]]

Proceed to [[ Raw data review (QA1) ]]

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

Deployment

2021-11-12T20:35:57Z

Jmmcmillan:

In order to collect useful measurements that '''actually resolve''' the turbulence statistics consistent with the application of the Kolmogorov hypotheses of [[isotropic turbulence]], it is important to configure and deploy your instrument using best practices. In setting up your instrument, it is recommended that consider the following recommendations:

# Environmental conditions
#* Ensure measurement velocity range is sufficient for anticipated background flow, tides, surface waves and internal waves
#* For pulse-pulse coherent measurements, minimise potential issues due to phase wrapping by setting the ambiguity velocity to be larger than the maximum flow speed that is expected
#* Ensure that spatial parameters (number of bins and bin size) are selected so that several bins [SHOULD WE QUANTIFY THIS] are within the expected [[velocity inertial subrange model | inertial subrange]] that extends from the Kolmogorov scale [[Nomenclature| <math>L_K</math>]] to the Ozmidov scale [[Nomenclature| <math>L_o</math>]] [IS IT POSSIBLE TO LINK TO APPROPRIATE TABLE?]. Use anticipated stratification and turbulence levels to determine <math>L_K</math> and <math>L_o</math> for the deployment location.
# Velocity measurements
#* Record raw data in along-beam coordinates
#* Maximise velocity accuracy whilst minimising averaging (pings per ensemble) [(JMM): IS THERE A TYPICAL PINGS PER ENSEMBLE WE CAN RECOMMEND? (i.e. 1 or 2 are all I used at a high flow site)
#* If using a duty cycle, ensure that each burst is long enough to obtain stationary statistics necessary for <math>\varepsilon</math> estimates [(JMM) I REWORDED THIS FROM '<math>\varepsilon</math> estimate observation period', but I don't know if it is any less confisuing]
#* Maximise the number of profiles (ensembles) per <math>\varepsilon</math> estimate observation period to improve statistics [(JMM) IS THIS ONLY DETERMINED BY THE SAMPLE RATE?]
#* Avoid/reduce interference with nearby instruments to reduce/avoid interference by sampling at different intervals.
#* For instruments with an extra (vertical) beam, select the desired configuration of the angled beams [(JMM) CAN WE BE MORE SPECIFIC HERE? ]]
# Motion control during deployment
#* For bottom mounted instruments, minimise motion by ensuring that the frame is sufficiently heavy and streamlined to withstand the flow conditions at the deployment location
#* For moored instruments, minimise motion by ensuring there is sufficient buoyancy on frame to hold position well. Also ensure that the buoyancy components do not obstruct beam path
#* For moored instruments, consider impact of knock-down on location of observations in the water column when designing the mooring
#* For moored instruments, collect depth and orientation data (heading, pitch and roll) at the same frequency as the velocity profiles data. Depth measurements could be made by the pressure sensor on ADCP or by adjacent instrument. Consider high resolution add-on’s such as AHRS. [(JMM) WHAT IS THIS ACRONYM]
# Power and Storage for self-contained deployments
#* Ensure the battery and memory capacity are sufficient for the planned deployment duration
#* Factor in the expected water temperature when estimating the battery capacity and energy consumption
#* Factor in the available memory and the manufacturer’s expected memory required per recorded profile (ensemble) when estimating the memory capacity
#* For long deployments, extend the interval between observation periods (burst mode). This allows for longer duration deployments but reduces the temporal resolution of <math>\varepsilon</math> estimates

'''SUGGEST REMOVING BOTH OF THE FOLLOWING PAGES'''

# [[ADCP Hardware]]
# [[ADCP Environment]]

Proceed to [[ Raw data review (QA1) ]]
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

Raw data review (QA1)

2021-11-12T20:32:11Z

Jmmcmillan:

== Raw data review ==

The objective of the raw data review is to ensure that the velocity data used for the calculation of the structure functions are of good quality before proceeding with the turbulence analysis. Bad data are typically identified from the velocity data themselves and other ancillary data (e.g. correlations). It is recommended that velocity data should be flagged if the following are observed:
# Data quality is poor
#* Low correlation values
#* Echo intensity anomalies
#* Low percent good values
#* Data return rate varies
# Unrealistic velocity values
#* Evidence of phase wrapping
#* Velocities outside nominal measurement range
#* Velocities outside expected distribution
# Significant instrument motion and orientation
#* High variability in pitch, heading and roll
#* Orientation deviates from expected values
# There is wave or periodic motion
#* Periodic motion at wave frequencies observed in velocity data
# Velocity shear is too large
#* earth velocities indicate significant horizontal shear
# Stationary assumption may be violated
#* variance of velocity bursts shows spatial or temporal trends [ALTHOUGH HARD TO DETERMINE WHAT IS REAL VARIABILITY]
# Stratification is too large
#* Temperature and salinity (if available) indicate local stratification
-----
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Talk:Raw data review (QA1)

2021-11-12T20:31:05Z

Jmmcmillan:

If I was new to ADCP data processing for turbulence, I think I would find this list a little confusing in the way it is currently organized. It might be easier list the QC criteria by the issue instead of data type. e.g.

It is recommended that velocity data should be flagged or removed if the following are observed:
# Data quality is poor
#* Low correlation values
#* Echo intensity anomalies
#* Low percent good values
#* Data return rate varies
# Unrealistic velocity values
#* Evidence of phase wrapping
#* Velocities outside nominal measurement range
#* Velocities outside expected distribution
# Significant instrument motion and orientation
#* High variability in pitch, heading and roll
#* Orientation deviates from expected values
# There is wave or periodic motion
#* Periodic motion at wave frequencies observed in velocity data
# Velocity shear is too large
#* earth velocities indicate significant horizontal shear
# Stationary assumption may be violated
#* variance of velocity bursts shows spatial or temporal trends [ALTHOUGH HARD TO DETERMINE WHAT IS REAL VARIABILITY]
# Stratification is too large
#* Temperature and salinity (if available) indicate local stratification

[[User:Brian scannell|Brian scannell]] ([[User talk:Brian scannell|talk]]) 15:58, 12 November 2021 (CET)
Agree that this is better. I guess users would like to be told the “right" criteria thresholds and “expected values” to use, but is best practice sufficiently well defined to make pronouncements on this? Personally, I wouldn’t feel comfortable doing so. Probably best not to refer to removing data - the netcdf approach just flags it as being bad data - is this the place to describe the QC criteria? Brian

[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:31, 12 November 2021 (CET) I don't think we need to give the thresholds because it is instrument dependent and configuration dependent. We are just telling them what things might possibly contaminate the data. We can link to other pages if we want to include more description and/or references. Since we are in agreement of the format, I will drop the "remove" and put this list directly on the page.

File talk:InertialSubrange.png

2021-11-12T16:58:00Z

Jmmcmillan: Created page with "File can be deleted. New file called 'InertialSubrangeSchematic.png' should be used instead.( ~~~~JMM)"

File can be deleted. New file called 'InertialSubrangeSchematic.png' should be used instead.( [[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 17:58, 12 November 2021 (CET)JMM)

Velocity inertial subrange model

2021-11-12T16:49:44Z

Jmmcmillan:

{{DefineConcept
|description=The inertial subrange separates the energy-containing production range from the viscous dissipation range.
|article_type=Fundamentals
|instrument_type=Velocity point-measurements, Velocity profilers
}}
== Inertial subrange for steady-flows ==
This theoretical model predicts the spectral shape of velocities in wavenumber space.

<math>\Psi_{Vj}(\hat{k})=a_jC_k\varepsilon^{2/3}\hat{k}^{-5/3}</math>

[[File:InertialSubrangeSchematic.png|thumb|Sketch of velocity power density spectrum in log-log space. The inertial subrange's -5/3 slope is highlighted. The vertical axis represents <math>\Psi_{Vj}(\hat{k})</math>. [[Large-scale turbulence anisotropy]] in low energy flow may alter the expected spectral shape]]

Here <math>\hat{k}</math> is expressed in rad/m and <math>Vj</math> represents the velocities <math>V</math> in direction <math>j</math>. <math>C_k</math> is the empirical Kolmogorov universal constant of C
= 1.5 <ref name="Sreenivasan">{{Cite journal
|authors= K. R. Sreenivasan
|journal_or_publisher= Phys. Fluids
|paper_or_booktitle= On the universality of the Kolmogorov constant
|year= 1995
|doi= 10.1063/1.868656
}}</ref>. Amongst the three direction, the spectra deviates by the constant <math>a_j</math>:
<ref name="Pope">{{Cite journal
|authors= S.B Pope
|journal_or_publisher= Cambridge Univ. Press
|paper_or_booktitle= Turbulent flows
|year= 2000
|doi= 10.1017/CBO9780511840531
}}</ref>

* In the longitudinal direction, i.e., the direction of mean advection (j=1), <math>a_1=\frac{18}{55}</math>
* In the other directions <math>a_2=a_3=\frac{4}{3}a_1</math>

== Inertial subrange for flows influenced by surface waves ==
Need to add equations and figures from Lumley & Terray<ref name="Lumley_Terray">
{{Cite journal
|authors=J. Lumley and E. Terray
|journal_or_publisher=J. Phys. Oceanogr
|paper_or_booktitle=Kinematics of turbulence convected by a random wave field
|year=1983
|doi= 10.1175/1520-0485(1983)<2000:KOTCBA>2.0.CO;2
}}
</ref>

== Notes ==

File:InertialSubrangeSchematic.png

2021-11-12T16:49:07Z

Jmmcmillan: Schematic of the velocity spectra in the inertial subrange

== Summary ==
Schematic of the velocity spectra in the inertial subrange

Velocity inertial subrange model

2021-11-12T16:45:56Z

Jmmcmillan:

{{DefineConcept
|description=The inertial subrange separates the energy-containing production range from the viscous dissipation range.
|article_type=Fundamentals
|instrument_type=Velocity point-measurements, Velocity profilers
}}
== Inertial subrange for steady-flows ==
This theoretical model predicts the spectral shape of velocities in wavenumber space.

<math>\Psi_{Vj}(\hat{k})=a_jC_k\varepsilon^{2/3}\hat{k}^{-5/3}</math>

[[File:InertialSubrange.png|thumb|Sketch of velocity power density spectrum in log-log space. The inertial subrange's -5/3 slope is highlighted. The vertical axis represents <math>\Psi_{Vj}(\hat{k})</math>. [[Large-scale turbulence anisotropy]] in low energy flow may alter the expected spectral shape]]

Here <math>\hat{k}</math> is expressed in rad/m and <math>Vj</math> represents the velocities <math>V</math> in direction <math>j</math>. <math>C_k</math> is the empirical Kolmogorov universal constant of C
= 1.5 <ref name="Sreenivasan">{{Cite journal
|authors= K. R. Sreenivasan
|journal_or_publisher= Phys. Fluids
|paper_or_booktitle= On the universality of the Kolmogorov constant
|year= 1995
|doi= 10.1063/1.868656
}}</ref>. Amongst the three direction, the spectra deviates by the constant <math>a_j</math>:
<ref name="Pope">{{Cite journal
|authors= S.B Pope
|journal_or_publisher= Cambridge Univ. Press
|paper_or_booktitle= Turbulent flows
|year= 2000
|doi= 10.1017/CBO9780511840531
}}</ref>

* In the longitudinal direction, i.e., the direction of mean advection (j=1), <math>a_1=\frac{18}{55}</math>
* In the other directions <math>a_2=a_3=\frac{4}{3}a_1</math>

== Inertial subrange for flows influenced by surface waves ==
Need to add equations and figures from Lumley & Terray<ref name="Lumley_Terray">
{{Cite journal
|authors=J. Lumley and E. Terray
|journal_or_publisher=J. Phys. Oceanogr
|paper_or_booktitle=Kinematics of turbulence convected by a random wave field
|year=1983
|doi= 10.1175/1520-0485(1983)<2000:KOTCBA>2.0.CO;2
}}
</ref>

== Notes ==