Atomix - User contributions [en]

Benchmark datasets for ADCP structure function

2025-02-18T16:50:01Z

Yuengdjern:

deployment This page provides an overview of the benchmark dataset for instruments that measure velocity profiles e.g., [[Acoustic-Doppler Current Profilers|acoustic-Doppler current profilers]] from diverse suppliers and models.

== Datasets available ==
Selection and preparation of benchmark datasets are a work in progress! These benchmark datasets will cover a range of marine environments, background stratification, and flow fields. They are temporarily in this [http://gofile.me/5BB4l/sApY8EvAp|read-only folder] and will eventually be housed at the BODC center with a DOI.

{| class="wikitable"
|+Summary of potential benchmark datasets for testing existing and future algorithms
|-
! style="width: 12%"| Dataset name
! Instrument
& configuration
!Deployment mode
! Total depth
! Deployment height above bottom
! Background speed
! <math>\varepsilon</math> range
! Stratification/shear information
! style="width: 20%"| Comment
|-
!
! Make and Model
!Broadband(BB)
Pulse coherent (PPC)
! [m]
! [m]
! [m/s]
! [W/kg]
!
!
|-
| MS
| Nortek Signature1000
bedframe
|BB
8 Hz

Burst mode
| 13
| 0.5
| <math>\lesssim</math> 0.3
| 1e-7 to 1e-4
| unstratified tidal channel
| pretty average
|
|-
| GP
| RDI Teledyne 600kHz
|BB
1.6 Hz

Mode 0

2 subpings
| 23
| 0.5
| 2.5
| 7e-6 to 2e-4
| unstratified tidal channel
| high quality data
|
|-
| CS_TOP
| RDI Workhorse 600 kHz
mooring
|PPC
1Hz
| 145
| 125
|0.5
| 1e-7 to 1e-5
| region of surface wave influence
| good
|
|-
| CS_BED
| RDI Workhorse 600kHz
bed frame
|BB
1Hz

Mode 12

4 subpings
| 145
| 1
|0.5
| 1e-7
| bottom boundary layer on NW European shelf
| good
|
|-
|NS
|Nortek Aquadopp
2 MHz

bedframe
|PPC
8 Hz
|73
|1.4
|0.03
|1e-8 to 1e-5
|ocean bottom boundary layer
|
|
|-
| LW
| Nortek Aquadopp 1MHz
BEDFRAME
|PPC
2 Hz
| 40
| 0.4
| <0.1
| 1e-9 to 1e-8
| lake bottom boundary layer
| good
|
|}

-------------------------
return to [[Velocity profilers| Velocity Profilers Welcome Page]]
[[Category:Velocity profilers]]

Benchmark datasets for ADCP structure function

2025-02-18T16:48:59Z

Yuengdjern:

deployment This page provides an overview of the benchmark dataset for instruments that measure velocity profiles e.g., [[Acoustic-Doppler Current Profilers|acoustic-Doppler current profilers]] from diverse suppliers and models.

== Datasets available ==
Selection and preparation of benchmark datasets are a work in progress! These benchmark datasets will cover a range of marine environments, background stratification, and flow fields. They are temporarily in this [http://gofile.me/5BB4l/sApY8EvAp|read-only folder] and will eventually be housed at the BODC center with a DOI.

{| class="wikitable"
|+Summary of potential benchmark datasets for testing existing and future algorithms
|-
! style="width: 12%"| Dataset name
! Instrument
& configuration
!Deployment mode
! Total depth
! Deployment height above bottom
! Background speed
! <math>\varepsilon</math> range
! Stratification/shear information
! style="width: 20%"| Comment
|-
!
! Make and Model
!Broadband(BB)
Pulse coherent (PPC)
! [m]
! [m]
! [m/s]
! [W/kg]
!
!
|-
| MS
| Nortek Signature1000
bedframe
|BB
Hz
Burst mode
| 13
| 0.5
| <math>\lesssim</math> 0.3
| 1e-7 to 1e-4
| unstratified tidal channel
| pretty average
|
|-
| GP
| RDI Teledyne 600kHz
|BB
1.6 Hz
Mode 0
2 subpings
| 23
| 0.5
| 2.5
| 7e-6 to 2e-4
| unstratified tidal channel
| high quality data
|
|-
| CS_TOP
| RDI Workhorse 600 kHz
mooring
|PPC
1Hz
| 145
| 125
|0.5
| 1e-7 to 1e-5
| region of surface wave influence
| good
|
|-
| CS_BED
| RDI Workhorse 600kHz
bed frame
|BB
1Hz

Mode 12

4 subpings
| 145
| 1
|0.5
| 1e-7
| bottom boundary layer on NW European shelf
| good
|
|-
|NS
|Nortek Aquadopp
2 MHz

bedframe
|PPC
8 Hz
|73
|1.4
|0.03
|1e-8 to 1e-5
|ocean bottom boundary layer
|
|
|-
| LW
| Nortek Aquadopp 1MHz
BEDFRAME
|PPC
2 Hz
| 40
| 0.4
| <0.1
| 1e-9 to 1e-8
| lake bottom boundary layer
| good
|
|}

-------------------------
return to [[Velocity profilers| Velocity Profilers Welcome Page]]
[[Category:Velocity profilers]]

Benchmark datasets for ADCP structure function

2025-02-18T16:47:32Z

Yuengdjern: updated benchmark datasets

deployment This page provides an overview of the benchmark dataset for instruments that measure velocity profiles e.g., [[Acoustic-Doppler Current Profilers|acoustic-Doppler current profilers]] from diverse suppliers and models.

== Datasets available ==
Selection and preparation of benchmark datasets are a work in progress! These benchmark datasets will cover a range of marine environments, background stratification, and flow fields. They are temporarily in this [http://gofile.me/5BB4l/sApY8EvAp|read-only folder] and will eventually be housed at the BODC center with a DOI.

{| class="wikitable"
|+Summary of potential benchmark datasets for testing existing and future algorithms
|-
! style="width: 12%"| Dataset name
! Instrument
& configuration
!Deployment mode
! Total depth
! Deployment height above bottom
! Background speed
! <math>\varepsilon</math> range
! Stratification/shear information
! style="width: 20%"| Comment
|-
!
! Make and Model
!Broadband(BB)
Pulse coherent (PPC)
! [m]
! [m]
! [m/s]
! [W/kg]
!
!
|-
| MS
| Nortek Signature1000
bedframe
|BB
Hz
Burst mode
| 13
| 0.5
| <math>\lesssim</math> 0.3
| 1e-7 to 1e-4
| unstratified tidal channel
| pretty average
|
|-
| GP
| RDI Teledyne 600kHz
|BB
1.6 Hz
Mode 0
2 subpings
| 23
| 0.5
| 2.5
| 7e-6 to 2e-4
| unstratified tidal channel
| high quality data
|
|-
| CS_TOP
| RDI Workhorse 600 kHz
mooring
|PPC
1Hz
| 145
| 125
|0.5
| 1e-7 to 1e-5
| region of surface wave influence
| good
|
|-
| CS_BED
| RDI Workhorse 600kHz
bed frame
|BB
1Hz
Mode 12
4 subpings
| 145
| 1
|0.5
| 1e-7
| bottom boundary layer on NW European shelf
| good
|
|-
|NS
|Nortek Aquadopp
2 MHz
bedframe
|PPC
8 Hz
|73
|1.4
|0.03
|1e-8 to 1e-5
|ocean bottom boundary layer
|
|
|-
| LW
| Nortek Aquadopp 1MHz
BEDFRAME
|PPC
2 Hz
| 40
| 0.4
| <0.1
| 1e-9 to 1e-8
| lake bottom boundary layer
| good
|
|}

-------------------------
return to [[Velocity profilers| Velocity Profilers Welcome Page]]
[[Category:Velocity profilers]]

Main Page

2024-07-23T12:51:15Z

Yuengdjern: added hyperlinks to Chairs external webpages and updated full member lists to reflect Toshi's retirement.

__NOTOC__
== Welcome to the ATOMIX wiki! ==
This wiki was initially developed by the [https://scor-int.org/group/analysing-ocean-turbulence-observations-to-quantify-mixing-atomix/ SCOR working group 160] - Analysing ocean turbulence observations to quantify mixing (ATOMIX). It is meant to consolidate knowledge about oceanic turbulence data acquisition and processing, and to provide guidelines for processing and publication of data sets aimed at both experienced and new users.

[https://landing.mailerlite.com/webforms/landing/e4j3z7 Please sign up to our mailing list!]

== Wiki structure ==
This wiki is organized to provide a best practice for turbulence data processing from the three main instruments used for this purpose. Below, you can select your preferred device and navigate to the corresponding subpage.

{|class="wikitable" style="margin-left: auto; margin-right: auto; border: none; text-align:center;"
|[[File:VMP500-CookSt.JPG|250px|link=Shear probes|Shear probes]]
|[[File:AdcpMTOrig.png|250px|link=Velocity profilers|Velocity profilers]]
|[[File:TIC Leopold 1.jpg|120px|link=Velocity point-measurements|Velocity point-measurements]]
|-
|[[Shear probes]]
|[[Velocity profilers]]
|[[Velocity point-measurements]]
|}

These three individual processing guidelines share the same [[Nomenclature]] and [[NetCDF parameter]] attributes.

== Turbulence Sensing Gallery ==
We also have the [[ATOMIX Gallery]] with images illustrating various sampling configurations and instruments.

== Members of the SCOR Working Group ATOMIX ==
Chairs: <br>
[https://www.researchgate.net/profile/C-Bluteau-2 Cynthia Bluteau (Canada)], [https://www4.uib.no/en/find-employees/Ilker.Fer Ilker Fer (Norway)], [https://www.bangor.ac.uk/staff/sos/yueng-djern-lenn-016420/en Yueng-Djern Lenn (UK)] <br><br>

Other Full Members:<br>
Toshiyuki Hibiya (Japan, 2020-2024), Arnaud LeBoyer (USA), Zhiyu Liu (China), Rolf Lueck (Canada), Amelie Meyer (Australia), Craig Stevens (New Zealand), Danielle Wain (USA), Ryuichiro Inoue (Japan, 2024-)<br><br>

Associate Members:<br>
Marcus Dengler (Germany), Jenson George (India), Peter Holtermann (Germany), Natasha Lucas (UK), Justine McMillan (Canada), Stephen Monismith (USA), Julia Mullarney (New Zealand), Ryuichiro Inoue (Japan, 2020-2024), Sarah Nicholson (South Africa), Kirstin Schulz (USA)<br><br>

== For new ATOMIX wiki users ==
This wiki is meant to be dynamic and grow with the advances achieved in the scientific turbulence community. To be successful, your active contribution to this effort is needed!
* [[The scope and approach of the ATOMIX wiki]].
* Please visit the [[How to use and contribute|contribute]] page to learn how to comment and edit these pages.
* Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.

Benchmark datasets for ADCP structure function

2023-01-20T14:55:42Z

Yuengdjern:

This page provides an overview of the benchmark dataset for instruments that measure velocity profiles e.g., [[Acoustic-Doppler Current Profilers|acoustic-Doppler current profilers]] from diverse suppliers and models.

== Datasets available ==
Selection and preparation of benchmark datasets are a work in progress! These benchmark datasets will cover a range of marine environments, background stratification, and flow fields. They are temporarily in a [https://www.dropbox.com/sh/06zw92s6kvsa3uc/AAC6AT0gkEe8dDAmLefbP2LTa?dl=0|read-only dropbox folder], and will eventually be housed at the BODC center with a DOI.

{| class="wikitable"
|+Summary of potential benchmark datasets for testing existing and future algorithms
|-
! style="width: 12%"| Dataset name
! Instrument
! Total depth
! Deployment height above bottom
! Background speed
! <math>\varepsilon</math> range
! Stratification/shear information
! style="width: 20%"| Comment
|-
!
! Make and Model
! [m]
! [m]
! [m/s]
! [W/kg]
!
!
|-
| Signature5beam_TidalShelf
| Nortek Signature1000
| 254
| 0.3
| <math>\lesssim</math> 0.3
|
| stratified boundary layer
| pretty average
|
|-
| RDI4beam_TidalChannel_GP130620BPb
| RDI Teledyne 600kHz
| 23
| 0.5
| 2.5
| 7e-6 to 2e-4
| unstratified
| high quality data
|
|-
| RDIWH600_CANDYFLOSS_TOP
| RDI Workhorse 600 kHz
| 145
| 125
|
| 1e-7
| region of surface wave influence
| good
|
|-
| RDIWH600_CANDYFLOSS_BEDFRAME
| RDI Workhorse 600kHz
| 145
| 1
|
| 1e-7
| bottom boundary layer on NW European shelf
| good
|
|-
| AQD_Windermere_BEDFRAME
| Nortek Aquadopp 1MHz
| 40
| 0.4
| <0.1
| 1e-9 to 1e-8
| lake bottom boundary layer
| good
|
|}

-------------------------
return to [[Velocity profilers| Velocity Profilers Welcome Page]]
[[Category:Velocity profilers]]

Velocity Profiler data flags

2022-07-11T15:53:55Z

Yuengdjern:

'''How ADCP structure function quality-control flags are applied'''

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:

'''Level 2:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| segment_outlier
| Flag data points <math> > 3 \sigma</math> away from segment mean for bin.
| 3 \sigma</math>
| 1
| 2
|-
| 4
| Bit 2
| profile_outlier
| Flag data points <math> > 3 \sigma</math> away from beam-profile-mean.
| 3 \sigma</math>
| 0
| 0
|-
|
|
|
|
|
|
| Final Q = 2
|}

'''Level 3:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| insufficient_number_velocity_samples
| Flag segments where the ratio of the number of viable values in the velocity difference to the total number of data points is less than the threshold
| 0.5
| 0
| 0
|-
| 4
| Bit 2
| too_close_to_range_limits
| Flag bins that are too close to the first or last range bin (I.e. not enough bins to do differencing)
| 3
| 1
| 4
|-
|
|
|
|
|
|
| Final Q = 4
|}

'''Level 4:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| Rsquared_too_low
| Flag values where <math> R^2 < </math> threshold
| 0.6
| 0
| 0
|-
| 4
| Bit 2
| delta_epsi_too_large
| Flag values where <math> \delta \epsilon / \epsilon > </math> threshold
| 0.6
| 1
| 4
|-
| 8
| Bit 3
| A3_coeff_invalid
| Flag values where <math> a_3 </math> < threshold
| 0
| 0
| 0
|-
| 16
| Bit 4
| dll_intercept_too_low
| Flag values where intercept (<math> a_0 </math>) < threshold
| 0
| 1
| 16
|-
| 32
| Bit 5
| dll_intercept_too_high
| Flag values where intercept > threshold (threshhold <math> = 2*yint\_{expected} = 2*(2*\sigma_v^2) </math>, where <math> \sigma_v </math> is expected accuracy in along-beam vel.
| Instrument dependent
| 0
| 0
|-
| 64
| Bit 6
| regression_poorly_conditioned
| Flag values where there are fewer than a certain number of points for the regression
| 3
| 0
| 0
|-
| 128
| Bit 7
| dll_slope_out_of_range
| Flag values where slope <math> a_1 < 0 </math> as this would lead to negative epsilon.
| 0
| 0
| 0
|-
|
|
|
|
|
|
| Final Q = 20
|}

<br />

The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.
Note that importance (severity of the failure) is ranked by the numeric value of the flag mask.

Level 4 data (velocity profilers)

2022-06-04T10:45:36Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]).
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profiler data flags | Velocity Profiler data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]])
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-06-04T10:41:56Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]).
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profilers data flags | Velocity Profilers data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]])
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-06-04T10:38:40Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]).
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profilers dat flags | Velocity Profilers data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]])
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-06-04T10:35:30Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients (see [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]).
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profilers dat flags | Velocity Profilers data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. {{FontColor|fg=red|text=Provide link.}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-06-04T10:32:14Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| For details see [[Velocity Profilers dat flags | Velocity Profilers data flags]]
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. {{FontColor|fg=red|text=Provide link.}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 3 data (velocity profilers)

2022-06-04T10:24:58Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
[[File:SF atomix ADCP.png|300px|thumb|Schematic of ADCP processing nomenclature]]
The required dimensions and variables for the structure-function processing level within NetCDF ATOMIX format for velocity ADCP measurements are described below. This NetCDF group contains the structure function (DLL) calculated as a function of the along-beam separation for the available/usable ADCP bins.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Segment midpoint time. Units in Days since reference time specified in variable attribute.
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_DEL
| along-beam_separation_distance_over_
which_DLL_is_evaluated_in_number_of_bins
| N_DEL
| Number of bins separating two velocity measurements used to calculate DLL
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| DLL
| second_order_structure_function
| TIME, Z_DIST, N_BEAM, N_DEL
| Differences in velocities squared have been time-averaged (units of m2/s2).
|-
| DLL_FLAGS
| second_order_structure_function_
status_flag
| TIME, Z_DIST, N_BEAM, N_DEL
| For details see [[Velocity Profiler data flags| Velocity profiler data flags]]
|-
| R_DEL
| along-beam_separation_distance_at_
which_structure_function_is_evaluated
| N_BEAM, N_DEL
| Estimated quantity (in meters) from N_DEL (Level 3), BIN_SIZE (Level 2) and THETA (Level 2).
|-
| R_DIST
| distance_from_sensor_along_beams
| Z_DIST, N_BEAM
| Along-beam bin centre distance (in meters) from the transducer
|-
| DLL_N
| second_order_structure_function_number_
of_observations
| TIME, Z_DIST, N_BEAM, N_DEL
| The number of available measurements in each segment i.e., data quality.
|-
| N_SEGMENT
| unique_identifier_for_each_segment_
in_the_entire_available_timeseries
| TIME
| Enables backtracking to [[Level 2 segmented (velocity profilers)|previous processing level]]
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the structure function Dll as a function of the separation distance. Any ancillary information required for estimating the dissipation of turbulent kinetic energy may also be stored here. ''</blockquote>
|-
| dll_calculation_type
| Specify differencing technique used to estimate DLL
| Examples include:
<blockquote>Central-differencing</blockquote>
<blockquote>Forward-differencing</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes <math>\ddagger</math>
|-
| stationarity_testing
| Any testing done on the segment to verify stationarity?
| {{FontColor|fg=white|bg=red|text=To be revisited once testing begins}}. Tentatively refer to [https://bitbucket.org/efm_cb/netcdf/src/master/TestData/adcp_atomix_metada.yml demo yaml] file.
|-
| noise_testing
| Details of testing the noise levels, or if the signal comprises mostly of noise?
|
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 2 data (velocity profilers)| Level 2 segmented velocities]]

Go to [[Level 4 data (velocity profilers)| Level 4 dissipation estimates]]

Velocity Profiler data flags

2022-06-03T21:39:20Z

Yuengdjern:

'''How ADCP structure function quality-control flags are applied'''

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:

'''Level 2:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| segment_outlier
| Flag data points <math> > 3 \sigma</math> away from segment mean for bin.
| 3 \sigma</math>
| 1
| 2
|-
| 4
| Bit 2
| profile_outlier
| Flag data points <math> > 3 \sigma</math> away from beam-profile-mean.
| 3 \sigma</math>
| 0
| 0
|-
|
|
|
|
|
|
| Final Q = 2
|}

'''Level 3:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| insufficient_number_velocity_samples
| Flag segments where the ratio of the number of viable values in the velocity difference to the total number of data points is less than the threshold
| 0.5
| 0
| 0
|-
| 4
| Bit 2
| too_close_to_range_limits
| Flag bins that are too close to the first or last range bin (I.e. not enough bins to do differencing)
| 3
| 1
| 4
|-
|
|
|
|
|
|
| Final Q = 4
|}

'''Level 4:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| Rsquared_too_low
| Flag values where <math> R^2 < </math> threshold
| 0.6
| 0
| 0
|-
| 4
| Bit 2
| delta_epsi_too_large
| Flag values where <math> \delta \epsilon / \epsilon > </math> threshold
| 0.6
| 1
| 4
|-
| 8
| Bit 3
| A3_coeff_invalid
| Flag values where <math> a_3 </math> < threshold
| 0
| 0
| 0
|-
| 16
| Bit 4
| dll_intercept_too_low
| Flag values where intercept (<math> a_0 </math>) < threshold
| 0
| 1
| 16
|-
| 32
| Bit 5
| dll_intercept_too_high
| Flag values where intercept > threshold (threshhold <math> = 2*yint\_expected = 2*(2*sigma_v^2) </math>, where <math> \sigma_v </math> is expected accuracy in along-beam vel.
| Instrument dependent
| 0
| 0
|-
| 64
| Bit 6
| regression_poorly_conditioned
| Flag values where there are fewer than a certain number of points for the regression
| 3
| 0
| 0
|-
| 128
| Bit 7
| dll_slope_out_of_range
| Flag values where slope <math> a_1 < 0 </math> as this would lead to negative epsilon.
| 0
| 0
| 0
|-
|
|
|
|
|
|
| Final Q = 20
|}

<br />

The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.
Note that importance (severity of the failure) is ranked by the numeric value of the flag mask.

Velocity Profiler data flags

2022-06-03T21:37:55Z

Yuengdjern: Created page with " '''How ADCP structure function quality-control flags are applied''' The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean..."

'''How ADCP structure function quality-control flags are applied'''

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:

'''Level 2:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| segment_outlier
| Flag data points <math> > 3 \sigma</math> away from segment mean for bin.
| 3 \sigma</math>
| 1
| 2
|-
| 4
| Bit 2
| profile_outlier
| Flag data points <math> > 3 \sigma</math> away from beam-profile-mean.
| 3 \sigma</math>
| 0
| 0
|-
|
|
|
|
|
|
| Final Q = 2
|}

'''Level 3:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| insufficient_number_velocity_samples
| Flag segments where the ratio of the number of viable values in the velocity difference to the total number of data points is less than the threshold
| 0.5
| 0
| 0
|-
| 4
| Bit 2
| too_close_to_range_limits
| Flag bins that are too close to the first or last range bin (I.e. not enough bins to do differencing)
| 3
| 1
| 4
|-
|
|
|
|
|
|
| Final Q = 4
|}

'''Level 4:'''
{| class="wikitable"
|-
! Flag Mask
! Bit
| Flag attribute
| Flag meaning
! Threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| Manually defined by user
| Manually defined by user
| Manually defined by user
| 0
| 0
|-
| 2
| Bit 1
| Rsquared_too_low
| Flag values where <math> R^2 < </math> threshold
| 0.6
| 0
| 0
|-
| 4
| Bit 2
| delta_epsi_too_large
| Flag values where <math> \delta \epsilon / \epsilon > </math> threshold
| 0.6
| 1
| 4
|-
| 8
| Bit 3
| A3_coeff_invalid
| Flag values where <math> a_3 </math> < threshold
| 0
| 0
| 0
|-
| 16
| Bit 4
| dll_intercept_too_low
| Flag values where intercept (<math> a_0 </math>) < threshold
| 0
| 1
| 16
|-
| 32
| Bit 5
| dll_intercept_too_high
| Flag values where intercept > threshold (threshhold <math> = 2*yint\_expected = 2*(2*sigma_v^2) </math>, where <math> \sigma_v </math> is expected accuracy in along-beam vel.
| Instrument dependent
| 0
| 0
|-
| 64
| Bit 6
| insufficient_dll_samples_for_regression
| Flag values where there are fewer than a certain number of points for the regression
| 3
| 0
| 0
|-
| 128
| Bit 7
| dll_slope_out_of_range
| Flag values where slope <math> a_1 < 0 </math> as this would lead to negative epsilon.
| 0
| 0
| 0
|-
|
|
|
|
|
|
| Final Q = 20
|}

<br />

The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.
Note that importance (severity of the failure) is ranked by the numeric value of the flag mask.

Quality control of ε estimates (QA2)

2022-06-03T20:58:09Z

Yuengdjern:

Quality control measures for each beam (flagged in benchmark datasets):
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Ensure sufficient <math> D_{ll} </math> samples were used in the regression.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. If noise is too high, <math> \epsilon </math> are rejected.
# Data segments for which the regression coefficient a<sub>0</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
# In the case of <math> \epsilon </math> estimated using the modified regression method that accounts for oscillatory motion, reject data for invalid values of <math> a_3 </math>.
# A better indication of the quality of the fit is usually provided by looking at the ratio of the estimated <math>\varepsilon</math> value to that based on the 95%-ile confidence interval estimate of the a<sub>1</sub> regression coefficient e.g. reject values where the ratio exceeds a specified threshold
# The goodness of fit (R<sup>2</sup>) for the regression provides a basic indication of the quality of the fit, data with low R<sup>2</sup> are typically rejected.

Other measures (not flagged):
# Examine the distribution of <math>\varepsilon</math> estimates - in most situations, this would be expected to be log-normal
# Comparison of observed values with nominal values based on established boundary-forced scalings may also be informative and help to identify observation or processing issues

Quality control measures for final <math> \epsilon </math> estimate:
# Examine the consistency of <math>\varepsilon</math> between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
------
To see how the data flags are applied, go to [[Velocity Profiler data flags| Velocity Profiler Data Flags]]

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Quality control of ε estimates (QA2)

2022-06-03T20:56:44Z

Yuengdjern:

Quality control measures for each beam:
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Ensure sufficient <math> D_{ll} </math> samples were used in the regression.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. If noise is too high, <math> \epsilon </math> are rejected.
# Data segments for which the regression coefficient a<sub>0</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
# In the case of <math> \epsilon </math> estimated using the modified regression method that accounts for oscillatory motion, reject data for invalid values of <math> a_3 </math>.
# A better indication of the quality of the fit is usually provided by looking at the ratio of the estimated <math>\varepsilon</math> value to that based on the 95%-ile confidence interval estimate of the a<sub>1</sub> regression coefficient e.g. reject values where the ratio exceeds a specified threshold
# The goodness of fit (R<sup>2</sup>) for the regression provides a basic indication of the quality of the fit, data with low R<sup>2</sup> are typically rejected.

Other measures (not flagged):
# Examine the distribution of <math>\varepsilon</math> estimates - in most situations, this would be expected to be log-normal
# Comparison of observed values with nominal values based on established boundary-forced scalings may also be informative and help to identify observation or processing issues

Quality control measures for final <math> \epsilon </math> estimate:
# Examine the consistency of <math>\varepsilon</math> between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
------
To see how the data flags are applied, go to [[Velocity Profiler data flags| Velocity Profiler Data Flags]]
'''[In progress]
'''

'''How ADCP structure function quality-control flags are applied'''

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:

{| class="wikitable"
|-
! Flag Mask
! Bit
! Flag Meaning
! Example threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| if FOM > FOM_limit
| 2
| 0
| 0
|-
| 2
| Bit 1
| if despike_fraction > despike_fraction_limit
| 40%
| 0
| 0
|-
| 4
| Bit 2
| if |log(e_max)-log(e_min)|> diss_ratio_limit X \sigma_{\ln\varepsilon}
| N/A
| 1
| 4
|-
| 8
| Bit 3
| if despike_iterations > despike_iterations_limit
| To be confirmed
| 0
| 0
|-
| 16
| Bit 4
| if variance resolved less than a threshold
| 50%
| 1
| 16
|-
| 32
| Bit 5
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 64
| Bit 6
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 128
| Bit 7
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|
|
|
|
|
| Final Q = 20
|}

<br />

The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Quality control of ε estimates (QA2)

2022-06-03T20:51:11Z

Yuengdjern:

Quality control measures for each beam:
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Ensure sufficient <math> D_{ll} </math> samples were used in the regression.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. If noise is too high, <math> \epsilon </math> are rejected.
# Data segments for which the regression coefficient a<sub>0</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
# In the case of <math> \epsilon </math> estimated using the modified regression method that accounts for oscillatory motion, reject data for invalid values of <math> a_3 </math>.
# A better indication of the quality of the fit is usually provided by looking at the ratio of the estimated <math>\varepsilon</math> value to that based on the 95%-ile confidence interval estimate of the a<sub>1</sub> regression coefficient e.g. reject values where the ratio exceeds a specified threshold
# The goodness of fit (R<sup>2</sup>) for the regression provides a basic indication of the quality of the fit, data with low R<sup>2</sup> are typically rejected.

Other measures (not flagged):
# Examine the distribution of <math>\varepsilon</math> estimates - in most situations, this would be expected to be log-normal
# Comparison of observed values with nominal values based on established boundary-forced scalings may also be informative and help to identify observation or processing issues

Quality control measures for final <math> \epsilon </math> estimate:
# Examine the consistency of <math>\varepsilon</math> between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
------
'''[In progress]
'''

'''How ADCP structure function quality-control flags are applied'''

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:

{| class="wikitable"
|-
! Flag Mask
! Bit
! Flag Meaning
! Example threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| if FOM > FOM_limit
| 2
| 0
| 0
|-
| 2
| Bit 1
| if despike_fraction > despike_fraction_limit
| 40%
| 0
| 0
|-
| 4
| Bit 2
| if |log(e_max)-log(e_min)|> diss_ratio_limit X \sigma_{\ln\varepsilon}
| N/A
| 1
| 4
|-
| 8
| Bit 3
| if despike_iterations > despike_iterations_limit
| To be confirmed
| 0
| 0
|-
| 16
| Bit 4
| if variance resolved less than a threshold
| 50%
| 1
| 16
|-
| 32
| Bit 5
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 64
| Bit 6
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 128
| Bit 7
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|
|
|
|
|
| Final Q = 20
|}

<br />

The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.

[[Category: Shear probes]]

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Quality control of ε estimates (QA2)

2022-06-03T20:50:15Z

Yuengdjern:

Quality control measures for each beam:
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Ensure sufficient <math> D_{ll} </math> samples were used in the regression.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. If noise is too high, <math> \epsilon </math> are rejected.
# Data segments for which the regression coefficient a<sub>0</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
# In the case of <math> \epsilon </math> estimated using the modified regression method that accounts for oscillatory motion, reject data for invalid values of <math> A_3 </math>.
# A better indication of the quality of the fit is usually provided by looking at the ratio of the estimated <math>\varepsilon</math> value to that based on the 95%-ile confidence interval estimate of the a<sub>1</sub> regression coefficient e.g. reject values where the ratio exceeds a specified threshold
# The goodness of fit (R<sup>2</sup>) for the regression provides a basic indication of the quality of the fit, data with low R<sup>2</sup> are typically rejected.

Other measures (not flagged):
# Examine the distribution of <math>\varepsilon</math> estimates - in most situations, this would be expected to be log-normal
# Comparison of observed values with nominal values based on established boundary-forced scalings may also be informative and help to identify observation or processing issues

Quality control measures for final <math> \epsilon </math> estimate:
# Examine the consistency of <math>\varepsilon</math> between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
------
'''[In progress]
'''

'''How ADCP structure function quality-control flags are applied'''

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:

{| class="wikitable"
|-
! Flag Mask
! Bit
! Flag Meaning
! Example threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| if FOM > FOM_limit
| 2
| 0
| 0
|-
| 2
| Bit 1
| if despike_fraction > despike_fraction_limit
| 40%
| 0
| 0
|-
| 4
| Bit 2
| if |log(e_max)-log(e_min)|> diss_ratio_limit X \sigma_{\ln\varepsilon}
| N/A
| 1
| 4
|-
| 8
| Bit 3
| if despike_iterations > despike_iterations_limit
| To be confirmed
| 0
| 0
|-
| 16
| Bit 4
| if variance resolved less than a threshold
| 50%
| 1
| 16
|-
| 32
| Bit 5
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 64
| Bit 6
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 128
| Bit 7
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|
|
|
|
|
| Final Q = 20
|}

<br />

The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.

[[Category: Shear probes]]

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Quality control of ε estimates (QA2)

2022-06-03T20:41:27Z

Yuengdjern:

Quality control measures for each beam:
# Data segments for which the regression coefficient a<sub>1</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative yield an imaginary <math>\varepsilon</math> value, which should be rejected
# Ensure sufficient <math> D_{ll} </math> samples were used in the regression.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. If noise is too high, <math> \epsilon </math> are rejected.
# Data segments for which the regression coefficient a<sub>0</sub> (see [[Processing your ADCP data using structure function techniques | previous step]]) is negative (implying a negative noise floor) are likely to be invalid and are typically rejected
# Examine the consistency of <math>\varepsilon</math> between bins (if evaluated) and between beams as an indication of estimate reliability - the geometric mean between beams is frequently used as the representative value
# Evaluate the impact of varying r<sub>max</sub> values (within the anticipated inertial range) on <math>\varepsilon</math>; an increase in <math>\varepsilon</math> with increasing r<sub>max</sub> is likely to indicate that v’ retains a non-turbulent contribution to the velocity difference between bins
# The goodness of fit (R<sup>2</sup>) for the regression provides a basic indication of the quality of the fit
# A better indication of the quality of the fit is usually provided by looking at the ratio of the estimated <math>\varepsilon</math> value to that based on the 95%-ile confidence interval estimate of the a<sub>1</sub> regression coefficient e.g. reject values where the ratio exceeds a specified threshold
# Examine the distribution of <math>\varepsilon</math> estimates - in most situations, this would be expected to be log-normal
# Comparison of observed values with nominal values based on established boundary-forced scalings may also be informative and help to identify observation or processing issues

------
'''[In progress]
'''

'''How ADCP structure function quality-control flags are applied'''

The Q (quality control) flags associated with shear-probe measurements are not compatible with the Ocean Sites [http://www.oceansites.org/ Ocean Sites] for quality control (QC) coding.

Every dissipation estimate from every probe must have Q flag.
The numerical values of the Q flags are as follows:

{| class="wikitable"
|-
! Flag Mask
! Bit
! Flag Meaning
! Example threshold value
| Ex: True =1 / False =0
| Ex: Q value
|-
| 1
| Bit 0
| if FOM > FOM_limit
| 2
| 0
| 0
|-
| 2
| Bit 1
| if despike_fraction > despike_fraction_limit
| 40%
| 0
| 0
|-
| 4
| Bit 2
| if |log(e_max)-log(e_min)|> diss_ratio_limit X \sigma_{\ln\varepsilon}
| N/A
| 1
| 4
|-
| 8
| Bit 3
| if despike_iterations > despike_iterations_limit
| To be confirmed
| 0
| 0
|-
| 16
| Bit 4
| if variance resolved less than a threshold
| 50%
| 1
| 16
|-
| 32
| Bit 5
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 64
| Bit 6
| manual flag to be defined by user
| N/A
| 0
| 0
|-
| 128
| Bit 7
| manual flag to be defined by user
| N/A
| 0
| 0
|-
|
|
|
|
|
| Final Q = 20
|}

<br />

The Q flags are combined by their addition.
For example a Q value of 20 means that the dissipation estimated failed both dissipation ratio limit test and the resolved variance test.
A value of 255 means that all tests failed.
The reasons for a failure can be decoded by breaking the value of Q down to its powers of 2.

[[Category: Shear probes]]

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Benchmark datasets for ADCP structure function

2022-06-02T18:43:10Z

Yuengdjern:

This page provides an overview of the benchmark dataset for instruments that measure velocity profiles e.g., [[Acoustic-Doppler Current Profilers|acoustic-Doppler current profilers]] from diverse suppliers and models.

== Datasets available ==
Selection and preparation of benchmark datasets are a work in progress! These benchmark datasets will cover a range of marine environments, background stratification and flow fields.

{| class="wikitable"
|+Summary of potential benchmark datasets for testing existing and future algorithms
|-
! style="width: 12%"| Dataset name
! Instrument
! Total depth
! Deployment height above bottom
! Background speed
! <math>\varepsilon</math> range
! Stratification/shear information
! style="width: 20%"| Comment
|-
!
! Make and Model
! [m]
! [m]
! [m/s]
! [W/kg]
!
!
|-
| Signature5beam_TidalShelf
| Nortek Signature1000
| 254
| 0.3
| <math>\lesssim</math> 0.3
|
| stratified boundary layer
| pretty average
|
|-
| RDI4beam_TidalChannel_GP130620BPb
| RDI Teledyne 600kHz
| 23
| 0.5
| 2.5
| 7e-6 to 2e-4
| unstratified
| high quality data
|
|-
| RDIWH600_CANDYFLOSS_TOP
| RDI Workhorse 600 kHz
| 145
| 125
|
| 1e-7
| region of surface wave influence
| good
|
|-
| RDIWH600_CANDYFLOSS_BEDFRAME
| RDI Workhorse 600kHz
| 145
| 1
|
| 1e-7
| bottom boundary layer on NW European shelf
| good
|
|}

-------------------------
return to [[Velocity profilers| Velocity Profilers Welcome Page]]
[[Category:Velocity profilers]]

Level 4 data (velocity profilers)

2022-06-02T15:37:38Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. {{FontColor|fg=red|text=Provide link.}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_(r^2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-06-02T15:36:54Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. {{FontColor|fg=red|text=Provide link.}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_r^(2/3)^3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 4 data (velocity profilers)

2022-06-02T15:36:14Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. {{FontColor|fg=red|text=Provide link.}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| REGRESSION_COEFF_A3
|structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression for modified method, i.e. <math>A_1</math> in <math>D_{LL} = A_3 r^2 + A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]

Level 3 data (velocity profilers)

2022-06-02T15:32:18Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
[[File:SF atomix ADCP.png|300px|thumb|Schematic of ADCP processing nomenclature]]
The required dimensions and variables for the structure-function processing level within NetCDF ATOMIX format for velocity ADCP measurements are described below. This NetCDF group contains the structure function (DLL) calculated as a function of the along-beam separation for the available/usable ADCP bins.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Segment midpoint time. Units in Days since reference time specified in variable attribute.
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_DEL
| along-beam_separation_distance_over_
which_DLL_is_evaluated_in_number_of_bins
| N_DEL
| Number of bins separating two velocity measurements used to calculate DLL
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| DLL
| second_order_structure_function
| TIME, Z_DIST, N_BEAM, N_DEL
| Differences in velocities squared have been time-averaged (units of m2/s2).
|-
| DLL_FLAGS
| second_order_structure_function_
status_flag
| TIME, Z_DIST, N_BEAM, N_DEL
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| R_DEL
| along-beam_separation_distance_at_
which_structure_function_is_evaluated
| N_BEAM, N_DEL
| Estimated quantity (in meters) from N_DEL (Level 3), BIN_SIZE (Level 2) and THETA (Level 2).
|-
| R_DIST
| distance_from_sensor_along_beams
| Z_DIST, N_BEAM
| Along-beam bin centre distance (in meters) from the transducer
|-
| DLL_N
| second_order_structure_function_number_
of_observations
| TIME, Z_DIST, N_BEAM, N_DEL
| The number of available measurements in each segment i.e., data quality.
|-
| N_SEGMENT
| unique_identifier_for_each_segment_
in_the_entire_available_timeseries
| TIME
| Enables backtracking to [[Level 2 segmented (velocity profilers)|previous processing level]]
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the structure function Dll as a function of the separation distance. Any ancillary information required for estimating the dissipation of turbulent kinetic energy may also be stored here. ''</blockquote>
|-
| dll_calculation_type
| Specify differencing technique used to estimate DLL
| Examples include:
<blockquote>Central-differencing</blockquote>
<blockquote>Forward-differencing</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes <math>\ddagger</math>
|-
| stationarity_testing
| Any testing done on the segment to verify stationarity?
| {{FontColor|fg=white|bg=red|text=To be revisited once testing begins}}. Tentatively refer to [https://bitbucket.org/efm_cb/netcdf/src/master/TestData/adcp_atomix_metada.yml demo yaml] file.
|-
| noise_testing
| Details of testing the noise levels, or if the signal comprises mostly of noise?
|
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 2 data (velocity profilers)| Level 2 segmented velocities]]

Go to [[Level 4 data (velocity profilers)| Level 4 dissipation estimates]]

Level 3 data (velocity profilers)

2022-06-02T15:30:34Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
[[File:SF atomix ADCP.png|300px|thumb|Schematic of ADCP processing nomenclature]]
The required dimensions and variables for the structure-function processing level within NetCDF ATOMIX format for velocity ADCP measurements are described below. This NetCDF group contains the structure function (DLL) calculated as a function of the along-beam separation for the available/usable ADCP bins.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Segment midpoint time. Units in Days since reference time specified in variable attribute.
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_DEL
| along-beam_separation_distance_over_
which_DLL_is_evaluated_in_number_of_bins
| N_DEL
| Number of bins separating two velocity measurements used to calculate DLL
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| DLL
| second_order_structure_function
| TIME, Z_DIST, N_BEAM, N_DEL
| Differences in velocities squared have been time-averaged (units of m2/s2).
|-
| DLL_FLAGS
| second_order_structure_function_
status_flag
| TIME, Z_DIST, N_BEAM, N_DEL
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| R_DEL
| along-beam_separation_distance_at_
which_structure_function_is_evaluated
| N_BEAM, N_DEL
| Estimated quantity (in meters) from N_DEL (Level 3), BIN_SIZE (Level 2) and THETA (Level 2).
|-
| R_DIST
| distance_from_sensor_along_beams
| Z_DIST, N_BEAM
| Along-beam bin centre distance (in meters) from the transducer
|-
| DLL_N
| second_order_structure_function_number_
of_observations
| TIME, Z_DIST, N_BEAM, N_DEL
| The number of available measurements in each segment i.e., data quality.
|-
| N_SEGMENT
| unique_identifier_for_each_segment_
in_the_entire_available_timeseries
| TIME
| Enables backtracking to [[Level 2 segmented (velocity profilers)|previous processing level]]
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables <math>\ddagger</math>
|-
| ENU_VEL
| east_north_up_error_water_velocity
| TIME, Z_DIST, N_BEAM
| This variable provides burst-averaged water velocity in geographical coordinates where N_BEAM = [1 2 3 4] corresponding to east, north, up components and error velocities for determining background flow.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the structure function Dll as a function of the separation distance. Any ancillary information required for estimating the dissipation of turbulent kinetic energy may also be stored here. ''</blockquote>
|-
| dll_calculation_type
| Specify differencing technique used to estimate DLL
| Examples include:
<blockquote>Central-differencing</blockquote>
<blockquote>Forward-differencing</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes <math>\ddagger</math>
|-
| stationarity_testing
| Any testing done on the segment to verify stationarity?
| {{FontColor|fg=white|bg=red|text=To be revisited once testing begins}}. Tentatively refer to [https://bitbucket.org/efm_cb/netcdf/src/master/TestData/adcp_atomix_metada.yml demo yaml] file.
|-
| noise_testing
| Details of testing the noise levels, or if the signal comprises mostly of noise?
|
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 2 data (velocity profilers)| Level 2 segmented velocities]]

Go to [[Level 4 data (velocity profilers)| Level 4 dissipation estimates]]

Level 3 data (velocity profilers)

2022-06-02T15:27:50Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
[[File:SF atomix ADCP.png|300px|thumb|Schematic of ADCP processing nomenclature]]
The required dimensions and variables for the structure-function processing level within NetCDF ATOMIX format for velocity ADCP measurements are described below. This NetCDF group contains the structure function (DLL) calculated as a function of the along-beam separation for the available/usable ADCP bins.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Segment midpoint time. Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_DEL
| along-beam_separation_distance_over_
which_DLL_is_evaluated_in_number_of_bins
| N_DEL
| Number of bins separating two velocity measurements used to calculate DLL
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| DLL
| second_order_structure_function
| TIME, Z_DIST, N_BEAM, N_DEL
| Differences in velocities squared have been time-averaged (units of m2/s2).
|-
| DLL_FLAGS
| second_order_structure_function_
status_flag
| TIME, Z_DIST, N_BEAM, N_DEL
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| R_DEL
| along-beam_separation_distance_at_
which_structure_function_is_evaluated
| N_BEAM, N_DEL
| Estimated quantity (in meters) from N_DEL (Level 3), BIN_SIZE (Level 2) and THETA (Level 2).
|-
| R_DIST
| distance_from_sensor_along_beams
| Z_DIST, N_BEAM
| Along-beam bin centre distance (in meters) from the transducer
|-
| DLL_N
| second_order_structure_function_number_
of_observations
| TIME, Z_DIST, N_BEAM, N_DEL
| The number of available measurements in each segment i.e., data quality.
|-
| N_SEGMENT
| unique_identifier_for_each_segment_
in_the_entire_available_timeseries
| TIME
| Enables backtracking to [[Level 2 segmented (velocity profilers)|previous processing level]]
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables <math>\ddagger</math>
|-
| ENU_VEL
| east_north_up_error_water_velocity
| TIME, Z_DIST, N_BEAM
| This variable provides burst-averaged water velocity in geographical coordinates where N_BEAM = [1 2 3 4] corresponding to east, north, up components and error velocities for determining background flow.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the structure function Dll as a function of the separation distance. Any ancillary information required for estimating the dissipation of turbulent kinetic energy may also be stored here. ''</blockquote>
|-
| dll_calculation_type
| Specify differencing technique used to estimate DLL
| Examples include:
<blockquote>Central-differencing</blockquote>
<blockquote>Forward-differencing</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes <math>\ddagger</math>
|-
| stationarity_testing
| Any testing done on the segment to verify stationarity?
| {{FontColor|fg=white|bg=red|text=To be revisited once testing begins}}. Tentatively refer to [https://bitbucket.org/efm_cb/netcdf/src/master/TestData/adcp_atomix_metada.yml demo yaml] file.
|-
| noise_testing
| Details of testing the noise levels, or if the signal comprises mostly of noise?
|
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 2 data (velocity profilers)| Level 2 segmented velocities]]

Go to [[Level 4 data (velocity profilers)| Level 4 dissipation estimates]]

Level 2 data (velocity profilers)

2022-06-02T15:26:12Z

Yuengdjern:

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Array of 1 to number of segments into which the data has been separated (Note TIME variable)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)
|}
</div>

=Variables <math>\ddagger</math>=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}. Corresponds to <math> b^\prime </math> in [[Nomenclature]]
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables''' <math>\dagger\dagger</math>
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile and is a proxy for TIME
|-
| R_VEL_DETRENDED_FLAGS
| water_radial_velocity_of_scatterers_towards_
instrument_status_flag
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| This variable provides flags for outliers in each segment <math> \geq 2 </math> standard deviations.
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after restructuring into segments.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Level 2 data (velocity profilers)

2022-06-02T15:23:28Z

Yuengdjern:

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Array of 1 to number of segments into which the data has been separated (Note TIME variable)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)
|}
</div>

=Variables <math>\ddagger</math>=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}. Corresponds to <math> b^\prime </math> in [[Nomenclature]]
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables''' <math>\dagger\dagger</math>
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile and is a proxy for TIME
|-
| R_VEL_FLAGS
| water_radial_velocity_of_scatterers_towards_
instrument_status_flag
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| This variable provides flags for outliers in each segment <math> \geq 2 </math> standard deviations.
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after restructuring into segments.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Level 2 data (velocity profilers)

2022-06-02T15:21:56Z

Yuengdjern:

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Array of 1 to number of segments into which the data has been separated (Note TIME variable)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)
|}
</div>

=Variables <math>\ddagger</math>=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}. Corresponds to <math> b^\prime </math> in [[Nomenclature]]
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables''' <math>\dagger\dagger</math>
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile and is a proxy for TIME
|-
| R_VEL_FLAGS
| water_radial_velocity_of_scatterers_towards_instrument_status_flag
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| This variable provides flags for outliers in each segment <math> \geq 2 </math> standard deviations.
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after restructuring into segments.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Level 2 data (velocity profilers)

2022-06-02T15:16:15Z

Yuengdjern:

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Array of 1 to number of segments into which the data has been separated (Note TIME variable)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)
|}
</div>

=Variables <math>\ddagger</math>=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}. Corresponds to <math> b^\prime </math> in [[Nomenclature]]
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables''' <math>\dagger\dagger</math>
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile and is a proxy for TIME
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after restructuring into segments.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Level 2 data (velocity profilers)

2022-06-02T15:14:10Z

Yuengdjern:

Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__

=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>

{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Array of 1 to number of segments into which the data has been separated (Note TIME variable)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)
|}
</div>

=Variables <math>\ddagger</math>=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL
| water_radial_velocity_of_scatterers_
towards_instrument
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| Along beam velocity [m/s] for each of the beams.Corresponds to <math> b </math> in [[Nomenclature]]
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}. Corresponds to <math> b^\prime </math> in [[Nomenclature]]
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables''' <math>\dagger\dagger</math>
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile and is a proxy for TIME
|-
| colspan="4" |
<math>\ddagger</math> One could re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after restructuring into segments.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]

Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Nomenclature

2022-06-02T15:09:47Z

Yuengdjern:

== Background (total) velocity ==
<div class="mw-collapsible mw-collapsed" data-collapsetext="Collapse" data-expandtext="Expand">

{| class="wikitable sortable"
|-
! Symbol
! Description
! Units
|-
| <math>u</math>
| zonal or longitudinal component of velocity
| <math> \mathrm{m\, s^{-1}}</math>
|-
| <math>v</math>
| meridional or transverse component of velocity
| <math>\mathrm{m\, s^{-1}}</math>
|-
| <math>w</math>
| vertical component of velocity
| <math> \mathrm{m\, s^{-1}}</math>
|-
| <math>u_e</math>
| error velocity
| <math>\mathrm{m\, s^{-1}}</math>
|-
| V
| velocity perpendicular to mean flow
| <math>\mathrm{m\, s^{-1}}</math>
|-
| <math>W_d</math>
| Profiler fall speed
| <math>\mathrm{m\, s^{-1}}</math>
|-
| <math>U_P</math>
| Flow speed past sensor
| <math>\mathrm{m\, s^{-1}}</math>
|-
| b
| Along-beam velocity from acoustic Doppler sensor
| <math>\mathrm{m\, s^{-1}}</math>
|-
| <math> b^{\prime}</math>
| Along-beam velocity from acoustic Doppler sensor with background flow deducted
| <math>\mathrm{m\, s^{-1}}</math>
|-
| <math> \delta{z}</math>
| Vertical size of measurement bin for acoustic Doppler sensor
| <math>\mathrm{m}</math>
|-
| r
| Along-beam distance from acoustic Doppler sensor
| <math>\mathrm{m}</math>
|-
| <math> \delta{r}_0</math>
| Along-beam bin size for acoustic Doppler sensor
| <math>\mathrm{m}</math>
|-
| <math> \delta{r}</math>
| Along-beam bin separation for acoustic Doppler sensor
| <math>\mathrm{m}</math>
|-
| <math> \theta</math>
| Beam transmit and receive angle relative to instrument axis for acoustic Doppler sensor
| <math>^{\circ}</math>
|}
</div>

== Turbulence properties ==
<div class="mw-collapsible mw-collapsed" data-collapsetext="Collapse" data-expandtext="Expand">

{| class="wikitable sortable"
|-
! Symbol
! Description
! Eqn
! Units
|-
| <math>\varepsilon</math>
| The rate of dissipation of turbulent kinetic energy per unit mass by viscosity
|
| <math>\mathrm{W\, kg^{-1}}</math>
|-
| <math>B</math>
| Buoyancy production -- the rate of production of potential energy by turbulence in a stratified flow through the vertical flux of buoyancy.
| <math>B= \frac{g}{\rho} \overline{\rho'w'} </math>
| <math>\mathrm{W\, kg^{-1}}</math>
|-
| <math>P</math>
| The production of turbulence kinetic energy. In a steady, spatially uniform and stratified shear flow, turbulence kinetic energy is produced by the product of the Reynolds stress and the shear, for example <math>P = -\overline{u'w'}\frac{\partial U}{\partial z} </math> . The production is balanced by the rate of dissipation turbulence kinetic energy, <math>\varepsilon</math>, and the production of potential energy by the buoyancy flux, <math>B</math>.
| <math>P = -\overline{u'w'}\frac{\partial U}{\partial z} = \varepsilon + B</math>
| <math>\mathrm{W\, kg^{-1}}</math>
|-
| <math>R_f</math>
| Flux Richardson number; the ratio of the buoyancy flux expended for the net change in potential energy (i.e., mixing) to the shear production of turbulent kinetic energy.
| <math>R_f = \frac{B}{P}</math>
|
|-
| <math>\Gamma</math>
| "Mixing coefficient"; The ratio of the rate of production of potential energy, <math>B</math>, to the rate of dissipation of kinetic energy, <math>\varepsilon</math>.
| <math>\Gamma = \frac{B}{\varepsilon} = \frac{R_f}{1-R_f}</math>
|
|-
| <math>R_i</math>
| (Gradient) Richardson number; the ratio of buoyancy freqency squared to velocity shear squared
| <math>R_i = \frac{N^2}{S^2} </math>
|
|-
| <math>\kappa_{\rho}</math>
| Turbulent eddy diffusivity via the Osborn (1980) model
| <math>\kappa_{\rho} = \Gamma \varepsilon N^{-2}</math>
| <math>\mathrm{m^2\, s^{-1}}</math>
|-
| <math>D_{ll}</math>
| Second-order longitudinal structure function
| <math>D_{ll} = \big\langle[b^{\prime}(r) - b^{\prime}(r+n\delta{r})]^2\big\rangle</math>
| <math>\mathrm{m^2\, s^{-2}}</math>
|}
</div>

== Fluid properties and background gradients for turbulence calculations ==
<div class="mw-collapsible mw-collapsed" data-collapsetext="Collapse" data-expandtext="Expand">

{| class="wikitable sortable"
|-
! Symbol
! Description
! Eqn
! Units
|-
| <math>S_P</math>
| Practical salinity
|
| <math> - </math>
|-
| <math>T</math>
| Temperature
|
| <math> \mathrm{^{\circ}C } </math>
|-
| <math>P</math>
| Pressure
|
| <math>\mathrm{dbar} </math>
|-
| <math>\rho</math>
| Density of water
| <math> \rho = \rho\left(T,S_a,P \right)</math>
| <math>\mathrm{kg\, m^{-3}} </math>
|-
| <math>\alpha</math>
| Temperature coefficient of expansion
| <math> \alpha = \frac{1}{\rho} \frac{\partial\rho}{\partial T}</math>
| <math> \mathrm{K^{-1}}</math>
|-
| <math>\beta</math>
| Saline coefficient of contraction
| <math> \beta = \frac{1}{\rho} \frac{\partial\rho}{\partial S_P}</math>
|
|-
| <math>S</math>
| Background velocity shear
| <math> S = \left[ \left( \frac{\partial U}{\partial z}\right)^2 + \left( \frac{\partial V}{\partial z}\right)^2 \right]^{1/2} </math>
| <math> \mathrm{s^{-1}} </math>
|-
| <math> \nu_{35} </math>
| Temperature dependent kinematic viscosity of seawater at a practical salinity of 35
| <math> \sim 1\times 10^{-6} </math>
| <math> \mathrm{m^2\, s^{-1} } </math>
|-
| <math>\nu_{00}</math>
| Temperature dependent kinematic viscosity of freshwater
| <math>\sim 1\times 10^{-6} </math>
| <math>\mathrm{m^2\, s^{-1} } </math>
|-
| <math>\Gamma_a </math>
| Adiabatic temperature gradient -- salinity, temperature and pressure dependent
| <math>\sim 1\times 10^{-4}</math>
| <math>\mathrm{K\, dbar^{-1} } </math>
|-
| <math>N </math>
| Background stratification, i.e buoyancy frequency
| <math>N^2 = g\left[ \alpha\left(\Gamma_a + \frac{\partial T}{\partial z} \right) - \beta \frac{\partial S_P}{\partial z} \right] </math>
| <math>\mathrm{rad\, s^{-1} } </math>
|}
</div>

== Theoretical Length and Time Scales ==
<div class="mw-collapsible mw-collapsed" data-collapsetext="Collapse" data-expandtext="Expand">

{| class="wikitable sortable"
|-
! Symbol
! Description
! Eqn
! Units
|-
| <math>\tau_N</math>
| Buoyancy timescale
| <math> \tau_N = \frac{1}{N}</math>
| <math> \mathrm{s} </math>
|-
| <math>T_N</math>
| Buoyancy period
| <math> T_N = \frac{2\pi}{N}</math>
| <math> \mathrm{s} </math>
|-
| <math>L_E</math>
| Ellison length scale (limit of vertical displacement without irreversible mixing)
| <math>L_E=\frac {\langle \rho'^2\rangle^{1/2}}{\partial \overline{\rho}/\partial z}</math>
| <math> \mathrm{m} </math>
|-
| <math> L_Z</math>
| Boundary (law of the wall) length scale
| <math> L_Z=0.39z_w </math> with 0.39 being von Kármán's constant
| <math> \mathrm{m} </math>
|-
| <math>L_S</math>
| Corssin length scale
| <math> L_S = \sqrt{\varepsilon/S^3} </math>
| <math> \mathrm{m} </math>
|-
| <math>L_K</math>
| Kolmogorov length scale (smallest overturns)
| <math>L_K=\left(\frac{\nu^3}{\varepsilon}\right)^{1/4}</math>
| <math> \mathrm{m} </math>
|-
| <math>L_o</math>
| Ozmidov length scale, measure of largest overturns in a stratified fluid
| <math>L_o=\left(\frac{\varepsilon}{N^3}\right)^{1/2}</math>
| <math> \mathrm{m} </math>
|-
| <math>L_T</math>
| Thorpe length scale
| <math>L_T</math>
| <math> \mathrm{m} </math>
|-
| <math>z_w</math>
| Distance from a boundary
| <math>z_w</math>
| <math> \mathrm{m} </math>
|}
</div>

== Turbulence Spectrum ==
<div class="mw-collapsible mw-collapsed" data-collapsetext="Collapse" data-expandtext="Expand">
These variables are used to express the [[Turbulence spectrum]] expected shapes.

{| class="wikitable sortable"
|- Style="font-weight:bold; "
! Symbol
! Description
! Eqn
! Units
|-
| <math>\Delta t</math>
| Sampling interval
| <math> \frac{1}{f_s} </math>
| <math> \mathrm{s} </math>
|-
| <math>f_s</math>
| Sampling rate
| <math>f_s=\frac{1}{\Delta t} </math>
| <math> \mathrm{s^{-1}} </math>
|-
| <math>\Delta s</math>
| Sample spacing
| <math> \Delta s = U_P \Delta t </math>
| <math> \mathrm{m} </math>
|-
| <math>\Delta l</math>
| Linear dimension of sampling volume (instrument dependent)
|
| <math> \mathrm{m} </math>
|-
| <math>f</math>
| Cyclic frequency
| <math>f=\frac{\omega}{2\pi}</math>
| <math> \mathrm{Hz} </math>
|-
| <math>\omega</math>
| Angular frequency
| <math>\omega = 2\pi f</math>
| <math> \mathrm{rad\, s^{-1}} </math>
|-
| <math>f_N</math>
| Nyquist frequency
| <math>f_N=0.5f_s</math>
| <math> \mathrm{Hz} </math>
|-
| <math>k</math>
| Cyclic wavenumber
| <math>k=\frac{f}{U_P}</math>
| <math> \mathrm{cpm} </math>
|-
| <math>\hat{k}</math>
| Angular wavenumber
| <math>\hat{k}=\frac{\omega}{U_P} = 2\pi k</math>
| <math> \mathrm{rad\, m^{-1}} </math>
|-
| <math>\tilde{k}</math>
| Normalized wavenumber
| e.g., <math>\tilde{k}=k L_K, L_K = \left(\nu^3/\varepsilon \right)^{1/4}</math>
| -
|-
| <math>\tilde{\Phi}</math>
| Normalized velocity spectrum
| e.g., <math>\tilde{\Phi}_u(\tilde{k}) = \left(\epsilon \nu^5\right)^{-1/4} \Phi_u(k)</math>
| -
|-
| <math>\tilde{\Psi}</math>
| Normalized shear spectrum
| e.g., <math>\tilde{\Psi}(\tilde{k}) = L_K^2 \left(\epsilon \nu^5\right)^{-1/4} \Psi(k)</math>
| -
|-
| <math>k_\Delta</math>
| Nyquist wavenumber, based on sampling volume size <math>\Delta l</math>
| <math>k_\Delta=\frac{0.5}{\Delta l}</math>
| <math> \mathrm{cpm} </math>
|-
| <math>k_N</math>
| Nyquist wavenumber, via Taylor's hypothesis
| <math>k_N=\frac{f_N}{U_P}</math>
| <math> \mathrm{cpm} </math>
|-
| <math>\Psi(k)</math>
| Shear spectrum. Use <math>\Psi_1</math>, <math>\Psi_2</math> to distinguish the orthogonal components of the shear. Use <math>\Psi_N</math> for the Nasmyth spectrum, <math>\Psi_{PK}</math> for the Panchev-Kesich spectrum and <math>\Psi_L</math> for the Lueck spectrum.
|
| <math> \mathrm{s^{-2}\, cpm^{-1}}</math>
|-
| <math>\Phi(k)</math>
| Velocity spectrum. Use <math>\Phi_u</math>, <math>\Phi_v</math>, <math>\Phi_v</math>, or <math>\Phi_1</math>, <math>\Phi_2</math> , <math>\Phi_3</math> for the different orthogonal components of the velocity. Use <math>\Phi_K</math> for the Kolmogorov spectrum.
|
| <math> \mathrm{m^2\, s^{-2}\, cpm^{-1}} </math>
|}
</div>

[[Category:Glossary]]

Talk:Regressing structure function against bin separation

2022-05-30T15:57:28Z

Yuengdjern:

[[User:Yuengdjern|Yuengdjern]] ([[User talk:Yuengdjern|talk]]) In the second bullet point for the centre-difference scheme, is there a typo? Should it say 'combining the data from all beams' instead of ' ....all bins'? But then this just gives a variable that matches EPSI_FINAL dimensions...

[[User:Yuengdjern|Yuengdjern]] ([[User talk:Yuengdjern|talk]] Confusing alternative regression method for bin-center differenced Dll is moved here until further notice:
* by combining the data for all of the bins, with each separation distance having a <math>D_{ll}(n, \delta)</math> value for each bin, with the regression again ultimately yielding a single <math>\varepsilon</math> value for the data segment.

Talk:Regressing structure function against bin separation

2022-05-30T15:56:54Z

Yuengdjern:

:[[User:Yuengdjern|Yuengdjern]] ([[User talk:Yuengdjern|talk]]) In the second bullet point for the centre-difference scheme, is there a typo? Should it say 'combining the data from all beams' instead of ' ....all bins'? But then this just gives a variable that matches EPSI_FINAL dimensions...

:[[User:Yuengdjern|Yuengdjern]] ([[User talk:Yuengdjern|talk]] Confusing alternative regression method for bin-center differenced Dll is moved here until further notice:
#* by combining the data for all of the bins, with each separation distance having a <math>D_{ll}(n, \delta)</math> value for each bin, with the regression again ultimately yielding a single <math>\varepsilon</math> value for the data segment.

Regressing structure function against bin separation

2022-05-30T15:53:24Z

Yuengdjern:

How the regressions are set up depends on the choice of differencing scheme, these are explained below.

== Forward-difference scheme regression ==
<div class="mw-collapsible" id="fw-diff regress" data-collapsetext="Collapse" data-expandtext="Expand">
# If <math>D_{ll}(n,\delta)</math> was evaluated using a forward-difference scheme, the regression is done for the combined data from all bins in the selected range, hence the maximum number of <math>D_{ll}(n, \delta)</math> values for each separation distance will be the number of bins in the range less 1 for <math>\delta</math> = 1, reducing by 1 for each increment in <math>\delta</math>, with the regression ultimately yielding a single <math>\varepsilon</math> value for the data segment
</div>

== Bin-centered difference scheme regression ==
<div class="mw-collapsible" id="Bin-center-diff regression" data-collapsetext="Collapse" data-expandtext="Expand">
# If <math>D_{ll}(n,\delta)</math> was evaluated using a bin-centered difference scheme, the regression is usually done for each bin individually, with a single <math>D(n, \delta)</math> for each separation distance, ultimately yielding an <math>\varepsilon</math> for each bin.
</div>

----
Back to [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]<br></br>
[[Category:Velocity profilers]]

Regressing structure function against bin separation

2022-05-30T15:53:11Z

Yuengdjern:

How the regressions are set up depends on the choice of differencing scheme, these are explained below.

== Forward-difference scheme regression ==
<div class="mw-collapsible" id="fw-diff regress" data-collapsetext="Collapse" data-expandtext="Expand">
# If <math>D_{ll}(n,\delta)</math> was evaluated using a forward-difference scheme, the regression is done for the combined data from all bins in the selected range, hence the maximum number of <math>D_{ll}(n, \delta)</math> values for each separation distance will be the number of bins in the range less 1 for <math>\delta</math> = 1, reducing by 1 for each increment in <math>\delta</math>, with the regression ultimately yielding a single <math>\varepsilon</math> value for the data segment
</div>

== Bin-centered difference scheme regression ==
<div class="mw-collapsible" id="Bin-center-diff regression" data-collapsetext="Collapse" data-expandtext="Expand">
# If <math>D_{ll}(n,\delta)</math> was evaluated using a bin-centered difference scheme, the regression is usually done for each bin individually, with a single <math>D(n, \delta)</math> for each separation distance, ultimately yielding an <math>\varepsilon</math> for each bin; or
</div>

----
Back to [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]<br></br>
[[Category:Velocity profilers]]

Processing your ADCP data using structure function techniques

2022-05-30T15:51:51Z

Yuengdjern:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:

[[File:ADCPschematic SF.png|thumb|Schematic showing along-beam distance <math> r </math> and radial velocities. ]]

# Extract or compute the [[along-beam bin center separation]] [<math>\delta r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, where [<math>b’(n, t_s)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file). Note <math> t_s </math> is the timeseries index within a segment.
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn, spectral range corresponding to <math> k^{-5/3} </math>). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / \delta r_0</math>]
# Calculate the structure function <math>D_{ll}</math> for all possible bin separations <math>\delta</math> within <math>r_{max}</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme.
# Perform a regression of <math>D_{ll}(n,\delta)</math> against <math>(\delta r)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r</sub> separation distances. Be aware of [[Regressing structure function against bin separation | special considerations for forward-difference, center-difference schemes]] in setting up the regression calculation. The regression is typically done as a least-squares fit, either as: <br /><br /> <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}</math>;
:: or as
:: <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}+a_3((\delta r)^{2/3})^3 </math> <br /><br /> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
<ol type="1" start=6>
<li> Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <br /><br /> <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> <br /><br /> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
</ol>

----
Next step: [[Final data review (QA2) | Apply quality-control on dissipation rates (QA2)]] <br></br>
Previous step:[[Raw data review (QA1) | Apply quality-control on velocity time series data (QA1)]]<br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Processing your ADCP data using structure function techniques

2022-05-30T15:51:12Z

Yuengdjern:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:

[[File:ADCPschematic SF.png|thumb|Schematic showing along-beam distance <math> r </math> and radial velocities. ]]

# Extract or compute the [[along-beam bin center separation]] [<math>\delta r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, where [<math>b’(n, t_s)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file). Note <math> t_s </math> is the timeseries index within a segment.
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn, spectral range corresponding to <math> k^{-5/3} <\math>). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / \delta r_0</math>]
# Calculate the structure function <math>D_{ll}</math> for all possible bin separations <math>\delta</math> within <math>r_{max}</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme.
# Perform a regression of <math>D_{ll}(n,\delta)</math> against <math>(\delta r)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r</sub> separation distances. Be aware of [[Regressing structure function against bin separation | special considerations for forward-difference, center-difference schemes]] in setting up the regression calculation. The regression is typically done as a least-squares fit, either as: <br /><br /> <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}</math>;
:: or as
:: <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}+a_3((\delta r)^{2/3})^3 </math> <br /><br /> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
<ol type="1" start=6>
<li> Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <br /><br /> <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> <br /><br /> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
</ol>

----
Next step: [[Final data review (QA2) | Apply quality-control on dissipation rates (QA2)]] <br></br>
Previous step:[[Raw data review (QA1) | Apply quality-control on velocity time series data (QA1)]]<br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Processing your ADCP data using structure function techniques

2022-05-30T15:13:06Z

Yuengdjern:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:

[[File:ADCPschematic SF.png|thumb|Schematic showing along-beam distance <math> r </math> and radial velocities. ]]

# Extract or compute the [[along-beam bin center separation]] [<math>\delta r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, where [<math>b’(n, t_s)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file). Note <math> t_s </math> is the timeseries index within a segment.
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / \delta r_0</math>]
# Calculate the structure function <math>D_{ll}</math> for all possible bin separations <math>\delta</math> within <math>r_{max}</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme.
# Perform a regression of <math>D_{ll}(n,\delta)</math> against <math>(\delta r)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r</sub> separation distances. Be aware of [[Regressing structure function against bin separation | special considerations for forward-difference, center-difference schemes]] in setting up the regression calculation. The regression is typically done as a least-squares fit, either as: <br /><br /> <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}</math>;
:: or as
:: <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}+a_3((\delta r)^{2/3})^3 </math> <br /><br /> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
<ol type="1" start=6>
<li> Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <br /><br /> <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> <br /><br /> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
</ol>

----
Next step: [[Final data review (QA2) | Apply quality-control on dissipation rates (QA2)]] <br></br>
Previous step:[[Raw data review (QA1) | Apply quality-control on velocity time series data (QA1)]]<br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Processing your ADCP data using structure function techniques

2022-05-30T15:12:26Z

Yuengdjern:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:

[[File:ADCPschematic SF.png|thumb|Schematic showing along-beam distance <math> r </math> and radial velocities. ]]

# Extract or compute the [[along-beam bin center separation]] [<math>\delta r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, where [<math>b’(n, t_s)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file). Note <math> t_s </math> is the timeseries index within a segment.
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / \delta r_0</math>]
# Calculate the structure function <math>D_{ll}</math> for all possible bin separations <math>\delta</math> within <math>r_{max}</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme.
# Perform a regression of <math>D_{ll}(n,\delta)</math> against <math>(\delta r)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r<sub>0</sub> separation distances. Be aware of [[Regressing structure function against bin separation | special considerations for forward-difference, center-difference schemes]] in setting up the regression calculation. The regression is typically done as a least-squares fit, either as: <br /><br /> <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}</math>;
:: or as
:: <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}+a_3((\delta r)^{2/3})^3 </math> <br /><br /> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
<ol type="1" start=6>
<li> Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <br /><br /> <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> <br /><br /> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
</ol>

----
Next step: [[Final data review (QA2) | Apply quality-control on dissipation rates (QA2)]] <br></br>
Previous step:[[Raw data review (QA1) | Apply quality-control on velocity time series data (QA1)]]<br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Processing your ADCP data using structure function techniques

2022-05-30T15:11:55Z

Yuengdjern:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:

[[File:ADCPschematic SF.png|thumb|Schematic showing along-beam distance <math> r </math> and radial velocities. ]]

# Extract or compute the [[along-beam bin center separation]] [<math>\delta r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, where [<math>b’(n, t_s)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file). Note <math> t_s </math> is the timeseries index within a segment.
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / \delta r_0</math>]
# Calculate the structure function <math>D_{ll}</math> for all possible bin separations <math>\delta</math> within <math>r_{max}</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme.
# Perform a regression of <math>D_{ll}(n,\delta)</math> against <math>(\delta r_0)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r<sub>0</sub> separation distances. Be aware of [[Regressing structure function against bin separation | special considerations for forward-difference, center-difference schemes]] in setting up the regression calculation. The regression is typically done as a least-squares fit, either as: <br /><br /> <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}</math>;
:: or as
:: <math>D_{ll} = a_0 + a_1 (\delta r)^{2/3}+a_3((\delta r)^{2/3})^3 </math> <br /><br /> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
<ol type="1" start=6>
<li> Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <br /><br /> <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> <br /><br /> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
</ol>

----
Next step: [[Final data review (QA2) | Apply quality-control on dissipation rates (QA2)]] <br></br>
Previous step:[[Raw data review (QA1) | Apply quality-control on velocity time series data (QA1)]]<br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Talk:Regressing structure function against bin separation

2022-05-30T13:40:23Z

Yuengdjern:

Talk:Regressing structure function against bin separation

2022-05-30T13:15:46Z

Yuengdjern:

Talk:Regressing structure function against bin separation

2022-05-30T13:15:08Z

Yuengdjern: Created page with ":Yuengdjern (talk) In the second bullet point for the centre-difference scheme, is there a typo? Should it say 'combine data fr..."

:[[User:Yuengdjern|Yuengdjern]] ([[User talk:Yuengdjern|talk]]) In the second bullet point for the centre-difference scheme, is there a typo? Should it say 'combine data from all beams' instead of ' ....all bins'?

Processing your ADCP data using structure function techniques

2022-05-30T12:32:04Z

Yuengdjern:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:

[[File:ADCPschematic SF.png|thumb|Schematic showing along-beam distance <math> r </math> and radial velocities. ]]

# Extract or compute the [[along-beam bin center separation]] [<math>\delta r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, where [<math>b’(n, t_s)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file). Note <math> t_s </math> is the timeseries index within a segment.
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / \delta r_0</math>]
# Calculate the structure function <math>D_{ll}</math> for all possible bin separations <math>\delta</math> within <math>r_{max}</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme.
# Perform a regression of <math>D_{ll}(n,\delta)</math> against <math>(\delta r_0)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r<sub>0</sub> separation distances. Be aware of [[Regressing structure function against bin separation | special considerations for forward-difference, center-difference schemes]] in setting up the regression calculation. The regression is typically done as a least-squares fit, either as: <br /><br /> <math>D_{ll} = a_0 + a_1 (\delta r_0)^{2/3}</math>;
:: or as
:: <math>D_{ll} = a_0 + a_1 (\delta r_0)^{2/3}+a_3((\delta r_0)^{2/3})^3 </math> <br /><br /> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
<ol type="1" start=6>
<li> Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <br /><br /> <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> <br /><br /> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
</ol>

----
Next step: [[Final data review (QA2) | Apply quality-control on dissipation rates (QA2)]] <br></br>
Previous step:[[Raw data review (QA1) | Apply quality-control on velocity time series data (QA1)]]<br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Along-beam velocity fluctuation

2022-05-27T15:11:42Z

Yuengdjern:

Methods to compute the along-beam velocity fluctuation, <math> b^{\prime}</math> :
# If using [[burst sampling]], calculations are done over the length of the burst or some sub-period over which the turbulent flow statistics can assumed to be stationary
# If using continuous sampling, calculations are done over segments with a duration over which the turbulent flow statistics can assumed to be stationary
# For each data segment consisting of N profiles (i.e. <math> t_s </math>), the turbulent fluctuations are calculated separately for each beam and bin by either:
#* ''The mean over the data segment''
#* ''A linear detrend of the segment''
#* ''A low pass filtered signal''

[[Category:Velocity profilers]]

Along-beam velocity fluctuation

2022-05-27T15:11:33Z

Yuengdjern:

Methods to compute the along-beam velocity fluctuation, <math> b^{\prime}</math> :
# If using [[burst sampling]], calculations are done over the length of the burst or some sub-period over which the turbulent flow statistics can assumed to be stationary
# If using continuous sampling, calculations are done over segments with a duration over which the turbulent flow statistics can assumed to be stationary
# For each data segment consisting of N profiles (i.e. <math> t_s </math>, the turbulent fluctuations are calculated separately for each beam and bin by either:
#* ''The mean over the data segment''
#* ''A linear detrend of the segment''
#* ''A low pass filtered signal''

[[Category:Velocity profilers]]

Processing your ADCP data using structure function techniques

2022-05-27T15:10:48Z

Yuengdjern:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:

[[File:ADCPschematic SF.png|thumb|Schematic showing along-beam distance <math> r </math> and radial velocities. ]]

# Extract or compute the [[along-beam bin center separation]] [<math>\delta r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, where [<math>b’(n, t_s)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file). Note <math> t_s </math> is the timeseries index within a segment.
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / \delta r_0</math>]
# Calculate the structure function <math>D_{ll}</math> for all possible bin separations <math>\delta</math> within <math>r_{max}</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme.
# Perform a regression of <math>D_{ll}(n,\delta)</math> against <math>(\delta r_0)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r<sub>0</sub> separation distances. Be aware of [[Regressing structure function against bin separation | special considerations for forward-difference, center-difference schemes]] in setting up the regression calculation. The regression is typically done as a least-squares fit, either as: <br /><br /> <math>D_{ll} = a_0 + a_1 (\delta r_0)^{2/3}</math>; or as <br /> <math>D_{ll} = a_0 + a_1 (\delta r_0)^{2/3}+a_3((\delta r_0)^{2/3})^3 </math> <br /><br /> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
# Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <br /><br /> <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> <br /><br /> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.

----
Next step: [[Final data review (QA2) | Apply quality-control on dissipation rates (QA2)]] <br></br>
Previous step:[[Raw data review (QA1) | Apply quality-control on velocity time series data (QA1)]]<br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Processing your ADCP data using structure function techniques

2022-05-27T14:37:08Z

Yuengdjern:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:

[[File:ADCPschematic SF.png|thumb|Schematic showing along-beam distance <math> r </math> and radial velocities. ]]

# Extract or compute the [[along-beam bin center separation]] [<math>\delta r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, where [<math>b’(n, t)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file)
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / \delta r_0</math>]
# Calculate the structure function <math>D_{ll}</math> for all possible bin separations <math>\delta</math> within <math>r_{max}</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme.
# Perform a regression of <math>D_{ll}(n,\delta)</math> against <math>(\delta r_0)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r<sub>0</sub> separation distances. Be aware of [[Regressing structure function against bin separation | special considerations for forward-difference, center-difference schemes]] in setting up the regression calculation. The regression is typically done as a least-squares fit, either as: <br /><br /> <math>D_{ll} = a_0 + a_1 (\delta r_0)^{2/3}</math>; or as <br /> <math>D_{ll} = a_0 + a_1 (\delta r_0)^{2/3}+a_3((\delta r_0)^{2/3})^3 </math> <br /><br /> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
# Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <br /><br /> <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> <br /><br /> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.

----
Next step: [[Final data review (QA2) | Apply quality-control on dissipation rates (QA2)]] <br></br>
Previous step:[[Raw data review (QA1) | Apply quality-control on velocity time series data (QA1)]]<br></br>
Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

[[Category:Velocity profilers]]

Level 4 data (velocity profilers)

2022-05-27T09:15:48Z

Yuengdjern:

{{ReviewStage
|toreview=Ready for review
|authors=Cynthia
|instrument_type=Velocity profilers
}}
This will dictate the data required at the final processing level, where we store the estimated dissipation estimates <math>\varepsilon</math> along with quality metrics.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

__TOC__

=Dimensions=
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| TIME
| time
| TIME
| Units in Days since reference time specified in variable attribute. Provide bounds attribute to designate the variable containing the limits of each segment ([http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex see CF-compliant example]).
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|}
</div>

=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| EPSI
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams.
|-
| EPSI_FINAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST
| Final (beam-averaged) dissipation rate of turbulent kinetic energy per unit mass of water [W/kg]. {{FontColor|fg=white|bg=red|text=best to state cell_methods attribute to indicate what was averaged}} e.g., [http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#methods-applied-to-a-timeseries-ex "cell_methods= N_BEAM:mean"] for averages across beams.
|-
| C2
| constant_used_in_the_second_order_structure_function
| Scalar [1 value]
| This constant appears when estimating the dissipation rate of turbulent kinetic energy from the regression coefficients. {{FontColor|fg=white|bg=red|text=provide link to equation}}
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Quality-control metrics
|-
| EPSI_FLAGS
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_status_flag
| TIME, Z_DIST, N_BEAM
| {{FontColor|fg=white|bg=red|text=To be linked, when boolean flags defined}}
|-
| EPSI_CI_HIGH
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_high_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| EPSI_CI_LOW
| specific_turbulent_kinetic_energy_dissipation_
in_sea_water_low_confidence_limit
| TIME, Z_DIST, N_BEAM
| Computed from the confidence interval of the regression slope as EPSI_CI_HIGH = (SLOPE_CI_HIGH/C2)^(3/2) {{FontColor|fg=red|text=To be verified.}}
|-
| R_MAX
| maximum_separation_distance_for_DLL_regression
| TIME, Z_DIST, N_BEAM
| maximum R_DEL separation distance [m] used when computing the regression of DLL vs r<math>^{2/3}</math>
|-
| REGRESSION_COEFF_A0
| structure_function_regression_intercept
| TIME, Z_DIST, N_BEAM
| Constant term in regression, i.e. <math>A_0</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^2</math>s<math>^{-2}</math> and value is proportional to instrument noise. {{FontColor|fg=red|text=Provide link.}}
|-
| REGRESSION_COEFF_A1
| structure_function_regression_coefficient for_r^2/3
| TIME, Z_DIST, N_BEAM
| Linear term in regression, i.e. <math>A_1</math> in <math>D_{LL} = A_1 r^{2/3} + A_0</math>. Units are m<math>^{4/3}</math>s<math>^{-2}</math>
|-
| REGRESSION_R2
| regression_goodness_of_fit_adjusted_for_number_of_terms
| TIME, Z_DIST, N_BEAM
| <math>R^2</math> computed from the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>. Specific method should be described in group attributes.
|-
| REGRESSION_N
| structure_function_regression_number_of_observations
| TIME, Z_DIST, N_BEAM
| number of data points used in the regression of <math>D_{LL}</math> vs <math>r^{2/3}</math>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"| Optional variables
|-
| EPSI_CANONICAL
| specific_turbulent_kinetic_energy_dissipation
_in_sea_water
| TIME, Z_DIST, N_BEAM
| Dissipation rate of turbulent kinetic energy per unit mass of water [W/kg] estimated from individual beams, using the canonical method in cases where use of modified method to account for contamination from oscillatory signals (e.g. waves, mooring motion) is best practise.
|}
</div>

=Group attributes (metadata)=
<div class="mw-collapsible" id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.

{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''This group includes the results associated with fitting the structure function to data in level 3. The results for each beam, along with quality indicators and errors are provided. A final estimate for the turbulent kinetic energy dissipation is also provided.''</blockquote>
|-
| dll_fitting_method
| statistical technique used for fitting the spectra.
| Examples include:
linear regression
|-
| rsquared_method
| method used to calculate the goodness of fit
| Examples include:
????
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>

----
Return to [[Level 3 data (velocity profilers)|Level 3 structure function]]

Go back to the beginning [[Dataset requirements for ADCP structure function]]