Atomix - User contributions [en]

Benchmark datasets for velocity measurements

2022-04-13T22:00:06Z

Djwain:

Information is currently being transferred onto the wiki about the additional datasets.

{| class="wikitable"
|+Summary of potential benchmark datasets for testing existing and future algorithms
|-
! style="width: 12%"| Dataset name
! Total depth
! Deployment height above bottom
! Background speed
! <math>\varepsilon</math> range
! Stratification/shear information
! style="width: 20%"| Comment
|-
! Units
! [m]
! [m]
! [m/s]
! [W/kg]
!
!
|-
| Tidal slough
| 2.8
| 0.15 and 0.45
| 0.15-0.2m/s (median)
| 1e-8 to1e-5
| Unstratified, but shear-induced anisotropy
| Unusual spikes at 0.15 m. At 0.45m, the viscous subrange is occasionally resolved
|-
| Tidal shelf
| 185
| 0.4 and 1.4
| <0.3m/s but usually 0.1m/s
|
| Bottom one in log-layer using the classical definition
| Low-quality dataset
|-
| Lake
| 4.3
| 0.12
| <0.1m/s
|
| Unstratified
| Shallow Lake (DW)
|-
| Surface waves
|
|
|
|
|
| large orbital sigma vs mean current (SM)
|-
| Surface waves
|
|
|
|
|
| sigma/U is roughly 1 (JM)
|-
| MAVS
|353
|248
|0.15 max.
|1e-8 to 1e-6
|weak stratification
|*under ice boundary layer, MAVS suspended 5m depth
|}

[[Category:Velocity point-measurements]]

Benchmark datasets for velocity measurements

2022-04-13T21:52:49Z

Djwain:

Information is currently being transferred onto the wiki about the additional datasets.

{| class="wikitable"
|+Summary of potential benchmark datasets for testing existing and future algorithms
|-
! style="width: 12%"| Dataset name
! Total depth
! Deployment height above bottom
! Background speed
! <math>\varepsilon</math> range
! Stratification/shear information
! style="width: 20%"| Comment
|-
! Units
! [m]
! [m]
! [m/s]
! [W/kg]
!
!
|-
| Tidal slough
| 2.8
| 0.15 and 0.45
| 0.15-0.2m/s (median)
| 1e-8 to1e-5
| Unstratified, but shear-induced anisotropy
| Unusual spikes at 0.15 m. At 0.45m, the viscous subrange is occasionally resolved
|-
| Tidal shelf
| 185
| 0.4 and 1.4
| <0.3m/s but usually 0.1m/s
|
| Bottom one in log-layer using the classical definition
| Low-quality dataset
|-
| Lake
|
|
|
|
|
| Shallow Lake (DW)
|-
| Surface waves
|
|
|
|
|
| large orbital sigma vs mean current (SM)
|-
| Surface waves
|
|
|
|
|
| sigma/U is roughly 1 (JM)
|-
| MAVS
|353
|248
|0.15 max.
|1e-8 to 1e-6
|weak stratification
|*under ice boundary layer, MAVS suspended 5m depth
|}

[[Category:Velocity point-measurements]]

Benchmark datasets for velocity measurements

2022-04-13T21:51:48Z

Djwain:

Information is currently being transferred onto the wiki about the additional datasets.

{| class="wikitable"
|+Summary of potential benchmark datasets for testing existing and future algorithms
|-
! style="width: 12%"| Dataset name
! Total depth
! Deployment height above bottom
! Background speed
! <math>\varepsilon</math> range
! Stratification/shear information
! style="width: 20%"| Comment
|-
! Units
! [m]
! [m]
! [m/s]
! [W/kg]
!
!
|-
| Tidal slough
| 2.8
| 0.15 and 0.45
| 0.15-0.2m/s (median)
| 1e-8 to1e-5
| Unstratified, but shear-induced anisotropy
| Unusual spikes at 0.15 m. At 0.45m, the viscous subrange is occasionally resolved
|-
| Tidal shelf
| 185
| 0.4 and 1.4
| <0.3m/s but usually 0.1m/s
|
| Bottom one in log-layer using the classical definition
| Low-quality dataset
|-
| Lake
|
|
|
|
|
| TBD (DW)
|-
| Surface waves
|
|
|
|
|
| large orbital sigma vs mean current (SM)
|-
| Surface waves
|
|
|
|
|
| sigma/U is roughly 1 (JM)
|-
| MAVS
|353
|248
|0.15 max.
|1e-8 to 1e-6
|weak stratification
|*under ice boundary layer, MAVS suspended 5m depth
|}

[[Category:Velocity point-measurements]]

Talk:Processing your ADCP data using structure function techniques

2021-11-14T21:53:47Z

Djwain:

[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:50, 12 November 2021 (CET) Can we change the title of this page to be more specific? Like "Computing the structure functions and dissipation rates"

[[User:Jmmcmillan|Jmmcmillan]] ([[User talk:Jmmcmillan|talk]]) 21:54, 12 November 2021 (CET) I find the use of <math>D(n,\delta)</math> a little confusing. In reality, for each range bin and each time ensemble, D only depends on <math>\delta</math>. Can we drop the n for simplicity?
:[[User:Djwain|Djwain]] ([[User talk:Djwain|talk]]) 22:44, 14 November 2021 (CET)I agree here - we don't actually define n on this page, so it is confusing to have it in the definition of v' and D. I think I wrote this originally and was basing it on the way my code is written, but really mean something like D_n(delta) where n denotes which bin we are differencing around or from.
::[[User:Djwain|Djwain]] ([[User talk:Djwain|talk]]) 22:44, 14 November 2021 (CET)Having just looked through the differencing examples which use the D(n,delta) nomenclature and are quite clear I think, we probably need to stick with this (otherwise we have to change a bunch of other pages).

[[User:Djwain|Djwain]] ([[User talk:Djwain|talk]]) 22:51, 14 November 2021 (CET)I agree with Justine on the three ways of dealing with the data being somewhat confusing.- I am having a very hard time visualizing what is being explained in item 1 because I don't process my data this way. I think we probably need to be giving some guidance on why you might do a forward vs a center difference and maybe branch based on that. The forward difference example is really nice and helpful I think!

[[User:Djwain|Djwain]] ([[User talk:Djwain|talk]]) 22:53, 14 November 2021 (CET)I also agree that step 7 should probably be in QA2

[[User:Djwain|Djwain]] ([[User talk:Djwain|talk]]) 22:53, 14 November 2021 (CET)A schematic is definitely needed here - SF is much easier to visualize I think than read! I don't have any schematics that I have made (and thus own for distribution).

Processing your ADCP data using structure function techniques

2021-11-14T21:51:49Z

Djwain:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:
# Extract or compute the [[along-beam bin center separation]] [<math>r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, [<math>v’(n, t)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file)
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / r_0</math>]
# Calculate the structure function <math>D</math> for all possible bin separations <math>\delta</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme. Some things to consider are: [JMM: SHOULD WE INCLUDE THESE HERE, OR MAKE A LINK TO A PAGE ABOUT QUALITY CONTROL METRICS TO COMPUTE?]
#* Including <math>D(n,\delta)</math> for <math>\delta=1</math> may be inappropriate since the velocity estimates from adjacent bins are not wholly independent, therefore the impact of its inclusion should be evaluated
#* Keep a record of the number of instances when the squared velocity difference is evaluated for each bin <math>n</math> and separation distance <math>\delta r_{0}</math> and their distribution because they are potential quality control metrics
#* The impact of additional quality criteria can also be tested e.g. valid data requirements for all intermediate separation distances, so for a forward-difference scheme with <math>n=2</math> and <math>\delta=5</math>, require all data in bins 2 to 7 to meet Level 1 QC requirements for the profile to be included when averaging to calculate <math>D(n,\delta)</math>
# Perform a regression of <math>D(n,\delta)</math> against <math>(\delta r_0)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r0 separation distances. [JMM: THE FOLLOWING ITEMS ARE CONFUSING. SINCE THIS IS BEST PRACTICE, CAN WE JUST RECOMMEND ONE METHOD?]
## If <math>D(n,\delta)</math> was evaluated using a forward-difference scheme, the regression is done for the combined data from all bins in the selected range, hence the maximum number of <math>D(n, \delta)</math> values for each separation distance will be the number of bins in the range less 1 for <math>\delta</math> = 1, reducing by 1 for each increment in <math>\delta</math>, with the regression ultimately yielding a single <math>\varepsilon</math> value for the data segment
## If <math>D(n,\delta)</math> was evaluated using a bin-centred difference scheme, the regression can either be done:
##* for each bin individually, with a single <math>D(n, \delta)</math> for each separation distance, ultimately yielding an <math>\varepsilon</math> for each bin; or
##* by combining the data for all of the bins, with each separation distance having a <math>D(n, \delta)</math> value for each bin, with the regression again ultimately yielding a single <math>\varepsilon</math> value for the data segment
## The regression is typically done as a least-squares fit, either as: <math>D = a_0 + a_1 (\delta r_0)^{2/3}</math>; or as <math>D = a_0 + a_1 (\delta r_0)^{2/3}+a_3((\delta r_0)^{2/3})^3 </math> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
# Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. [JMM: MOVE TO QA2 STEPS?]

PERHAPS WE CAN INCLUDE A FIGURE LIKE THIS TO HELP DEFINE VARIABLES.
[[File:ADCPschematic SF.png]]

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

Talk:Processing your ADCP data using structure function techniques

2021-11-14T21:51:13Z

Djwain:

Processing your ADCP data using structure function techniques

2021-11-14T21:49:17Z

Djwain:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:
# Extract or compute the [[along-beam bin center separation]] [<math>r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, [<math>v’(n, t)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file)
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / r_0</math>]
# Calculate the structure function <math>D</math> for all possible bin separations <math>\delta</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme. Some things to consider are: [JMM: SHOULD WE INCLUDE THESE HERE, OR MAKE A LINK TO A PAGE ABOUT QUALITY CONTROL METRICS TO COMPUTE?]
#* Including <math>D(n,\delta)</math> for <math>\delta=1</math> may be inappropriate since the velocity estimates from adjacent bins are not wholly independent, therefore the impact of its inclusion should be evaluated
#* Keep a record of the number of instances when the squared velocity difference is evaluated for each bin <math>n</math> and separation distance <math>\delta r_{0}</math> and their distribution because they are potential quality control metrics
#* The impact of additional quality criteria can also be tested e.g. valid data requirements for all intermediate separation distances, so for a forward-difference scheme with <math>n=2</math> and <math>\delta=5</math>, require all data in bins 2 to 7 to meet Level 1 QC requirements for the profile to be included when averaging to calculate <math>D(n,\delta)</math>
# Perform a regression of <math>D(n,\delta)</math> against <math>(\delta r_0)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r0 separation distances. [JMM: THE FOLLOWING ITEMS ARE CONFUSING. SINCE THIS IS BEST PRACTICE, CAN WE JUST RECOMMEND ONE METHOD?]
## If <math>D(n,\delta)</math> was evaluated using a forward-difference scheme, the regression is done for the combined data from all bins in the selected range, hence the maximum number of <math>D(n, \delta)</math> values for each separation distance will be the number of bins in the range less 1 for <math>\delta</math> = 1, reducing by 1 for each increment in <math>\delta</math>, with the regression ultimately yielding a single <math>\varepsilon</math> value for the data segment
## If <math>D(n,\delta)</math> was evaluated using a bin-centred difference scheme, the regression can either be done:
##* for each bin individually, with a single <math>D(n, \delta)</math> for each separation distance, ultimately yielding an <math>\varepsilon</math> for each bin; or
##* by combining the data for all of the bins, with each separation distance having a <math>D(n, \delta)</math> value for each bin, with the regression again ultimately yielding a single <math>\varepsilon</math> value for the data segment
## The regression is typically done as a least-squares fit, either as: <math>D = a_0 + a_1 (\delta r_0)^{2/3}</math>; or as <math>D = a_0 + a_1 (\delta r_0)^{2/3}+a_3((\delta r_0)^{2/3})^3 </math> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
# Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. [JMM: MOVE TO QA2 STEPS? DJW: Yes, I think so.]

PERHAPS WE CAN INCLUDE A FIGURE LIKE THIS TO HELP DEFINE VARIABLES. [DJW: Yes! SF is sometimes easier to visualize than explain.]
[[File:ADCPschematic SF.png]]

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

Talk:Processing your ADCP data using structure function techniques

2021-11-14T21:44:09Z

Djwain:

Talk:Processing your ADCP data using structure function techniques

2021-11-14T21:43:32Z

Djwain:

Processing your ADCP data using structure function techniques

2021-11-14T21:37:14Z

Djwain:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:
# Extract or compute the [[along-beam bin center separation]] [<math>r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, [<math>v’(n, t)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file)
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / r_0</math>]
# Calculate the structure function <math>D</math> for all possible bin separations <math>\delta</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme. Some things to consider are: [JMM: SHOULD WE INCLUDE THESE HERE, OR MAKE A LINK TO A PAGE ABOUT QUALITY CONTROL METRICS TO COMPUTE?]
#* Including <math>D(n,\delta)</math> for <math>\delta=1</math> may be inappropriate since the velocity estimates from adjacent bins are not wholly independent, therefore the impact of its inclusion should be evaluated
#* Keep a record of the number of instances when the squared velocity difference is evaluated for each bin <math>n</math> and separation distance <math>\delta r_{0}</math> and their distribution because they are potential quality control metrics
#* The impact of additional quality criteria can also be tested e.g. valid data requirements for all intermediate separation distances, so for a forward-difference scheme with <math>n=2</math> and <math>\delta=5</math>, require all data in bins 2 to 7 to meet Level 1 QC requirements for the profile to be included when averaging to calculate <math>D(n,\delta)</math>
# Perform a regression of <math>D(n,\delta)</math> against <math>(\delta r_0)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r0 separation distances. [JMM: THE FOLLOWING ITEMS ARE CONFUSING. SINCE THIS IS BEST PRACTICE, CAN WE JUST RECOMMEND ONE METHOD? DJW: I agree - I am having a very hard time visualizing what is being explained in item 1 because I don't process my data this way.]
## If <math>D(n,\delta)</math> was evaluated using a forward-difference scheme, the regression is done for the combined data from all bins in the selected range, hence the maximum number of <math>D(n, \delta)</math> values for each separation distance will be the number of bins in the range less 1 for <math>\delta</math> = 1, reducing by 1 for each increment in <math>\delta</math>, with the regression ultimately yielding a single <math>\varepsilon</math> value for the data segment
## If <math>D(n,\delta)</math> was evaluated using a bin-centred difference scheme, the regression can either be done:
##* for each bin individually, with a single <math>D(n, \delta)</math> for each separation distance, ultimately yielding an <math>\varepsilon</math> for each bin; or
##* by combining the data for all of the bins, with each separation distance having a <math>D(n, \delta)</math> value for each bin, with the regression again ultimately yielding a single <math>\varepsilon</math> value for the data segment
## The regression is typically done as a least-squares fit, either as: <math>D = a_0 + a_1 (\delta r_0)^{2/3}</math>; or as <math>D = a_0 + a_1 (\delta r_0)^{2/3}+a_3((\delta r_0)^{2/3})^3 </math> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
# Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. [JMM: MOVE TO QA2 STEPS? DJW: Yes, I think so.]

PERHAPS WE CAN INCLUDE A FIGURE LIKE THIS TO HELP DEFINE VARIABLES. [DJW: Yes! SF is sometimes easier to visualize than explain.]
[[File:ADCPschematic SF.png]]

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

Processing your ADCP data using structure function techniques

2021-11-14T21:34:04Z

Djwain:

To calculate the dissipation rate at a specific range bin and a specific time ensemble:
# Extract or compute the [[along-beam bin center separation]] [<math>r_0</math>] based on the instrument geometry
# Calculate the [[along-beam velocity fluctuation]] time-series in each bin <math>n</math>, [<math>v’(n, t)</math>] from the along-beam velocity data that has met the QC criteria (i.e. the data in Level 2 of the netcdf file)
# Select the maximum distance (<math>r_{max}</math>) over which to compute the structure function based on conditions of the flow (e.g., expected max overturn). The corresponding number of bins is [<math>n_{\text{rmax}} = r_{max} / r_0</math>]
# Calculate the structure function <math>D</math> for all possible bin separations <math>\delta</math> using either a [[bin-centred difference scheme]] or a [[forward-difference]] scheme. Some things to consider are: [JMM: SHOULD WE INCLUDE THESE HERE, OR MAKE A LINK TO A PAGE ABOUT QUALITY CONTROL METRICS TO COMPUTE?]
#* Including <math>D(n,\delta)</math> for <math>\delta=1</math> may be inappropriate since the velocity estimates from adjacent bins are not wholly independent, therefore the impact of its inclusion should be evaluated
#* Keep a record of the number of instances when the squared velocity difference is evaluated for each bin <math>n</math> and separation distance <math>\delta r_{0}</math> and their distribution because they are potential quality control metrics
#* The impact of additional quality criteria can also be tested e.g. valid data requirements for all intermediate separation distances, so for a forward-difference scheme with <math>n=2</math> and <math>\delta=5</math>, require all data in bins 2 to 7 to meet Level 1 QC requirements for the profile to be included when averaging to calculate <math>D(n,\delta)</math>
# Perform a regression of <math>D(n,\delta)</math> against <math>(\delta r_0)^{2/3}</math> for the appropriate range of bins and <math>\delta</math>r0 separation distances. [JMM: THE FOLLOWING ITEMS ARE CONFUSING. SINCE THIS IS BEST PRACTICE, CAN WE JUST RECOMMEND ONE METHOD? DJW: I agree - I am having a very hard time visualizing what is being explained in item 1 because I don't process my data this way.]
## If <math>D(n,\delta)</math> was evaluated using a forward-difference scheme, the regression is done for the combined data from all bins in the selected range, hence the maximum number of <math>D(n, \delta)</math> values for each separation distance will be the number of bins in the range less 1 for <math>\delta</math> = 1, reducing by 1 for each increment in <math>\delta</math>, with the regression ultimately yielding a single <math>\varepsilon</math> value for the data segment
## If <math>D(n,\delta)</math> was evaluated using a bin-centred difference scheme, the regression can either be done:
##* for each bin individually, with a single <math>D(n, \delta)</math> for each separation distance, ultimately yielding an <math>\varepsilon</math> for each bin; or
##* by combining the data for all of the bins, with each separation distance having a <math>D(n, \delta)</math> value for each bin, with the regression again ultimately yielding a single <math>\varepsilon</math> value for the data segment
## The regression is typically done as a least-squares fit, either as: <math>D = a_0 + a_1 (\delta r_0)^{2/3}</math>; or as <math>D = a_0 + a_1 (\delta r_0)^{2/3}+a_3((\delta r_0)^{2/3})^3 </math> the former being the [[canonical structure function method | canonical method]] that excludes non-turbulent velocity differences between bins, whereas the latter is a [[modified structure function method | modified method]] that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).
# Use the coefficient <math>a_1</math> to calculate <math>\varepsilon</math> as <math>\varepsilon = \left(\frac{a_1}{C_2}\right)^{2/3}</math> where <math>C_2</math> is an [[ Structure function empirical constant | empirical constant]], typically taken as 2.0 or 2.1.
# Use the coefficient <math>a_0</math> (the intercept of the regression) to estimate the noise of the velocity observations and compare to the expected value based on the instrument settings. [JMM: MOVE TO QA2 STEPS?]

PERHAPS WE CAN INCLUDE A FIGURE LIKE THIS TO HELP DEFINE VARIABLES.
[[File:ADCPschematic SF.png]]

Return to [[ADCP structure function flow chart| ADCP Flow Chart front page]]

Processing your ADCP data using structure function techniques

2021-11-14T21:30:33Z

Djwain:

Talk:Processing your ADCP data using structure function techniques

2021-11-14T21:29:45Z

Djwain:

Forward-difference

2021-11-14T21:27:19Z

Djwain:

For the '''forward-difference''' scheme
# start with <math>n</math> being the lowest number bin of the range over which the structure function is to be evaluated (number of bins in range must not exceed <math>n_{\text{max}}</math>)
## start with <math>\delta = 1</math>
## compute the second order forward-difference structure function <math>D(n,\delta)</math> as the segment mean of the square of the velocity difference between the bin <math>n</math> and bin <math>n + \delta</math>: <math>D(n, \delta) = \Big\langle \big[v^\prime(n, t) - v^\prime(n+\delta,\ t)\big]^2 \Big\rangle</math> where the angled brackets indicate the mean across all <math>t</math> for the data segment yielding a velocity difference after the application of the Level 1 QC criteria
## increment <math>\delta</math> and repeat steps until <math>\delta = n_{\text{rmax}}</math> or <math>n + \delta</math> exceeds the last bin of the range over which the structure function is to be evaluated
# increment <math>n</math> and repeat steps until <math>n + 1</math> is the last bin of the range over which the structure function is to be evaluated

See [[Example forward-difference | example forward-difference calculation]] for more detail regarding the calculation

Return to [[Processing your ADCP data using structure function techniques | Compute structure functions and dissipation estimates]]

Talk:Processing your ADCP data using structure function techniques

2021-11-14T21:19:29Z

Djwain:

ADCP Environment

2021-10-22T20:14:33Z

Djwain:

The best sampling strategy will depend on the environmental flow regimes, specifically the following parameters:

# Time and length-scales
#: Compute these to horizontal and vertical in order to make the best choices for:
## Sampling time
### Repetition rate
### Sample as fast as possible (spread out the spectral density of noise)
### Optimize trade off against power/storage
## Bin size
### No smaller than advective lengthscale if using across beam methods
### Factor this into ambiguity velocity
### Links between bin size/nominal velocity accuracy, sub-pings (averaging across pings) duration of data aggregation (burst)
## No of samples
### will be determined by whether sampling is burst vs continuous
### Note burst periods can save power and reduce required storage capacity, but must be carefully chosen to capture the required dynamics
# Time averaging (important to specify whether the averaging is over the ensemble period for the full deployment period.)
# Max expected flow velocities used to set the ADCP ambiguity velocity
# Account for likely SNR
## scatterers if known.
## Water velocities; energy.
## Likely turbulence ROT 10%
# Stratification for stratified environments
## Stratified flows can have significant background shear (boundary layer or otherwise)
##. Estimate expected shear if possible: in the presence of shear, any variation in instrument orientation (pitch, roll or heading) will result in velocity differences due to background shear not being correctly removed, leading to bias in epsilon estimates.
# Waves
## Expected influence of wave field Stokes drift on surface measurements
## Consider becoming familiar with wave removal algorithms.

Return to [[Deployment]]

QA/QC specific to spectral analysis

2021-10-18T20:44:25Z

Djwain:

Once the raw data has gone through QA/QC, then you must:

* Split the time series into sections of appropriate length based on stationarity
* Rotate the measurements into the frame of reference of the mean flow
* Determine the contribution of the [[mean flow versus turbulence]] to the velocity field
* Calculate the mean flow past the sensor

Compute the spectra

2021-10-18T20:43:17Z

Djwain:

To compute the spectrum of the turbulent velocity fluctuations, you need to:

* Determine appropriate window length and overlap for averaging spectra within each data section
* Compute the spectrum
* Convert the spectrum from the time domain to the space domain using the mean speed past the sensor

Data processing of raw measurements

2021-10-18T20:42:21Z

Djwain:

The following are non-instrument specific recommendations for processing of raw measurements. As such, an initial removal of bad data based on recommendations of the instrument manufacturer should be performed first (e.g. removing data with low coherence). Following that, you must

* Despike the data
* Remove segments of the time series where there is [[interference from the instrument frame]]
* Determine the noise

QA/QC specific to spectral analysis

2021-10-18T20:41:07Z

Djwain:

Computing turbulent spectra requires further QA/QC to generate meaningful results. Once the raw data has gone through QA/QC, then you must:

* Rotate the measurements into the frame of reference of the mean flow
* Determine the contribution of the [[mean flow versus turbulence]] to the velocity field
* Calculate the mean flow past the sensor

Interference from the instrument frame

2021-10-18T20:38:49Z

Djwain: Created page with "Instruments that measure velocities at a point have to be mounted on something. Ideally they are deployed so as to minimize the likelihood of the instrument being in the wake..."

Instruments that measure velocities at a point have to be mounted on something. Ideally they are deployed so as to minimize the likelihood of the instrument being in the wake of any frame components. You can check for wakes by looking for a consistent directional dependence of TKE that is not consistent with the general flow dynamics. You can also check for vortex shedding from the frame by looking at the spectra to check for unexplained peaks.

Data processing of raw measurements

2021-10-18T20:35:00Z

Djwain:

Quality control measures

2021-10-18T20:30:38Z

Djwain: Created page with "Some elements of quality control can only be done after spectra have been computed and should be done before final estimates of epsilon are reported: * Reject time series seg..."

Some elements of quality control can only be done after spectra have been computed and should be done before final estimates of epsilon are reported:

* Reject time series segments as inadequate for tubulence analysis
* Reject spectra (or parts of spectra)
* Identify frame interferences from the epsilon vales
* Plot spectra from all three velocity components to check for large-scale anisotropy

Velocity point-measurements

2021-10-18T20:17:23Z

Djwain:

= Welcome to the velocity point-measurements subgroup! =
This subgroup addresses best practices in obtaining [[turbulent kinetic energy dissipation]] rate estimates from time series of velocities measured at a point in space. These temporal measurements are converted into spectral observations in the wavenumber (space) domain before being fitted with a model spectrum to obtain [[turbulent kinetic energy dissipation]].

= Scope =
The subgroup will provide recommendations on:
# [[data processing of raw measurements]]
# [[spectral computations]]
# [[quality control measures]]
# [[benchmark datasets for velocity measurements]] impacted and unaffected by surface waves

Although some work has been done in obtaining turbulence estimates from moored platforms, we will not be providing benchmark datasets at this time. However, appropriate references to existing literature will be included.

Our recommendations are intended to be applicable to measurements irrespective of the manufacturer provided the data is of sufficient quality for resolving the turbulence subrange of the velocity spectra.

The benchmark datasets are intended to be a resource that can be used by the community to evaluate routines in any programming language.

Estimate epsilon

2021-10-18T20:16:36Z

Djwain:

Once the spatial spectral estimates have been computed, the following stes are recommended for obtaining epsilon:

# Establish the most likely wavenumber range for the inertial subrange
# Fit the spectrum of all three velocity components to estimate epsilon using the inertial subrange model

Compute the spectra

2021-10-18T20:15:11Z

Djwain:

To compute the spectrum of the turbulent velocity fluctuations, you need to:

* Split the time series into sections of appropriate length based on stationarity
* Determine appropriate window length and overlap for averaging spectra within each data section
* Compute the spectrum
* Convert the spectrum from the time domain to the space domain using the mean speed past the sensor

Estimate epsilon

2021-10-18T20:14:36Z

Djwain: Created page with "Once the spatial spectral estimates have been computed, the following stes are recommended for obtaining epsilon: 1. Establish the most likely wavenumber range for the inerti..."

Once the spatial spectral estimates have been computed, the following stes are recommended for obtaining epsilon:

1. Establish the most likely wavenumber range for the inertial subrange
2. Fit the spectrum of all three velocity components to estimate epsilon using the inertial subrange model

Spectral computations

2021-10-18T20:12:15Z

Djwain:

After the initial round of data QA/QC, there are three stages to estimating the turbulent dissipation rate:

* Conduct further [[QA/QC specific to spectral analysis]]
* [[Compute the spectra]]
* [[Estimate epsilon]]

Compute the spectra

2021-10-18T20:11:26Z

Djwain: Created page with "To compute the spectrum of the turbulent velocity fluctuations, you need to: * Split the time series into sections of appropriate length based on stationarity * Determine app..."

Spectral computations

2021-10-18T20:01:46Z

Djwain:

QA/QC specific to spectral analysis

2021-10-18T20:01:05Z

Djwain:

Computing turbulent spectra requires further QA/QC to generate meaningful results. Once the raw data has gone through QA/QC, then you must:

* Rotate the measurements into the frame of reference of the mean flow
* Determine the contribution of the mean flow versus turbulence to the velocity field
* Calculate the mean flow past the sensor

Data processing of raw measurements

2021-10-18T20:00:02Z

Djwain:

The following are non-instrument specific recommendations for processing of raw measurements. As such, an initial removal of bad data based on recommendations of the instrument manufacturer should be performed first (e.g. removing data with low coherence). Following that, you must

* Despike the data
* Remove segments of the time series where there is interference from the instrument frame
* Remove the noise

QA/QC specific to spectral analysis

2021-10-18T19:58:20Z

Djwain: Created page with "Computing turbulent spectra requires further QA/QC to generate meaningful results. Once the raw data has gone through QA/QC, then: * The measurements need to be rotated into..."

Computing turbulent spectra requires further QA/QC to generate meaningful results. Once the raw data has gone through QA/QC, then:

* The measurements need to be rotated into the frame of reference of the mean flow
* The contribution of the mean flow versus turbulence to the velocity field must be determined
* The mean flow past the sensor must be calculated

Spectral computations

2021-10-18T19:53:41Z

Djwain: Created page with "After the initial round of data QA/QC, there are three stages to estimating the turbulent dissipation rate: * Conduct further QA/QC specific to spectral analysis * Comput..."

After the initial round of data QA/QC, there are three stages to estimating the turbulent dissipation rate:

* Conduct further [[QA/QC specific to spectral analysis]]
* Compute the spectra
* Estimate epsilon

Velocity point-measurements

2021-10-18T19:50:31Z

Djwain:

= Welcome to the velocity point-measurements subgroup! =
This subgroup addresses best practices in obtaining [[turbulent kinetic energy dissipation]] rate estimates from time series of velocities measured at a point in space. These temporal measurements are converted into spectral observations in the wavenumber (space) domain before being fitted with a model spectrum to obtain [[turbulent kinetic energy dissipation]].

= Scope =
The subgroup will provide recommendations on:
# [[data processing of raw measurements]]
# [[spectral computations]]
# quality control measures
# [[benchmark datasets for velocity measurements]] impacted and unaffected by surface waves

Although some work has been done in obtaining turbulence estimates from moored platforms, we will not be providing benchmark datasets at this time. However, appropriate references to existing literature will be included.

Our recommendations are intended to be applicable to measurements irrespective of the manufacturer provided the data is of sufficient quality for resolving the turbulence subrange of the velocity spectra.

The benchmark datasets are intended to be a resource that can be used by the community to evaluate routines in any programming language.

Data processing of raw measurements

2021-10-18T19:49:47Z

Djwain: Created page with "The following are non-instrument specific recommendations for processing of raw measurements. As such, an initial removal of bad data based on recommendations of the instrumen..."

The following are non-instrument specific recommendations for processing of raw measurements. As such, an initial removal of bad data based on recommendations of the instrument manufacturer should be performed first (e.g. removing data with low coherence). Following that:

* The data will need to be despiked
* Segments of the time series where there is interference from the frame need to be removed
* The noise needs to be removed

Velocity point-measurements

2021-10-18T19:33:45Z

Djwain:

= Welcome to the velocity point-measurements subgroup! =
This subgroup addresses best practices in obtaining [[turbulent kinetic energy dissipation]] rate estimates from time series of velocities measured at a point in space. These temporal measurements are converted into spectral observations in the wavenumber (space) domain before being fitted with a model spectrum to obtain [[turbulent kinetic energy dissipation]].

= Scope =
The subgroup will provide recommendations on:
# [[data processing of raw measurements]]
# spectral computations
# quality control measures
# [[benchmark datasets for velocity measurements]] impacted and unaffected by surface waves

Although some work has been done in obtaining turbulence estimates from moored platforms, we will not be providing benchmark datasets at this time. However, appropriate references to existing literature will be included.

Our recommendations are intended to be applicable to measurements irrespective of the manufacturer provided the data is of sufficient quality for resolving the turbulence subrange of the velocity spectra.

The benchmark datasets are intended to be a resource that can be used by the community to evaluate routines in any programming language.