http://atomix.app.uib.no/api.php?action=feedcontributions&feedformat=atom&user=AleboyerAtomix - User contributions [en]2026-06-04T16:43:51ZUser contributionsMediaWiki 1.44.2http://atomix.app.uib.no/index.php?title=Quality-control_parameters&diff=4688Quality-control parameters2024-06-06T20:15:46Z<p>Aleboyer: Created page with "FOM_limit, var_explain limit, dissrate std ,.."</p>
<hr />
<div>FOM_limit, var_explain limit, dissrate std ,..</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4687Flow chart for shear probes2024-06-06T20:15:03Z<p>Aleboyer: /* Apply quality-control metrics */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the lowest wavenumber of spectral estimation (as a function of the vehicle's length) – [[fft-length]].<br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
:# Choose [[quality-control parameters]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4686Flow chart for shear probes2024-06-06T20:14:54Z<p>Aleboyer: /* Compute the dissipation rate estimates */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the lowest wavenumber of spectral estimation (as a function of the vehicle's length) – [[fft-length]].<br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
:# Choose [[quality-control parameters]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4685Flow chart for shear probes2024-06-06T20:14:45Z<p>Aleboyer: /* Choosing the processing parameters */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the lowest wavenumber of spectral estimation (as a function of the vehicle's length) – [[fft-length]].<br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
:# Choose [[quality-control parameters]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Fft-length&diff=4684Fft-length2024-06-06T20:11:14Z<p>Aleboyer: </p>
<hr />
<div>{{DefineConcept<br />
|description=fast Fourier transform length<br />
|article_type=Concepts<br />
}}<br />
FFT length is the number of samples used to compute the fast Fourier Transform. '''It is recommended that the displacement of the vehicle during fft-length (converted in second) should not exceed the length of the profiler''', unless the profiler is a rigidly fixed platform that is not swayed by the eddies in the flow. FFT length should be [math]2^N[/math] where N is the power of 2 the closest to time required for the vehicle to travel over a full body length. Consequently, the FFT-length and length of the vehicle sets a lower limit to the wavenumber of shear that can be resolved. <br />
<br />
== Additional considerations ==<br />
The lowest wavenumber that one wishes to resolve in a spectrum is determined by the length (in units of meters) of the segments of data that are processed by a fast Fourier transform. <br />
The lowest wavenumber resolved by a spectrum is the inverse of the length of the fft-segments. <br />
This choice is influenced by the (so far mostly unknown) rate of dissipation, statistical reliability, and the length of the vehicle that carries the shear probe. <br />
Very low <math>\varepsilon</math> values (<math>\sim 10^{-10}</math> W/kg) require spectra down to 0.5 to 1 cpm. Moderate rates (<math>10^{-8}</math> to <math>10^{-7}</math> W/kg) require resolutions of <math>\sim</math>1 cpm, while higher rates require <math>\sim</math>2 cpm. <br />
<br />
A fairly common processing technique is to window each fft segment with a cosine bell and to overlap the segments by 50%. The degrees of freedom (dof) produced by this method is 1.9 times the number of fft segments used to estimate the spectrum. The statistical reliability of a spectrum increases with the number of dof. Thus, the ratio of dissipation length to fft length is also driven by the statistical reliability that you wish to achieve. '''As a general rule, this ratio should never be less than 2, and a ratio of 5 or larger is highly desirable'''.<br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Vibration-coherent_noise_removal&diff=4682Vibration-coherent noise removal2024-06-06T19:49:50Z<p>Aleboyer: </p>
<hr />
<div>The shear probe, like nearly all other velocity sensors, measures the velocity of the fluid relative to the platform that holds the probe. Thus, platform vibrations induce a signal that is due to platform motions and does not represent environmental shear. <br />
<br />
The [[The_Goodman_algorithm|algorithm]] described by Goodman et al (2006)<ref> Goodman, L., Levine, E. R., & Lueck, R. G. (2006). On measuring the terms of the turbulent kinetic energy budget from an AUV. Journal of Atmospheric and Oceanic Technology, 23(7), 977-990. </ref> is often used to remove vibration-induced components from shear-probe spectra. <br />
This algorithm estimates the transfer functions that relate the vibration (or acceleration) signals to the shear-probe signals.<br />
Like all transfer function estimates, the algorithm relies on the coherency between the shear-probe and vibration signals in order to achieve a statistically significant estimate of the transfer functions among these signals. <br />
<br />
Focusing on one specific direction, one specific shear probe, one can simply: <br />
<br />
- compute the coherence squared <math>\Gamma^2(f)</math> between the observed velocity or shear frequency spectrum <math>E_{\mathrm{obs}}(f)</math> and the vibration frequency spectrum <math>E_{\mathrm{vib}}(f)</math>.<br />
<br />
- and remove the vibration-coherent content of the shear spectrum using <math>E_{\mathrm{clean}}(f)=E_{\mathrm{obs}}(f)(1-\Gamma^2(f))</math><br />
<br />
The statistical significance increases with increasing number of fft-segments used to make a spectral estimate. However, this removal [[The_bias_induced_by_the_Goodman_algorithm|biases the spectrum of shear low]], in a wavenumber-independent manner, and must be corrected <ref> Lueck, R. G., 2022: The bias in coherent-noise removal. Journal of Atmospheric and Oceanic Technology –, submitted, doi:--.</ref>.<br />
<br />
A desire to achieve a high spatial resolution of <math>\varepsilon</math>-estimates by using short lengths of data with few fft-segments conflicts with the need to achieve good statistical reliability of the transfer function and, thus, the correction for vibration induced signals.<br />
<br />
==References==<br />
<references /><br />
<br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Vibration-coherent_noise_removal&diff=4681Vibration-coherent noise removal2024-06-06T19:31:46Z<p>Aleboyer: </p>
<hr />
<div>The shear probe, like nearly all other velocity sensors, measures the velocity of the fluid relative to the platform that holds the probe. Thus, platform vibrations induce a signal that is due to platform motions and does not represent environmental shear. <br />
<br />
The [[The_Goodman_algorithm|algorithm]] described by Goodman et al (2006)<ref> Goodman, L., Levine, E. R., & Lueck, R. G. (2006). On measuring the terms of the turbulent kinetic energy budget from an AUV. Journal of Atmospheric and Oceanic Technology, 23(7), 977-990. </ref> is often used to remove vibration-induced components from shear-probe spectra. <br />
This algorithm estimates the transfer functions that relate the vibration (or acceleration) signals to the shear-probe signals.<br />
Like all transfer function estimates, the algorithm relies on the coherency between the shear-probe and vibration signals in order to achieve a statistically significant estimate of the transfer functions among these signals. The statistical significance increases with increasing number of fft-segments used to make a spectral estimate. However, this removal [[The_bias_induced_by_the_Goodman_algorithm|biases the spectrum of shear low]], in a wavenumber-independent manner, and must be corrected <ref> Lueck, R. G., 2022: The bias in coherent-noise removal. Journal of Atmospheric and Oceanic Technology –, submitted, doi:--.</ref>.<br />
<br />
A desire to achieve a high spatial resolution of <math>\varepsilon</math>-estimates by using short lengths of data with few fft-segments conflicts with the need to achieve good statistical reliability of the transfer function and, thus, the correction for vibration induced signals.<br />
<br />
==References==<br />
<references /><br />
<br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Diss-length&diff=4680Diss-length2024-06-06T19:11:59Z<p>Aleboyer: </p>
<hr />
<div>This is the length of data over which one estimate of dissipation rate is made for each shear probe. Using our recommended fft processing, each [[Fft-length|FFT-length]] segment will have 1.9 degrees of freedom (dof).<br />
<br />
'''As a general rule, we recommend a diss-length of minimum 3 fft-lengths.''' Similar to FFT-length, Diss-length should be a power of 2, converted in time and saved in the metadata. <br />
<br />
Five fft-lengths is desirable. It may sometimes, however, be desirable to achieve high temporal (or spatial) resolution in dissipation profiles and/or the user chooses not to apply [[the Goodman algorithm]]. In those cases the user may choose single diss-length = fft-length, with the caveat of the poor statistical reliability of the dissipation estimate. A single fft-length as diss-length is ill-advised and has poor statistical reliability.<br />
<br />
<br />
Choosing the length of data, in units of meters, for the estimation of dissipation rates makes this choice fairly platform and vehicle independent, and allows it to be mainly driven by the science one wishes to accomplish.<br />
<br />
<br />
<br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4677Flow chart for shear probes2024-06-06T19:07:11Z<p>Aleboyer: /* Apply quality-control metrics */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the lowest wavenumber of spectral estimation (as a function of the vehicle's length) – [[fft-length]].<br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4676Flow chart for shear probes2024-06-06T19:07:04Z<p>Aleboyer: /* Compute the dissipation rate estimates */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the lowest wavenumber of spectral estimation (as a function of the vehicle's length) – [[fft-length]].<br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4675Flow chart for shear probes2024-06-06T19:06:54Z<p>Aleboyer: /* Choosing the processing parameters */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the lowest wavenumber of spectral estimation (as a function of the vehicle's length) – [[fft-length]].<br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4674Flow chart for shear probes2024-06-06T19:06:09Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Processing_parameters&diff=4672Processing parameters2024-06-06T19:05:36Z<p>Aleboyer: </p>
<hr />
<div>:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
Please note that most choices made must be included in a data set, as described in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
<br />
-----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Diss-length&diff=4671Diss-length2024-06-06T19:04:48Z<p>Aleboyer: </p>
<hr />
<div>This is the length of data over which one estimate of dissipation rate is made for each shear probe. Using our recommended fft processing, each [[Fft-length|FFT-length]] segment will have 1.9 degrees of freedom (dof).<br />
<br />
'''As a general rule, we recommend a diss-length of minimum 3 fft-lengths.''' <br />
<br />
Five fft-lengths is desirable. It may sometimes, however, be desirable to achieve high temporal (or spatial) resolution in dissipation profiles and/or the user chooses not to apply [[the Goodman algorithm]]. In those cases the user may choose single diss-length = fft-length, with the caveat of the poor statistical reliability of the dissipation estimate. A single fft-length as diss-length is ill-advised and has poor statistical reliability.<br />
<br />
<br />
Choosing the length of data, in units of meters, for the estimation of dissipation rates makes this choice fairly platform and vehicle independent, and allows it to be mainly driven by the science one wishes to accomplish.<br />
<br />
<br />
<br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Diss-length&diff=4670Diss-length2024-06-06T19:04:11Z<p>Aleboyer: </p>
<hr />
<div>This is the length of data over which one estimate of dissipation rate is made for each shear probe. Using our recommended fft processing, each [[Fft-length|FFT-length]] segment will have 1.9 degrees of freedom (dof). '''As a general rule, we recommend a diss-length of minimum 3 fft-lengths.''' Five fft-lengths is desirable. It may sometimes, however, be desirable to achieve high temporal (or spatial) resolution in dissipation profiles and/or the user chooses not to apply [[the Goodman algorithm]]. In those cases the user may choose single diss-length = fft-length, with the caveat of the poor statistical reliability of the dissipation estimate. A single fft-length as diss-length is ill-advised and has poor statistical reliability.<br />
<br />
<br />
Choosing the length of data, in units of meters, for the estimation of dissipation rates makes this choice fairly platform and vehicle independent, and allows it to be mainly driven by the science one wishes to accomplish.<br />
<br />
<br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Processing_parameters&diff=4668Processing parameters2024-06-06T19:00:53Z<p>Aleboyer: </p>
<hr />
<div>:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
Please note that most choices made must be included in a data set, as described in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
<br />
-----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4667Flow chart for shear probes2024-06-06T19:00:31Z<p>Aleboyer: /* Choosing the processing parameters */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Fft-length&diff=4666Fft-length2024-06-06T18:59:51Z<p>Aleboyer: /* Additional considerations */</p>
<hr />
<div>{{DefineConcept<br />
|description=fast Fourier transform length<br />
|article_type=Concepts<br />
}}<br />
FFT length is the number of samples used to compute the fast Fourier Transform. '''It is recommended that the displacement of the vehicle during fft-length (converted in second) should not exceed the length of the profiler''', unless the profiler is a rigidly fixed platform that is not swayed by the eddies in the flow. FFT length should be [math]2^N[\math] where N is the power of 2 the closest to time required for the vehicle to travel over a full body length. Consequently, the FFT-length and length of the vehicle sets a lower limit to the wavenumber of shear that can be resolved. <br />
<br />
== Additional considerations ==<br />
The lowest wavenumber that one wishes to resolve in a spectrum is determined by the length (in units of meters) of the segments of data that are processed by a fast Fourier transform. <br />
The lowest wavenumber resolved by a spectrum is the inverse of the length of the fft-segments. <br />
This choice is influenced by the (so far mostly unknown) rate of dissipation, statistical reliability, and the length of the vehicle that carries the shear probe. <br />
Very low <math>\varepsilon</math> values (<math>\sim 10^{-10}</math> W/kg) require spectra down to 0.5 to 1 cpm. Moderate rates (<math>10^{-8}</math> to <math>10^{-7}</math> W/kg) require resolutions of <math>\sim</math>1 cpm, while higher rates require <math>\sim</math>2 cpm. <br />
<br />
A fairly common processing technique is to window each fft segment with a cosine bell and to overlap the segments by 50%. The degrees of freedom (dof) produced by this method is 1.9 times the number of fft segments used to estimate the spectrum. The statistical reliability of a spectrum increases with the number of dof. Thus, the ratio of dissipation length to fft length is also driven by the statistical reliability that you wish to achieve. '''As a general rule, this ratio should never be less than 2, and a ratio of 5 or larger is highly desirable'''.<br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Fft-length&diff=4665Fft-length2024-06-06T18:59:35Z<p>Aleboyer: </p>
<hr />
<div>{{DefineConcept<br />
|description=fast Fourier transform length<br />
|article_type=Concepts<br />
}}<br />
FFT length is the number of samples used to compute the fast Fourier Transform. '''It is recommended that the displacement of the vehicle during fft-length (converted in second) should not exceed the length of the profiler''', unless the profiler is a rigidly fixed platform that is not swayed by the eddies in the flow. FFT length should be [math]2^N[\math] where N is the power of 2 the closest to time required for the vehicle to travel over a full body length. Consequently, the FFT-length and length of the vehicle sets a lower limit to the wavenumber of shear that can be resolved. <br />
<br />
== Additional considerations ==<br />
The lowest wavenumber that one wishes to resolve in a spectrum is determined by the length (in units of meters) of the segments of data that are processed by a fast Fourier transform. <br />
The lowest wavenumber resolved by a spectrum is the inverse of the length of the fft-segments. <br />
This choice is influenced by the (so far mostly unknown) rate of dissipation, statistical reliability, and the length of the vehicle that carries the shear probe. <br />
Very low <math>\varepsilon</math> values (<math>\sim 10^{-10}</math> W/kg) require spectra down to 0.5 to 1 cpm. Moderate rates (<math>10^{-8}</math> to <math>10^{-7}</math> W/kg) require resolutions of <math>\sim</math>1 cpm, while higher rates require <math>\sim</math>2 cpm. <br />
<br />
A fairly common processing technique is to window each fft segment with a cosine bell and to overlap the segments by 50%. The degrees of freedom (dof) produced by this method is 1.9 times the number of fft segments used to estimate the spectrum. The statistical reliability of a spectrum increases with the number of dof. Thus, the ratio of dissipation length to fft length is also driven by the statistical reliability that you wish to achieve. '''As a general rule, this ratio should never be less than 2, and a ratio of 5 or larger is highly desirable'''. Finally, <br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Fft-length&diff=4661Fft-length2024-06-06T18:50:59Z<p>Aleboyer: </p>
<hr />
<div>{{DefineConcept<br />
|description=fast Fourier transform length<br />
|article_type=Concepts<br />
}}<br />
'''<nowiki>It is recommend that the fft-length (in time) should not exceed the ratio length of the profiler divided by the speed of the profiler [math]\frac{L_{profiler}}{V_{profiler}} [/math]</nowiki>''' , unless the profiler is a rigidly fixed platform that is not swayed by the eddies in the flow. <br />
<br />
The length of the vehicle that carries the shear probe sets a lower limit to the wavenumber of shear that can be resolved. <br />
<br />
The lowest wavenumber that one wishes to resolve in a spectrum is determined by the length (in units of meters) of the segments of data that are processed by a fast Fourier transform. <br />
The lowest wavenumber resolved by a spectrum is the inverse of the length of the fft-segments. <br />
This choice is influenced by the (so far mostly unknown) rate of dissipation, statistical reliability, and the length of the vehicle that carries the shear probe. <br />
Very low <math>\varepsilon</math> values (<math>\sim 10^{-10}</math> W/kg) require spectra down to 0.5 to 1 cpm. Moderate rates (<math>10^{-8}</math> to <math>10^{-7}</math> W/kg) require resolutions of <math>\sim</math>1 cpm, while higher rates require <math>\sim</math>2 cpm. <br />
<br />
A fairly common processing technique is to window each fft segment with a cosine bell and to overlap the segments by 50%. The degrees of freedom (dof) produced by this method is 1.9 times the number of fft segments used to estimate the spectrum. The statistical reliability of a spectrum increases with the number of dof. Thus, the ratio of dissipation length to fft length is also driven by the statistical reliability that you wish to achieve. '''As a general rule, this ratio should never be less than 2, and a ratio of 5 or larger is highly desirable'''. Finally, <br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Fft-length&diff=4660Fft-length2024-06-06T18:50:05Z<p>Aleboyer: </p>
<hr />
<div>{{DefineConcept<br />
|description=fast Fourier transform length<br />
|article_type=Concepts<br />
}}<br />
'''It is recommend that the fft-length (in time) should not exceed the length of the profiler [math]\frac{Length of the profiler}{Speed of the profiler} [/math]''' , unless the profiler is a rigidly fixed platform that is not swayed by the eddies in the flow. <br />
<br />
The length of the vehicle that carries the shear probe sets a lower limit to the wavenumber of shear that can be resolved. <br />
<br />
The lowest wavenumber that one wishes to resolve in a spectrum is determined by the length (in units of meters) of the segments of data that are processed by a fast Fourier transform. <br />
The lowest wavenumber resolved by a spectrum is the inverse of the length of the fft-segments. <br />
This choice is influenced by the (so far mostly unknown) rate of dissipation, statistical reliability, and the length of the vehicle that carries the shear probe. <br />
Very low <math>\varepsilon</math> values (<math>\sim 10^{-10}</math> W/kg) require spectra down to 0.5 to 1 cpm. Moderate rates (<math>10^{-8}</math> to <math>10^{-7}</math> W/kg) require resolutions of <math>\sim</math>1 cpm, while higher rates require <math>\sim</math>2 cpm. <br />
<br />
A fairly common processing technique is to window each fft segment with a cosine bell and to overlap the segments by 50%. The degrees of freedom (dof) produced by this method is 1.9 times the number of fft segments used to estimate the spectrum. The statistical reliability of a spectrum increases with the number of dof. Thus, the ratio of dissipation length to fft length is also driven by the statistical reliability that you wish to achieve. '''As a general rule, this ratio should never be less than 2, and a ratio of 5 or larger is highly desirable'''. Finally, <br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Fft-length&diff=4658Fft-length2024-06-06T18:48:17Z<p>Aleboyer: </p>
<hr />
<div>{{DefineConcept<br />
|description=fast Fourier transform length<br />
|article_type=Concepts<br />
}}<br />
'''The fft-length (in time) should never exceed the length of the profiler [math]\times[/math]''' , unless the profiler is a rigidly fixed platform that is not swayed by the eddies in the flow. <br />
<br />
The length of the vehicle that carries the shear probe sets a lower limit to the wavenumber of shear that can be resolved. <br />
<br />
The lowest wavenumber that one wishes to resolve in a spectrum is determined by the length (in units of meters) of the segments of data that are processed by a fast Fourier transform. <br />
The lowest wavenumber resolved by a spectrum is the inverse of the length of the fft-segments. <br />
This choice is influenced by the (so far mostly unknown) rate of dissipation, statistical reliability, and the length of the vehicle that carries the shear probe. <br />
Very low <math>\varepsilon</math> values (<math>\sim 10^{-10}</math> W/kg) require spectra down to 0.5 to 1 cpm. Moderate rates (<math>10^{-8}</math> to <math>10^{-7}</math> W/kg) require resolutions of <math>\sim</math>1 cpm, while higher rates require <math>\sim</math>2 cpm. <br />
<br />
A fairly common processing technique is to window each fft segment with a cosine bell and to overlap the segments by 50%. The degrees of freedom (dof) produced by this method is 1.9 times the number of fft segments used to estimate the spectrum. The statistical reliability of a spectrum increases with the number of dof. Thus, the ratio of dissipation length to fft length is also driven by the statistical reliability that you wish to achieve. '''As a general rule, this ratio should never be less than 2, and a ratio of 5 or larger is highly desirable'''. Finally, <br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4656Flow chart for shear probes2024-06-06T18:32:35Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Round these up to [[nearest power-of-two number]] of samples.<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4655Flow chart for shear probes2024-06-06T18:27:09Z<p>Aleboyer: /* Choosing the processing parameters */ remong step three which not a processing step.</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Round these up to [[nearest power-of-two number]] of samples.<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4653Flow chart for shear probes2024-06-06T18:17:19Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Translate [[diss-length]] and [[fft-length]] into [[duration]] (time).<br />
:# Round these up to [[nearest power-of-two number]] of samples.<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4652Flow chart for shear probes2024-06-06T18:16:59Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Translate [[diss-length]] and [[fft-length]] into [[duration]] (time).<br />
:# Round these up to [[nearest power-of-two number]] of samples.<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4651Flow chart for shear probes2024-06-06T18:16:29Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Translate [[diss-length]] and [[fft-length]] into [[duration]] (time).<br />
:# Round these up to [[nearest power-of-two number]] of samples.<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
<br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
<br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4650Flow chart for shear probes2024-06-06T18:15:14Z<p>Aleboyer: /* Apply quality-control metrics */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Translate [[diss-length]] and [[fft-length]] into [[duration]] (time).<br />
:# Round these up to [[nearest power-of-two number]] of samples.<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible mw-collapsed" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4649Flow chart for shear probes2024-06-06T18:14:57Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
</div><br />
<br />
<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Translate [[diss-length]] and [[fft-length]] into [[duration]] (time).<br />
:# Round these up to [[nearest power-of-two number]] of samples.<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4648Flow chart for shear probes2024-06-06T18:13:11Z<p>Aleboyer: /* Compute the dissipation rate estimates */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Translate [[diss-length]] and [[fft-length]] into [[duration]] (time).<br />
:# Round these up to [[nearest power-of-two number]] of samples.<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<div class="mw-collapsible mw-collapsed" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4647Flow chart for shear probes2024-06-06T18:11:13Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:# Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]].<br />
:# Choose the lowest wavenumber of spectral estimation – [[fft-length]].<br />
:# Translate [[diss-length]] and [[fft-length]] into [[duration]] (time).<br />
:# Round these up to [[nearest power-of-two number]] of samples.<br />
:# Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]].<br />
:# Choose [[de-spiking parameters]].<br />
:# Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4646Flow chart for shear probes2024-06-06T18:10:01Z<p>Aleboyer: /* Choosing the processing parameters */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4644Flow chart for shear probes2024-06-06T18:07:04Z<p>Aleboyer: /* Choosing the processing parameters */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4643Flow chart for shear probes2024-06-06T18:06:21Z<p>Aleboyer: /* Choosing the processing parameters */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible mw-collapsed" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4642Flow chart for shear probes2024-06-06T18:05:43Z<p>Aleboyer: /* Conversion to physical units */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4641Flow chart for shear probes2024-06-06T18:05:20Z<p>Aleboyer: /* "Section" selection */</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand conditions"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible mw-collapsed" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4640Flow chart for shear probes2024-06-06T18:04:53Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible mw-collapsed" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand conditions"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4639Flow chart for shear probes2024-06-06T18:04:08Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into five major steps, which apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand conditions"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4638Flow chart for shear probes2024-06-06T18:03:14Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into the following five major steps and these steps apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible" id="physical units" data-collapsetext="Collapsed" data-expandtext="Expand conditions"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4637Flow chart for shear probes2024-06-06T18:01:39Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into the following five major steps and these steps apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:# [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:# [[Determine the temperature of the water]].<br />
:# [[Convert the shear probe data]] samples into physical units<br />
:# Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals.<br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:# Choose the [[minimum speed]] of profiling.<br />
:# Choose the [[direction of the vertical velocity]] of the profiler.<br />
:# Choose the [[minimum depth]].<br />
:# Choose the [[maximum pitch and roll]] of the profiler.<br />
:# Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Fft-length&diff=4636Fft-length2024-06-06T18:00:52Z<p>Aleboyer: </p>
<hr />
<div>{{DefineConcept<br />
|description=fast Fourier transform length<br />
|article_type=Concepts<br />
}}<br />
The lowest wavenumber that one wishes to resolve in a spectrum is determined by the length (in units of meters) of the segments of data that are processed by a fast Fourier transform. <br />
The lowest wavenumber resolved by a spectrum is the inverse of the length of the fft-segments. <br />
This choice is influenced by the (so far mostly unknown) rate of dissipation, statistical reliability, and the length of the vehicle that carries the shear probe. <br />
Very low <math>\varepsilon</math> values (<math>\sim 10^{-10}</math> W/kg) require spectra down to 0.5 to 1 cpm. Moderate rates (<math>10^{-8}</math> to <math>10^{-7}</math> W/kg) require resolutions of <math>\sim</math>1 cpm, while higher rates require <math>\sim</math>2 cpm. <br />
<br />
A fairly common processing technique is to window each fft segment with a cosine bell and to overlap the segments by 50%. The degrees of freedom (dof) produced by this method is 1.9 times the number of fft segments used to estimate the spectrum. The statistical reliability of a spectrum increases with the number of dof. Thus, the ratio of dissipation length to fft length is also driven by the statistical reliability that you wish to achieve. '''As a general rule, this ratio should never be less than 2, and a ratio of 5 or larger is highly desirable'''. Finally, the length of the vehicle that carries the shear probe also sets a lower limit to the wavenumber of shear that can be resolved. '''The fft-length should never exceed the length of the profiler''', unless the profiler is a rigidly fixed platform that is not swayed by the eddies in the flow.<br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Diss-length&diff=4635Diss-length2024-06-06T17:58:55Z<p>Aleboyer: </p>
<hr />
<div>This is the length of data over which one estimate of dissipation rate is made for each shear probe. Choosing the length of data, in units of meters, for the estimation of dissipation rates makes this choice fairly platform and vehicle independent, and allows it to be mainly driven by the science one wishes to accomplish.<br />
<br />
Using our recommended fft processing, each [[Fft-length|FFT-length]] segment will have 1.9 degrees of freedom (dof). Normally a single fft-length as diss-length is ill-advised and has poor statistical reliability. '''As a general rule, we recommend a diss-length of minimum 3 fft-lengths.''' Five fft-lengths is desirable. It may sometimes, however, be desirable to achieve high temporal (or spatial) resolution in dissipation profiles and/or the user chooses not to apply [[the Goodman algorithm]]. In those cases the user may choose single diss-length = fft-length, with the caveat of the poor statistical reliability of the dissipation estimate.<br />
<br />
<br />
----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Maximum_pitch_and_roll&diff=4633Maximum pitch and roll2024-06-06T16:17:20Z<p>Aleboyer: </p>
<hr />
<div>When vertical profilers and gliders start their profile, there may be considerable pitching and rolling of the vehicle before it settles into a more consistent flight. Thus, these parameters can also be used to exclude regions that produce poor data.<br />
There are no hard criterion for pitching and rolling.<br />
In quiesent water, the principal axis of a vertical profiler typically pitches and rolls about <math>1\, ^{\circ}</math> about their mean values.<br />
In waters that are vigorously turbulent, the large eddies may make a '''vertical profiler''' '''pitch and roll by about <math>5\, ^{\circ}</math>''' .<br />
'''Gliders''' pitch and roll similarly around their '''typical flight angle''' of <math>30^{\circ}</math>.<br />
Pitch and roll angles must not be confused with the angle-of-attack (aoa).<br />
The aoa is the angle of the flow relative to the angle of the principal axis of the vehicle (vertical profiler, glider, AUV, etc.) and may be much smaller than the pitch or roll angles.<br />
For example, a glider typically has a pitch of <math>30^{\circ}</math> with respect to the horizontal plane.<br />
However, once it is gliding stably, the angle of the incoming flow is about <math>3^{\circ}</math> with respect to the longitudinal axis of the glider.<br />
The aoa produces lift on the glider which is required for "flight".<br />
<br />
<br />
<br />
<br />
<br />
-----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Maximum_pitch_and_roll&diff=4632Maximum pitch and roll2024-06-06T16:16:49Z<p>Aleboyer: </p>
<hr />
<div>When vertical profilers and gliders start their profile, there may be considerable pitching and rolling of the vehicle before it settles into a more consistent flight. Thus, these parameters can also be used to exclude regions that produce poor data.<br />
There are no hard criterion for pitching and rolling.<br />
In quiesent water, the principal axis of a vertical profiler typically pitches and rolls about <math>1\, ^{\circ}</math> about their mean values.<br />
In waters that are vigorously turbulent, the large eddies may make a vertical profiler '''pitch and roll by about <math>5\, ^{\circ}</math>''' .<br />
Gliders pitch and roll similarly around their '''typical flight angle''' of <math>30^{\circ}</math>.<br />
Pitch and roll angles must not be confused with the angle-of-attack (aoa).<br />
The aoa is the angle of the flow relative to the angle of the principal axis of the vehicle (vertical profiler, glider, AUV, etc.) and may be much smaller than the pitch or roll angles.<br />
For example, a glider typically has a pitch of <math>30^{\circ}</math> with respect to the horizontal plane.<br />
However, once it is gliding stably, the angle of the incoming flow is about <math>3^{\circ}</math> with respect to the longitudinal axis of the glider.<br />
The aoa produces lift on the glider which is required for "flight".<br />
<br />
<br />
<br />
<br />
<br />
-----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Maximum_pitch_and_roll&diff=4631Maximum pitch and roll2024-06-06T16:16:28Z<p>Aleboyer: </p>
<hr />
<div>When vertical profilers and gliders start their profile, there may be considerable pitching and rolling of the vehicle before it settles into a more consistent flight. Thus, these parameters can also be used to exclude regions that produce poor data.<br />
There are no hard criterion for pitching and rolling.<br />
In quiesent water, the principal axis of a vertical profiler typically pitches and rolls about <math>1\, ^{\circ}</math> about their mean values.<br />
In waters that are vigorously turbulent, the large eddies may make a vertical profiler '''pitch and roll by about <math>5\, ^{\circ}</math>''' .<br />
Gliders pitch and roll similarly around their typical flight angle of <math>30^{\circ}</math>.<br />
Pitch and roll angles must not be confused with the angle-of-attack (aoa).<br />
The aoa is the angle of the flow relative to the angle of the principal axis of the vehicle (vertical profiler, glider, AUV, etc.) and may be much smaller than the pitch or roll angles.<br />
For example, a glider typically has a pitch of <math>30^{\circ}</math> with respect to the horizontal plane.<br />
However, once it is gliding stably, the angle of the incoming flow is about <math>3^{\circ}</math> with respect to the longitudinal axis of the glider.<br />
The aoa produces lift on the glider which is required for "flight".<br />
<br />
<br />
<br />
<br />
<br />
-----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Direction_of_the_vertical_velocity&diff=4629Direction of the vertical velocity2024-06-06T16:07:02Z<p>Aleboyer: </p>
<hr />
<div>The direction of the vertical velocity of the profiler is important. <br />
Most vertical profilers collect data while they are '''descending''', and the data collected while '''ascending''' is useless because the probes are in the wake of the instrument and there is no oncoming flow over the shear probes. <br />
However, some profilers are designed to collect data while they are ascending and the data collected during it descent is not useable.<br />
'''Gliders can produce meaningful data while descending and ascending'''.<br />
So, it is important to know what type of instrument was used to collect your shear-probe data, so that you select data from when it was travelling in the appropriate direction. <br />
<br />
<br />
<br />
<br />
-----------------------------<br />
return to [[Flow chart for shear probes]]<br />
<br />
<br />
[[Category:Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4628Flow chart for shear probes2024-06-06T15:27:17Z<p>Aleboyer: </p>
<hr />
<div><br />
The processing of shear-probe data can be divided into the following five major steps and these steps apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:* [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:* [[Determine the temperature of the water]]. <br />
:* [[Convert the shear probe data]] samples into physical units <br />
:* Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals. <br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:* Choose the [[minimum speed]] of profiling.<br />
:* Choose the [[direction of the vertical velocity]] of the profiler.<br />
:* Choose the [[minimum depth]].<br />
:* Choose the [[maximum pitch and roll]] of the profiler. <br />
:* Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each [[detrending time series|detrended]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Flow_chart_for_shear_probes&diff=4627Flow chart for shear probes2024-06-06T15:26:07Z<p>Aleboyer: add link to detrend method page</p>
<hr />
<div><br />
The processing of shear-probe data can be divided into the following five major steps and these steps apply to data collected with any platform or vehicle. There are many sub-steps to these major steps. The major steps are:<br />
<br />
__TOC__<br />
<br />
== Conversion to physical units ==<br />
<div class="mw-collapsible" id="physical units" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
As a first step, the raw binary data needs to be transformed into physical units.<br />
:* [[Determine the speed of profiling]] of the shear-probe through the water.<br />
:* [[Determine the temperature of the water]]. <br />
:* [[Convert the shear probe data]] samples into physical units <br />
:* Convert all other signals per the recommendations of the manufacturer of the sensor or instruments that produce these signals. <br />
</div><br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
== "Section" selection ==<br />
<div class="mw-collapsible" id="Section" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Before you can process your shear-probe data to derive the rate of dissipation you must select the [[section]] of data that you wish to process. For a vertically profiling instrument, this is traditionnally referred to as a "profile". We adopt the term "section" as this is platform independent and will include time series for dissipation estimates along horizontal or slanted trajectories as well as from moored shear probes. You must make sure that the selection is meaningful and sensible. For example, the shear probe must be profiling through the water with a speed, direction, and orientation that is fairly stationary. The selection of data can be partially automated by requiring that the kinematics of your instrument achieve certain minimum criteria. The steps to section selection are as follows: <br />
:* Choose the [[minimum speed]] of profiling.<br />
:* Choose the [[direction of the vertical velocity]] of the profiler.<br />
:* Choose the [[minimum depth]].<br />
:* Choose the [[maximum pitch and roll]] of the profiler. <br />
:* Choose the [[minimum duration]] over which the [[minimum speed]] through [[maximum pitch and roll]] must be satisfied. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
<br />
== Choosing the [[processing parameters]] ==<br />
<div class="mw-collapsible" id="processing parameters" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
:* Choose the length of data (in meters) used for each dissipation estimate – [[diss-length]]. <br />
:* Choose the lowest wavenumber of spectral estimation – [[fft-length]]. <br />
:* Translate [[diss-length]] and [[fft-length]] into [[duration]] (time). <br />
:* Round these up to [[nearest power-of-two number]] of samples. <br />
:* Choose a [[high-pass filter cut-off frequency]] to be consistent with duration of the [[fft-length]]. <br />
:* Choose [[de-spiking parameters]]. <br />
:* Choose [[vibration-coherent noise removal]].<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Compute the [[dissipation rate estimates]] ==<br />
<br />
<div class="mw-collapsible" id="estimates" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
The following steps are recommended to obtain estimates of the turbulent dissipation rate of kinetic energy (<math>\varepsilon</math>). <br />
<br />
# Extract the section to estimate dissipation time series ("Section" selection). <br />
# High-pass filter the shear-probe and (optionally) the vibration data. <br />
# Identify each diss-length segment in the section. <br />
# [[De-spike the shear-probe data]], and track the fraction of data affected by de-spiking within each diss-length segment. This will become a [[Shear_probes_quality_control_metrics|quality-control metric]]. <br />
# Calculate the [[frequency spectra and cross-spectra of shear and vibrations]] for each detrended [[detrending time series]] diss-length segment. <br />
# Extract the original and the vibration-coherent clean shear-probe frequency spectra with [[the Goodman algorithm]]. <br />
# Correct shear and vibration frequency spectra for [[the high-pass filter]]. <br />
# Correct the cleaned frequency spectra for [[the bias induced by the Goodman algorithm]]. <br />
# Convert the frequency spectra into wavenumber spectra using the mean speed, <math>U</math>, for each diss-length segment. That is, make the wavenumber <math>k=f/U</math> and the spectrum <math>E(k)=UE(f)</math> .<br />
# Correct the spectra of shear for the [[wavenumber response of the shear probe]]. <br />
# Apply an [[iterative spectral integration algorithm]] to estimate the variance of shear.<br />
# If the dissipation estimate is larger than [[shear inertial subrange fit]] use the method fit to the inertial subrange<br />
# Calculate the turbulent dissipation rate by multiplying the shear variance by <math>\frac{15}{2}\nu</math> where <math>\nu </math> is the temperature-dependent kinematic viscosity.<br />
# Determine the [[figure of merit (FOM)]] for each shear-probe spectrum. <br />
# Calculate the expected variance of each dissipation estimate.<br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
</div><br />
<br />
== Apply [[Shear_probes_quality_control_metrics|quality-control metrics]] ==<br />
<br />
<div class="mw-collapsible" id="quality_control" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
Shear-probe data can be corrupted or compromised in several different ways.<br />
These include but are not limited to (''i'') collision with plankton and other materials, (''ii'') unremovable vibrational contamination. (''iii'') electronic noise, and (''iv'') interference from other instrumentation on a platform that carries the shear probes.<br />
This section describes the quality control metrics and the coding used to [[Quality_control_coding|identify]] them.<br />
Quality-control metrics that are currently identified include;<br />
<br />
:* [[figure of merit (FOM)]] <br />
:* [[fraction of shear-probe data altered by the de-spiking routine]] <br />
:* number of [[iterations]] of the de-spiking routine required to clean the data <br />
:* [[agreement between dissipation estimates]] from redundant sensors (i.e. two or more shear probes) <br />
<br />
The numerical threshold for these metrics should depend, as much as possible, on the known statistical properties of a turbulence shear measurement. <br />
The numerical values of the QC codes (or flags) is found in [[Quality_control_coding|QC-flags]]. <br />
<br />
<br />
Please note that most choices made must be included in a data set, as described <br />
in the [[Netcdf meta data (shear probes)|list of meta data]].<br />
<br />
<br />
</div><br />
--------------------<br />
Return to [[ Shear probes | Shear Probe Welcome Page]]<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Level_1_data_(shear_probes)&diff=4626Level 1 data (shear probes)2024-06-06T15:13:24Z<p>Aleboyer: Removing the link the Global NetCDF parameter page.</p>
<hr />
<div><br />
<br />
Level 1 data refers to the full resolution data in physical units, e.g. the regularly sampled time series of [[Convert the shear probe data|converted shear probe data]]. If needed, a transfer function for shear is given. Below, a table with parameters included in the level 1 data set is compiled. The data are stored as a NetCDF group with the name "L1_converted".<br />
<br />
In the standard names, "sea_water" can be replaced with "water" if working in freshwater environments. This is specified in some entries below but not all.<br />
<br />
<br />
=Dimensions=<br />
<div class="mw-collapsible mw-expand" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
<br />
{| class="wikitable sortable"<br />
! Dimension !! Description<br />
|-<br />
| TIME || length of the record from turbulence (fast) data channels<br />
|-<br />
| TIME_*** <math>^a</math> || length of the record from slow data channels (if different from fast)<br />
|-<br />
|N_SHEAR_SENSORS ||number of shear channels (shear sensors)<br />
|-<br />
|N_***_SENSORS <math>^b</math> ||number of *** channel (sensors)<br />
|-<br />
|}<br />
<math>^a</math> Typically TIME is assumed for the fast-sampled microstructure channels. Use, e.g., TIME_SLOW or TIME_CTD for slower sampled channels such as CTD and tilt sensors. If the application requires different time stamps for different sensors, this can be utilized like TIME_PITCH, TIME_ACC etc.<br />
<br><br />
<math>^b</math> Please use these examples for related sensors:<br><br />
N_VIB_SENSORS for vibration (piezo-acceleration) sensors, <br><br />
N_ACC_SENSORS for vibration acceleration sensors, <br><br />
N_GRADT_SENSORS for thermistors, <br><br />
N_GRADC_SENSORS for microconductivity sensors. <br><br />
</div><br />
<br />
=Variables=<br />
<div class="mw-collapsible mw-expand" data-collapsetext="Collapse" data-expandtext="Expand"><br />
<br><br />
<br />
{| class="wikitable sortable"<br />
|-<br />
! Variable Name !! Req.<math>^a</math> !! Standard Name !! Unit !! Dimensions <br />
|-<br />
| TIME || R || time || [http://cfconventions.org/cf-conventions/cf-conventions CF-Convention]<math>^b</math> || TIME <br />
|-<br />
| SHEAR || R || [sea]<math>^c</math>_water_velocity_shear || s-1 || TIME, N_SHEAR_SENSORS<br />
|-<br />
| PSPD_REL || HR|| platform_speed_wrt_[sea]<math>^c</math>_water || m s-1 || TIME<br />
|-<br />
| VIB || HR || platform_vibration || - || TIME, N_VIB_SENSORS<br />
|-<br />
| PRES || HR || [sea]<math>^c</math>_water_pressure|| dbar || TIME<br />
|-<br />
| TEMP || HR || [sea]<math>^c</math>_water_temperature || degree_Celsius || TIME, N_T_SENSORS <br />
|-<br />
| ACC || O || platform_acceleration || m s-2 || TIME, N_ACC_SENSORS<br />
|-<br />
| CNDC || O || [sea]<math>^c</math>_water_electrical_conductivity || S m-1 || TIME_CTD, N_C_SENSORS<br />
|-<br />
| GRADT || O || derivative_of_[sea]<math>^c</math>_water_temperature_wrt_<math>^{*d}</math>|| degree_Celcius m-1 || TIME, N_GRADT_SENSORS <br />
|-<br />
| GRADC || O || derivative_of_[sea]<math>^c</math>_water_conductivity_wrt_<math>^{*d}</math> || - || TIME, N_GRADC_SENSORS<br />
|-<br />
| PITCH || O || platform_pitch_angle || degree || TIME<br />
|-<br />
| ROLL || O || platform_roll_angle || degree || TIME<br />
|}<br />
<math>^a</math>: Code for the requirement of variable, R: Required, HR: Highly recommended, O: Optional<br />
<br><math>^b</math>: Unit and offset need to be compatible with the Climate and Forecast (CF)-convention<br />
<br><math>^c</math>: User can choose between water or sea_water depending on the environment <br />
<br><math>^d</math>: wrt_x or wrt_s; spatial derivative. Typically derived from the rate of change of temperature and divided by the profiling speed. <br />
</div><br />
<br />
<br />
------<br />
return to [[Dataset requirements for shear probes]] <br><br />
go to next: [[Level 2 data (shear probes)| Level 2 data]].<br />
<br />
<br />
[[Category: Shear probes]]</div>Aleboyerhttp://atomix.app.uib.no/index.php?title=Benchmark_datasets_for_shear_probes&diff=4625Benchmark datasets for shear probes2024-06-06T15:01:18Z<p>Aleboyer: </p>
<hr />
<div>We compiled a collection of five benchmark data sets described in Scientific Data: [https://www.nature.com/articles/s41597-024-03323-y Fer et al. (2024)] <br />
<br />
These datasets can be accessed from the BODC center.<br />
<br />
We provide a number of Matlab routines to read in and compare data from grouped NetCDF files that follow our recommended structure on our [https://github.com/SCOR-ATOMIX/shear-probes GitHub repository]. The Matlab function ATOMIX_load.m there can be used to load a benchmark data file.<br />
<br />
{| class="wikitable sortable" <br />
|-<br />
! Link<br />
! Region<br />
! Instrument<br />
! Platform<br />
! PI (ATOMIX)<br />
! Comment<br />
|-<br />
| [https://doi.org/10.5285/0ec16a65-abdf-2822-e063-6c86abc06533]<br />
| Haro Strait<br />
| VMP-250<br />
| Ship<br />
| Lueck<br />
| Intense turbulence. Large up/down drafts. Most estimates require fitting to the inertial subrange.<br />
|-<br />
| [https://doi.org/10.5285/0ebffc86-ed32-5dde-e063-6c86abc08b3a]<br />
| Rockall Trough<br />
| Epsilometer<br />
| Ship<br />
| Le Boyer<br />
| Strong turbulence in a canyon in the North Atlantic (West of Ireland)<br />
|-<br />
| [https://doi.org/10.5285/05f21d1d-bf9c-5549-e063-6c86abc0b846]<br />
| Faroe Bank Channel (North Atlantic)<br />
| VMP-2000<br />
| Ship<br />
| Fer<br />
| Ranging from quiescent mid-water to turbulent, deep gravity current<br />
|-<br />
| [https://doi.org/10.5285/0e35f96f-57e3-540b-e063-6c86abc06660]<br />
| Baltic Sea<br />
| MSS<br />
| Ship<br />
| Holtermann<br />
| <br />
|-<br />
| [https://doi.org/10.5285/0ec17274-7a64-2b28-e063-6c86abc0ee02]<br />
| Minas Passage (Bay of Fundy, NS)<br />
| MicroRider<br />
| Mooring<br />
| Lueck<br />
| A swift tidal channel. All dissipation estimated from the inertial subrange<br />
|-<br />
|}<br />
<br />
<br />
<br />
[[Category: Shear probes]]<br />
<br />
-------------------------<br />
return to [[Shear probes]] <br><br><br />
ATOMIX members: see also [[Tentative benchmarks for shear probes| tentative benchmarks for testers]]</div>Aleboyer