Agreement between dissipation estimates: Difference between revisions

From Atomix
Rolf (talk | contribs)
No edit summary
No edit summary
 
(15 intermediate revisions by 3 users not shown)
Line 1: Line 1:
When two or more shear probes, in close proximity, are used to collect simultaneous data, the rate of dissipation derived from the simultaneous data will not agree exactly.
When two or more shear probes, in close proximity, are used to collect simultaneous data, the rate of dissipation derived from such data will not agree exactly.
Even for nearly flawless measurements, there will be disagreement for purely statistical reasons.
Even for nearly flawless measurements, there will be disagreement for purely statistical reasons.
Measuring a turbulent shear is sampling a statistical process.
Measuring a turbulent shear is sampling a statistical process.
The sample variance will differ from the population variance and this difference will reduce with increasing length of data in the sample.
The sample variance will differ from the population variance and this difference will reduce with the increasing length of data in the sample.
The sampling uncertainty is distributed lognormally with a variance of  
The sampling uncertainty is distributed log-normally with a variance of  


<math> \sigma^2_{\ln\varepsilon} = \frac{5.5}{1 + \left(\hat{L}_f/4\right)^{7/9}}\ \ ,\ \ \hat{L}_f \equiv \hat{L} V_f^{3/4} = \frac{L}{L_K} V_f^{3/4} </math>
<math> \sigma^2_{\ln\varepsilon} = \frac{5.5}{1 + \left(\hat{L}_f/4\right)^{7/9}}\ \ ,\ \ \hat{L}_f \equiv \hat{L} V_f^{3/4} = \frac{L}{L_K} V_f^{3/4} </math>


where <math>L_K=\left(\nu^3/\varepsilon \right)^{1/4}</math> is the Kolmogorov length, and <math>V_f</math> is the fraction of the shear variance that is resolved by terminating the spectral integration at an upper wavenumber of <math>k_u</math><ref name=“Lueck2022a”> Lueck, R. G., 2022a: The statistics of oceanic turbulence measurements. Part 1: Shear variance and dissipation rates. J. Atmos. Oceanic Technol., –, in press, doi:--.</ref> .
where <math>L_K=\left(\nu^3/\varepsilon \right)^{1/4}</math> is the Kolmogorov length, and <math>V_f</math> is the fraction of the shear variance that is resolved by terminating the spectral integration at an upper wavenumber of <math>k_u</math><ref name=“Lueck2022a”> Lueck, R. G., 2022a: The statistics of oceanic turbulence measurements. Part 1: Shear variance and dissipation rates. J. Atmos. Oceanic Technol., –, in press, doi:--.</ref>. The <math>L</math> is the physical length of data, in m, used for producing the <math>\varepsilon</math> estimate. 


The 95% confidence interval on an individual dissipation estimate, <math>\varepsilon_1</math> is thus  
The 95% confidence interval on an individual dissipation estimate, <math>\varepsilon_1</math> is thus  
Line 15: Line 15:
The 95% confidence interval for the geometric mean of a pair of dissipation estimates is
The 95% confidence interval for the geometric mean of a pair of dissipation estimates is


<math> \mathrm{CF_{95}} = \sqrt{\varepsilon_1\,\varepsilon_2} \, \exp\left(\pm1.96\,\sigma_{\ln\varepsilon}/\sqrt{2} \right) \ \ .</math>
<math> \mathrm{CF_{95}} = \sqrt{\varepsilon_1\,\varepsilon_2} \, \exp\left(\pm1.96\,\sigma_{\ln\varepsilon}\sqrt{1/2} \right) \ \ .</math>
 
Thus, there is less than a 5% chance that the ratio of two simultaneous dissipation estimates is outside of the range of
 
<math> \exp\left(\pm1.96\,\sigma_{\ln\varepsilon}\,\sqrt{2} \right) \ \ .</math>
 
 
Because the rate of dissipation is not identical for a pair of probes, the estimate of the standard deviation, <math>\sigma_{\ln\varepsilon}</math>, will also differ.
But, only slightly because of the quarter-power dependence of the Kolmogorov length on the rate of dissipation.
So, one could use either the smaller of the two standard deviations or their average for the testing of the dissipation ratios. In practice, if we have two dissipation estimates, we sort them into ascending order and the natural logarithm of their ratio or the difference <math>\ln\varepsilon_2 - \ln\varepsilon_1</math>, should be less than the threshold <math>1.96\sqrt{2} = 2.772</math> times <math>\sigma_{\ln\varepsilon}</math>.
 
If there are more than two simultaneous dissipation estimates, then they should be sorted into ascending order.
The ratio test is first applied to the smaller and the largest dissipation estimate.
If this pair passes the test and all other pairs will also pass the ratio test.
If this pair does not pass the test, then the larger of the two should be flagged, and the test applied to the next largest (and the smallest) until a pair passes the test, or all pairs have been tested.
 
It is most likely that the larger of a pair of estimates is the erroneous one because signal contamination and other data flaws usually act to increase the variance of the shear=probe signal.
 
Very large dissipation rates require a fit to the inertial subrange in order to estimate the rate of dissipation because the shear probe will not resolve the shear variance.
In the inertial subrange, the spectrum of shear is
 
<math> \Psi(k) = A\, \varepsilon^{2/3}\left(2\pi\right)^2 k^{1/3}</math>
 
where the factor of <math>A</math> is approximately 0.2, depending on the model one wishes to use.
 
The sampling uncertainty of a spectrum is also distributed log-normally with a sampling variance of
 
<math> \sigma^2_{\ln\Psi} = \frac{5}{4}\, \left(N_f - N_V  \right)^{-7/9} </math>
 
where <math>N_f</math> is the number of fft-segments used to estimate the spectrum and <math>N_V</math> is the number of vibration and other signals that were used to clean the shear spectrum
<ref name=“Lueck2022b”> Lueck, R. G., 2022b: The statistics of oceanic turbulence measurements. Part 2: Shear spectra and a new spectral model. J. Atmos. Oceanic Technol., –, in press, doi:--.</ref> .
Thus, dividing the spectrum (in the inertial subrange) by <math>A(2\pi)^2\, k^{1/3}</math> and taking its logarithm provides samples of <math>\ln\varepsilon^{2/3}</math> that have a population standard deviation of <math>\sigma_{\ln\Psi}</math>.
Their average has a standard deviation of <math>\sigma_{\ln\Psi}/\sqrt{N_s}</math> where <math>N_s</math> is the number of spectral values in the inertial subrange.
 
Thus, the 95% confidence range of the rate of dissipation derived from a fit to the logarithm of the spectrum in the inertial subrange is
 
<math> \mathrm{CF}_{95} = \varepsilon\, \left[\exp\left(\pm1.96\,\sigma_{\ln\Psi}/\sqrt{N_s} \right) \right]^{3/2} \ .</math>
 
There is less than a 5% chance that the ratio of a pair of such dissipation estimates falls outside of the range of
 
<math>  \exp\left(\pm1.96\,(3/2)\sqrt{2}\sigma_{\ln\Psi}\,/\sqrt{N_s} \right)  \ .</math>




Line 21: Line 61:
==References==
==References==
<references />
<references />
 
-----------------------------
return to [[Flow chart for shear probes]]


[[Category:Shear probes]]
[[Category:Shear probes]]

Latest revision as of 09:49, 3 August 2023

When two or more shear probes, in close proximity, are used to collect simultaneous data, the rate of dissipation derived from such data will not agree exactly. Even for nearly flawless measurements, there will be disagreement for purely statistical reasons. Measuring a turbulent shear is sampling a statistical process. The sample variance will differ from the population variance and this difference will reduce with the increasing length of data in the sample. The sampling uncertainty is distributed log-normally with a variance of

[math]\displaystyle{ \sigma^2_{\ln\varepsilon} = \frac{5.5}{1 + \left(\hat{L}_f/4\right)^{7/9}}\ \ ,\ \ \hat{L}_f \equiv \hat{L} V_f^{3/4} = \frac{L}{L_K} V_f^{3/4} }[/math]

where [math]\displaystyle{ L_K=\left(\nu^3/\varepsilon \right)^{1/4} }[/math] is the Kolmogorov length, and [math]\displaystyle{ V_f }[/math] is the fraction of the shear variance that is resolved by terminating the spectral integration at an upper wavenumber of [math]\displaystyle{ k_u }[/math][1]. The [math]\displaystyle{ L }[/math] is the physical length of data, in m, used for producing the [math]\displaystyle{ \varepsilon }[/math] estimate.

The 95% confidence interval on an individual dissipation estimate, [math]\displaystyle{ \varepsilon_1 }[/math] is thus

[math]\displaystyle{ \mathrm{CF_{95}}(\varepsilon_1) = \varepsilon_1\, \exp\left(\pm1.96\,\sigma_{\ln\varepsilon} \right) \ \ . }[/math]

The 95% confidence interval for the geometric mean of a pair of dissipation estimates is

[math]\displaystyle{ \mathrm{CF_{95}} = \sqrt{\varepsilon_1\,\varepsilon_2} \, \exp\left(\pm1.96\,\sigma_{\ln\varepsilon}\sqrt{1/2} \right) \ \ . }[/math]

Thus, there is less than a 5% chance that the ratio of two simultaneous dissipation estimates is outside of the range of

[math]\displaystyle{ \exp\left(\pm1.96\,\sigma_{\ln\varepsilon}\,\sqrt{2} \right) \ \ . }[/math]


Because the rate of dissipation is not identical for a pair of probes, the estimate of the standard deviation, [math]\displaystyle{ \sigma_{\ln\varepsilon} }[/math], will also differ. But, only slightly because of the quarter-power dependence of the Kolmogorov length on the rate of dissipation. So, one could use either the smaller of the two standard deviations or their average for the testing of the dissipation ratios. In practice, if we have two dissipation estimates, we sort them into ascending order and the natural logarithm of their ratio or the difference [math]\displaystyle{ \ln\varepsilon_2 - \ln\varepsilon_1 }[/math], should be less than the threshold [math]\displaystyle{ 1.96\sqrt{2} = 2.772 }[/math] times [math]\displaystyle{ \sigma_{\ln\varepsilon} }[/math].

If there are more than two simultaneous dissipation estimates, then they should be sorted into ascending order. The ratio test is first applied to the smaller and the largest dissipation estimate. If this pair passes the test and all other pairs will also pass the ratio test. If this pair does not pass the test, then the larger of the two should be flagged, and the test applied to the next largest (and the smallest) until a pair passes the test, or all pairs have been tested.

It is most likely that the larger of a pair of estimates is the erroneous one because signal contamination and other data flaws usually act to increase the variance of the shear=probe signal.

Very large dissipation rates require a fit to the inertial subrange in order to estimate the rate of dissipation because the shear probe will not resolve the shear variance. In the inertial subrange, the spectrum of shear is

[math]\displaystyle{ \Psi(k) = A\, \varepsilon^{2/3}\left(2\pi\right)^2 k^{1/3} }[/math]

where the factor of [math]\displaystyle{ A }[/math] is approximately 0.2, depending on the model one wishes to use.

The sampling uncertainty of a spectrum is also distributed log-normally with a sampling variance of

[math]\displaystyle{ \sigma^2_{\ln\Psi} = \frac{5}{4}\, \left(N_f - N_V \right)^{-7/9} }[/math]

where [math]\displaystyle{ N_f }[/math] is the number of fft-segments used to estimate the spectrum and [math]\displaystyle{ N_V }[/math] is the number of vibration and other signals that were used to clean the shear spectrum [2] . Thus, dividing the spectrum (in the inertial subrange) by [math]\displaystyle{ A(2\pi)^2\, k^{1/3} }[/math] and taking its logarithm provides samples of [math]\displaystyle{ \ln\varepsilon^{2/3} }[/math] that have a population standard deviation of [math]\displaystyle{ \sigma_{\ln\Psi} }[/math]. Their average has a standard deviation of [math]\displaystyle{ \sigma_{\ln\Psi}/\sqrt{N_s} }[/math] where [math]\displaystyle{ N_s }[/math] is the number of spectral values in the inertial subrange.

Thus, the 95% confidence range of the rate of dissipation derived from a fit to the logarithm of the spectrum in the inertial subrange is

[math]\displaystyle{ \mathrm{CF}_{95} = \varepsilon\, \left[\exp\left(\pm1.96\,\sigma_{\ln\Psi}/\sqrt{N_s} \right) \right]^{3/2} \ . }[/math]

There is less than a 5% chance that the ratio of a pair of such dissipation estimates falls outside of the range of

[math]\displaystyle{ \exp\left(\pm1.96\,(3/2)\sqrt{2}\sigma_{\ln\Psi}\,/\sqrt{N_s} \right) \ . }[/math]


References

  1. Lueck, R. G., 2022a: The statistics of oceanic turbulence measurements. Part 1: Shear variance and dissipation rates. J. Atmos. Oceanic Technol., –, in press, doi:--.
  2. Lueck, R. G., 2022b: The statistics of oceanic turbulence measurements. Part 2: Shear spectra and a new spectral model. J. Atmos. Oceanic Technol., –, in press, doi:--.

return to Flow chart for shear probes