Segmenting datasets: Difference between revisions
Line 9: | Line 9: | ||
* Statistical significance of the resulting spectra | * Statistical significance of the resulting spectra | ||
Measurements are typically collected in the following two ways: | Measurements are typically collected in the following two ways: | ||
Line 25: | Line 16: | ||
<div><ul> | <div><ul> | ||
<li style="display: inline-block; vertical-align: top;"> [[File:Short timeseries.png|thumb|none|350px| | <li style="display: inline-block; vertical-align: top;"> [[File:Short timeseries.png|thumb|none|350px|Z512 s segment of the measured velocities after applying different [[Detrending time series|detrending methods]]]] | ||
</li> | </li> | ||
<li style="display: inline-block; vertical-align: top;"> [[File:Short_spectra.png|thumb|none|350px|Example velocity spectra of the short 512 s of records before and after different detrending techniques applied to the original 6h time series. The impact of the detrending method can be seen at the lowest frequencies only]] </li> | <li style="display: inline-block; vertical-align: top;"> [[File:Short_spectra.png|thumb|none|350px|Example velocity spectra of the short 512 s of records before and after different detrending techniques applied to the original 6h time series. The impact of the detrending method can be seen at the lowest frequencies only]] </li> |
Revision as of 01:52, 5 July 2022
Once the raw observations have been quality-controlled, then you must split the time series into shorter segments by considering:
- Time and length scales of turbulence
- Stationarity of the segment and Taylor's frozen turbulence hypothesis
- Statistical significance of the resulting spectra
Measurements are typically collected in the following two ways:
- continuously, or in such long bursts that they can be considered continuous
- short bursts that are typically at most 2-3x the expected largest turbulence time scales (e.g., 10 min in ocean environments)
This segmenting step dictates the minimum burst duration when setting up your equipment. The act of chopping a time series into smaller subsets, i.e., segments, is effectively a form of low-pass (box-car) filtering. How to segment the time series is usually a more important consideration than detrending the time series since estimating
-
Z512 s segment of the measured velocities after applying different detrending methods -
Example velocity spectra of the short 512 s of records before and after different detrending techniques applied to the original 6h time series. The impact of the detrending method can be seen at the lowest frequencies only
Trade-offs
The shorter the segment, the higher the temporal resolution of the final
Rules of thumb
A good rule of thumb for tidally-influenced environments is 5 to 15 min segments.
