Replacement strategies for missing velocities

Quality-control of raw velocities results in data loss, which usually must be replaced before computing the spectra necessary for obtaining [math]\displaystyle{ \varepsilon }[/math]. The number of missing samples that can be tolerated for computing reliable spectra was also investigated.

Data analysis tests

Considerations

Several techniques were considered for replacing the missing samples:

• Linear interpolation
• Using the variance of the signal, which is commonly used by those intending to compute eddy-covariances
• Unevenly spaced least-square Fourier transform (i.e., no replacement at all)

These replacement strategies were trialed with one of the cleanest benchmarks (Underice MAVS sampling at 8 Hz) for different

• Number of missing samples to identify a threshold where the segment should be completely discarded from further analysis
  • 10, 25, and 50% of the 30 min timeseries were removed
• Data loss (gap) duration
  • 1 sample
  • 8 samples (1 s)
  • 480 samples (60 s)
  • 960 samples (120 s)

A total section of 30 min was chosen as this coincides with the required segment length for deriving an estimate of [math]\displaystyle{ \varepsilon }[/math].

Results

Insert graphs with example spectra for different tests For all tests, the linear interpolation did the best job in recovering the original spectra, followed by the unevenly spaced techniques. However, the unevenly spaced fourier transforms behaved similarly (if not worse) than the variance replacement when the data loss was intermittent. Unsurprisingly, unevenly spaced techniques fair better if the data loss form long continuous gaps.

• For a given data loss (e.g., 10%), the original spectra was easier to recover with longer continuous gaps.

Recommendations

Always use linear interpolation, and record the percent good samples in each segment so that this variable can be used to reject epsilon with more than 10% data loss. This threshold may be relaxed (20%) if the data loss forms long continuous chunks of several seconds. The person processing the data should record the threshold used and report it in the NetCDF flags.