De-spike the shear-probe data

There is currently no standard method of de-spiking shear-probe data.

One method consists of calculating the absolute value of the (high-pass filtered) shear. A copy of this absolute shear signal is smoothed with a low-pass filter that has a cut-off frequency that is approximately the expected minimum duration of turbulence patches (usually about 1 meter divided by the speed of profiling). When the ratio of the instantaneous absolute shear divided by the smoothed absolute shear exceeds a threshold, the data is deemed to be a spike. A typical threshold is 8.

A spike usually consists of a number of contiguous samples. A region surrounding the spike is then replaced by a local mean calculated using data from both sides of the spike (but excluding the spike itself). Because the response of the shear probe to a collision with plankton -- the ringing of anomalously large amplitude – is a temporal response, the amount of data replaced by a local mean is usually of fixed duration and not fixed in length. Typically, the amount of data replaced is 20ms before a spike and 40ms after a spike. This algorithm is applied iteratively until no more spikes are detected.

The iterative application permits longer anomalies to be removed, such as those that might occur because of collisions with jelly fish and seaweed. The fraction of the data that is altered by a de-spiking routine must be noted for each diss-length segment because this is a quality-control metric. Dissipation estimates should be treated with caution if the fraction of altered data exceeds a few percent. But, there is currently no standard for what is an acceptable fraction.