De-spike the shear-probe data - Revision history

CynthiaBluteau: Category added

2021-11-23T15:11:13Z

Category added

← Older revision		Revision as of 15:11, 23 November 2021
Line 34:		Line 34:

	return to [[Flow chart for shear probes]]		return to [[Flow chart for shear probes]]


			[[Category:Shear probes]]

Rolf at 20:18, 19 November 2021

2021-11-19T20:18:51Z

← Older revision		Revision as of 20:18, 19 November 2021
Line 32:		Line 32:
	----------------------------		----------------------------
	return to [[De-spiking parameters]]		return to [[De-spiking parameters]]

	return to [[Flow chart for shear probes]]		return to [[Flow chart for shear probes]]

Rolf at 20:18, 19 November 2021

2021-11-19T20:18:33Z

← Older revision		Revision as of 20:18, 19 November 2021
Line 31:		Line 31:

	----------------------------		----------------------------
			return to [[De-spiking parameters]]
	return to [[Flow chart for shear probes]]		return to [[Flow chart for shear probes]]

Rolf at 20:14, 19 November 2021

2021-11-19T20:14:44Z

← Older revision		Revision as of 20:14, 19 November 2021
Line 23:		Line 23:

	A spike usually consists of a number of contiguous samples.		A spike usually consists of a number of contiguous samples.
	A region surrounding the spike is then replaced by a local mean calculated using data from both sides of the spike (but excluding the spike itself).		A region surrounding the spike is then replaced by a local mean calculated using data from both sides of the spike (but excluding the <math>N</math> samples after and the <math>N/2</math> samples before the spike itself).
	Because the response of the shear probe to a collision with plankton -- the ringing of anomalously large amplitude – is a temporal response, the amount of data replaced by a local mean is usually of fixed duration and not fixed in length. Typically, the amount of data replaced is 20ms before a spike and 40ms after a spike, because 40ms seems to be the typical relaxation time due to a collision with zooplankton. Some data before the collision is removed because of the run up to the peak of a spike.		Because the response of the shear probe to a collision with plankton -- the ringing of anomalously large amplitude – is a temporal response, the amount of data replaced by a local mean is usually of fixed duration and not fixed in length. Typically, the amount of data replaced is 20ms before a spike and 40ms after a spike, because 40ms seems to be the typical relaxation time due to a collision with zooplankton. Some data before the collision is removed because of the run up to the peak of a spike.

Rolf at 20:11, 19 November 2021

2021-11-19T20:11:18Z

← Older revision		Revision as of 20:11, 19 November 2021
Line 20:		Line 20:
	Thus, if a vertical profiler is moving at a speed of <math>0.5\ \mathrm{m\, s^{-1}}</math>, then the cutoff frequency should be no higher than <math>1\ \mathrm{Hz}</math>.		Thus, if a vertical profiler is moving at a speed of <math>0.5\ \mathrm{m\, s^{-1}}</math>, then the cutoff frequency should be no higher than <math>1\ \mathrm{Hz}</math>.
	Gliders that move more slowly and at an angle of <math>30^{\circ}</math> with respect to the horizontal should use a lower cutoff frequency to establish a neighbourhood for comparison of the ratio of signals, because they will take longer to pass through a patch of turbulence.		Gliders that move more slowly and at an angle of <math>30^{\circ}</math> with respect to the horizontal should use a lower cutoff frequency to establish a neighbourhood for comparison of the ratio of signals, because they will take longer to pass through a patch of turbulence.
			A cutoff frequency of <math>0.5\ \mathrm{Hz}</math> is often used for gliders.

	A spike usually consists of a number of contiguous samples.		A spike usually consists of a number of contiguous samples.

Rolf at 20:09, 19 November 2021

2021-11-19T20:09:34Z

← Older revision		Revision as of 20:09, 19 November 2021
Line 16:		Line 16:

	What is a suitable neighbourhood and low-pass cutoff frequency?		What is a suitable neighbourhood and low-pass cutoff frequency?
	Turbulent patches in the ocean ~~seem to~~ seldom be thinner than about <math>0.5\ \mathrm{m}</math> in the vertical direction.		Turbulent patches in the ocean are seldom thinner than about <math>0.5\ \mathrm{m}</math> in the vertical direction.
	This can serve as a lower limit to the neighbourhood and an upper limit to the cutoff frequency.		This can serve as a lower limit to the neighbourhood and an upper limit to the cutoff frequency.
	Thus, if a vertical profiler is moving at a speed of <math>0.5\ \mathrm{m\, s^{-1}}</math>, then the cutoff frequency should be no higher than <math>1\ \mathrm{Hz}</math>.		Thus, if a vertical profiler is moving at a speed of <math>0.5\ \mathrm{m\, s^{-1}}</math>, then the cutoff frequency should be no higher than <math>1\ \mathrm{Hz}</math>.

Rolf at 20:06, 19 November 2021

2021-11-19T20:06:56Z

← Older revision		Revision as of 20:06, 19 November 2021
Line 21:		Line 21:
	Gliders that move more slowly and at an angle of <math>30^{\circ}</math> with respect to the horizontal should use a lower cutoff frequency to establish a neighbourhood for comparison of the ratio of signals, because they will take longer to pass through a patch of turbulence.		Gliders that move more slowly and at an angle of <math>30^{\circ}</math> with respect to the horizontal should use a lower cutoff frequency to establish a neighbourhood for comparison of the ratio of signals, because they will take longer to pass through a patch of turbulence.

	A spike usually consists of a number of contiguous samples. A region surrounding the spike is then replaced by a local mean calculated using data from both sides of the spike (but excluding the spike itself). Because the response of the shear probe to a collision with plankton -- the ringing of anomalously large amplitude – is a temporal response, the amount of data replaced by a local mean is usually of fixed duration and not fixed in length. Typically, the amount of data replaced is 20ms before a spike and 40ms after a spike~~. This algorithm is applied iteratively until no more spikes are detected.~~		A spike usually consists of a number of contiguous samples.
			A region surrounding the spike is then replaced by a local mean calculated using data from both sides of the spike (but excluding the spike itself).
	~~The question of how many points around a spike should~~ be ~~removed is determined by~~ the typical relaxation time ~~of the shear-probe~~ to a collision with zooplankton.		Because the response of the shear probe to a collision with plankton -- the ringing of anomalously large amplitude – is a temporal response, the amount of data replaced by a local mean is usually of fixed duration and not fixed in length. Typically, the amount of data replaced is 20ms before a spike and 40ms after a spike, because 40ms seems to be the typical relaxation time due to a collision with zooplankton. Some data before the collision is removed because of the run up to the peak of a spike.
	~~Such anomalies seem~~ to ~~last about <math>0.04\ \mathrm{s}</math>, and so~~ a ~~good choice is <math>N=0.04\, f_s</math> where <math>f_s</math> is the sampling rate~~.

	The iterative application permits longer anomalies to be removed, such as those that might occur because of collisions with jelly fish and seaweed. The fraction of the data that is altered by a de-spiking routine must be noted for each diss-length segment because this is a quality-control metric. Dissipation estimates should be treated with caution if the fraction of altered data exceeds a few percent. But, there is currently no standard for what is an acceptable fraction.		The iterative application permits longer anomalies to be removed, such as those that might occur because of collisions with jelly fish and seaweed. The fraction of the data that is altered by a de-spiking routine must be noted for each diss-length segment because this is a quality-control metric. Dissipation estimates should be treated with caution if the fraction of altered data exceeds a few percent. But, there is currently no standard for what is an acceptable fraction.


	~~-------~~


	~~This algorithm is often applied iteratively until no more anomalies are detected.~~

	One method consists of calculating the absolute value of the (high-pass filtered) shear. A copy of this absolute shear signal is smoothed with a low-pass filter that has a cut-off frequency that is approximately the expected minimum duration of turbulence patches (usually about 1 meter divided by the speed of profiling). When the ratio of the instantaneous absolute shear divided by the smoothed absolute shear exceeds a threshold, the data is deemed to be a spike. A typical threshold is 8.

	A spike usually consists of a number of contiguous samples. A region surrounding the spike is then replaced by a local mean calculated using data from both sides of the spike (but excluding the spike itself). Because the response of the shear probe to a collision with plankton -- the ringing of anomalously large amplitude – is a temporal response, the amount of data replaced by a local mean is usually of fixed duration and not fixed in length. Typically, the amount of data replaced is 20ms before a spike and 40ms after a spike. This algorithm is applied iteratively until no more spikes are detected.

	The iterative application permits longer anomalies to be removed, such as those that might occur because of collisions with jelly fish and seaweed. The fraction of the data that is altered by a de-spiking routine must be noted for each diss-length segment because this is a quality-control metric. Dissipation estimates should be treated with caution if the fraction of altered data exceeds a few percent. But, there is currently no standard for what is an acceptable fraction.

Rolf at 19:57, 19 November 2021

2021-11-19T19:57:04Z

@@ Line 1: / Line 1: @@
 There is currently no standard method of de-spiking shear-probe data.
 One method consists of calculating the absolute value of the (high-pass filtered) shear. A copy of this absolute shear signal is smoothed with a low-pass filter that has a cut-off frequency that is approximately the expected minimum duration of turbulence patches (usually about 1 meter divided by the speed of profiling). When the ratio of the instantaneous absolute shear divided by the smoothed absolute shear exceeds a threshold, the data is deemed to be a spike. A typical threshold is 8.
@@ Line 6: / Line 39: @@
 The iterative application permits longer anomalies to be removed, such as those that might occur because of collisions with jelly fish and seaweed. The fraction of the data that is altered by a de-spiking routine must be noted for each diss-length segment because this is a quality-control metric. Dissipation estimates should be treated with caution if the fraction of altered data exceeds a few percent. But, there is currently no standard for what is an acceptable fraction.

Aleboyer at 17:43, 20 August 2021

2021-08-20T17:43:02Z

← Older revision		Revision as of 17:43, 20 August 2021
Line 1:		Line 1:
	There is currently no standard method of de-spiking shear-probe data.		There is currently no standard method of de-spiking shear-probe data.

	One method consists of calculating the absolute value of the (high-pass filtered) shear. A copy of this absolute shear signal is smoothed with a low-pass filter that has a cut-off frequency that is approximately the expected minimum duration of turbulence patches (usually about 1 meter divided by the speed of profiling). When the ratio of the instantaneous absolute shear divided by the smoothed absolute shear exceeds a threshold, the data is deemed to be a spike.		One method consists of calculating the absolute value of the (high-pass filtered) shear. A copy of this absolute shear signal is smoothed with a low-pass filter that has a cut-off frequency that is approximately the expected minimum duration of turbulence patches (usually about 1 meter divided by the speed of profiling). When the ratio of the instantaneous absolute shear divided by the smoothed absolute shear exceeds a threshold, the data is deemed to be a spike. A typical threshold is 8.

	~~A typical threshold is 8.~~ A spike usually consists of a number of contiguous samples. A region surrounding the spike is then replaced by a local mean calculated using data from both sides of the spike (but excluding the spike itself). Because the response of the shear probe to a collision with plankton -- the ringing of anomalously large amplitude – is a temporal response, the amount of data replaced by a local mean is usually of fixed duration and not fixed in length. Typically, the amount of data replaced is 20ms before a spike and 40ms after a spike. This algorithm is applied iteratively until no more spikes are detected.		A spike usually consists of a number of contiguous samples. A region surrounding the spike is then replaced by a local mean calculated using data from both sides of the spike (but excluding the spike itself). Because the response of the shear probe to a collision with plankton -- the ringing of anomalously large amplitude – is a temporal response, the amount of data replaced by a local mean is usually of fixed duration and not fixed in length. Typically, the amount of data replaced is 20ms before a spike and 40ms after a spike. This algorithm is applied iteratively until no more spikes are detected.

	The iterative application permits longer anomalies to be removed, such as those that might occur because of collisions with jelly fish and seaweed. The fraction of the data that is altered by a de-spiking routine must be noted for each diss-length segment because this is a quality-control metric. Dissipation estimates should be treated with caution if the fraction of altered data exceeds a few percent. But, there is currently no standard for what is an acceptable fraction.		The iterative application permits longer anomalies to be removed, such as those that might occur because of collisions with jelly fish and seaweed. The fraction of the data that is altered by a de-spiking routine must be noted for each diss-length segment because this is a quality-control metric. Dissipation estimates should be treated with caution if the fraction of altered data exceeds a few percent. But, there is currently no standard for what is an acceptable fraction.

Aleboyer at 17:42, 20 August 2021

2021-08-20T17:42:28Z

← Older revision		Revision as of 17:42, 20 August 2021
Line 1:		Line 1:
	There is currently no standard method of de-spiking shear-probe data. One method consists of calculating the absolute value of the (high-pass filtered) shear. A copy of this absolute shear signal is smoothed with a low-pass filter that has a cut-off frequency that is approximately the expected minimum duration of turbulence patches (usually about 1 meter divided by the speed of profiling). When the ratio of the instantaneous absolute shear divided by the smoothed absolute shear exceeds a threshold, the data is deemed to be a spike. A typical threshold is 8. A spike usually consists of a number of contiguous samples. A region surrounding the spike is then replaced by a local mean calculated using data from both sides of the spike (but excluding the spike itself). Because the response of the shear probe to a collision with plankton -- the ringing of anomalously large amplitude – is a temporal response, the amount of data replaced by a local mean is usually of fixed duration and not fixed in length. Typically, the amount of data replaced is 20ms before a spike and 40ms after a spike. This algorithm is applied iteratively until no more spikes are detected. The iterative application permits longer anomalies to be removed, such as those that might occur because of collisions with jelly fish and seaweed. The fraction of the data that is altered by a de-spiking routine must be noted for each diss-length segment because this is a quality-control metric. Dissipation estimates should be treated with caution if the fraction of altered data exceeds a few percent. But, there is currently no standard for what is an acceptable fraction.		There is currently no standard method of de-spiking shear-probe data.

			One method consists of calculating the absolute value of the (high-pass filtered) shear. A copy of this absolute shear signal is smoothed with a low-pass filter that has a cut-off frequency that is approximately the expected minimum duration of turbulence patches (usually about 1 meter divided by the speed of profiling). When the ratio of the instantaneous absolute shear divided by the smoothed absolute shear exceeds a threshold, the data is deemed to be a spike.

			A typical threshold is 8. A spike usually consists of a number of contiguous samples. A region surrounding the spike is then replaced by a local mean calculated using data from both sides of the spike (but excluding the spike itself). Because the response of the shear probe to a collision with plankton -- the ringing of anomalously large amplitude – is a temporal response, the amount of data replaced by a local mean is usually of fixed duration and not fixed in length. Typically, the amount of data replaced is 20ms before a spike and 40ms after a spike. This algorithm is applied iteratively until no more spikes are detected.

			The iterative application permits longer anomalies to be removed, such as those that might occur because of collisions with jelly fish and seaweed. The fraction of the data that is altered by a de-spiking routine must be noted for each diss-length segment because this is a quality-control metric. Dissipation estimates should be treated with caution if the fraction of altered data exceeds a few percent. But, there is currently no standard for what is an acceptable fraction.