Level 2 data (velocity profilers): Difference between revisions

From Atomix
mNo edit summary
 
(28 intermediate revisions by 4 users not shown)
Line 1: Line 1:
The second processing level is separated into two NetCDF groups:
Velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which allows  [[Segmenting datasets|segmenting]] data using overlapping windows i.e., some velocity samples can belong to more than one segment.  
* [[Level 2 qaqc velocities (velocity profilers)| Level 2 qaqc velocities]]:  velocities from level 1 have now been quality-controlled and despiked.


* [[Level 2 segmented (velocity profilers)|Level 2 segmented]] (optional): velocities have now been [[Segmenting datasets|segmented]] and [[Detrending time series|detrended]]. Each segment is stored separately from each other, which accommodates  [[Segmenting datasets|segmenting datasets]] using an overlapping window i.e., some velocity samples can belong to more than one segment. This processing level is only required when an overlapping window is used during segmenting. For all other cases, the segment number and detrended velocities can be  provided in  [[Level 2 quality-controlled velocities (velocity profilers)| Level 2 qaqc velocities]].  
The dimensions and variables for this processing level are described below.  
Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the {{FontColor|bg=#fca1fd|text= [[NetCDF_parameter|complete list]]}} for the additional attributes related to each variable (e.g., units, bounds, cell_methods).
__TOC__


-----
=Dimensions=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand dimensions">
<br>
 
{| class="wikitable sortable"
|-
! Short name
! Standard name
! Dimensions
! Comments
|-
| N_SEGMENT
| unique_identifier_for_each_segment_in_the_
entire_available_timeseries
| N_SEGMENT
| Array of 1 to number of segments into which the data has been separated (Note TIME variable)
|-
| Z_DIST
| distance_from_sensor_along_vertical
| Z_DIST
| bin centre distance (in meters) from the transducer along the instrument's vertical axis
|-
| N_BEAM
| unique_identifier_for_each_beam
| N_BEAM
| Array of 1 to number of beams (3 to 5 typically)
|-
| N_SAMPLE
| unique_identifier_for_each_sample_within_
the_segment
| N_SAMPLE
| Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)
|}
</div>
 
=Variables=
<div class="mw-collapsible" id="raw" data-collapsetext="Collapse" data-expandtext="Expand variables">
<br>
{| class="wikitable"
|-
! Short name
! Standard name
! Dimensions
! Comment
|-
| R_VEL_DETRENDED
| water_radial_velocity_of_scatterers_
towards_instrument_detrended
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). {{FontColor|fg=white|bg=red|text= Link to own detrending page}}.  Corresponds to <math> b^\prime </math> in [[Nomenclature]]
|-
| TIME
| TIME
| N_SEGMENT, N_SAMPLE
| Units in Days since reference time specified in variable attribute
|-
| THETA
| beam_angle_from_instrument_z_axis
| N_BEAM
| Units in degrees, positive (usually ~20-30<math>^\circ</math>) except vertical-pointing beams (0<math>^\circ</math>)
|-
| BIN_SIZE
| instrument_measurement_volume_bin_size
| N_BEAM
| Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd" | '''Optional variables'''  <math>\ddagger</math>
|-
| PROFILE_NUMBER
| unique_identifier_for_each_profile
| N_SEGMENT, N_SAMPLE
| This variable identifies each record (velocity) profile and is a proxy for TIME
|-
| R_VEL_DETRENDED_FLAGS
| water_radial_velocity_of_scatterers_towards_
instrument_status_flag
| N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE
| This variable provides flags for outliers in each segment <math> \geq 2 </math> standard deviations.
|-
| colspan="4" |
<math>\ddagger</math> One could also re-write for convenience the optional variables in [[Level_1_data_(velocity_profilers)#Optional_variables|Level 1]] after restructuring into segments.
|}
</div>
 
=Group attributes (metadata)=
<div class="mw-collapsible"  id="raw_att" data-expandtext="Expand group attributes" data-collapsetext="Collapse attributes">
 
This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.
 
 
{| class="wikitable"
|-
! Attribute name
! Purpose
! Suggested content
|-
| processing_level
| Boilerplate about the content of the NetCDF group.
| <blockquote>''In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.''</blockquote>
|-
| segment_length
| Provide length in seconds of each segment
| Usually about 300 to 600 s dependent on [[Time_and_length_scales_of_turbulence|time and length scales of turbulence]]
|-
| segment_overlap_proportion
| Provide proportion overlap of each segment, i.e., window overlap
| Often set to 0 or 0.5
|-
| detrending_method
| Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis.
| Some examples include <blockquote>''High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries''</blockquote> <blockquote>''Linear detrending on each segment''</blockquote>
|-
| colspan="4" style="text-align:center; font-weight:bold; background-color:#f9eddd"|  Optional group attributes
|-
| comment (optional)
| Any additional information pertinent to other users who test their algorithms against the file.
|
|}
</div>
 
----
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]
Return to: [[Level 1 data (velocity profilers)|Level 1 raw]]


Go to: [[Level 2 qaqc velocities (velocity profilers)|Level 2 qaqc velocities]]
Go to [[Level 3 data (velocity profilers)|Level 3 structure function]]
[[Category:Velocity profilers]]

Latest revision as of 09:29, 11 July 2022

Velocities have now been segmented and detrended. Each segment is stored separately from each other, which allows segmenting data using overlapping windows i.e., some velocity samples can belong to more than one segment.

The dimensions and variables for this processing level are described below.

Only a few attributes for each variable are listed since the page's purpose is to describe the information layout within each NetCDF file. Please refer to the complete list for the additional attributes related to each variable (e.g., units, bounds, cell_methods).

Dimensions


Short name Standard name Dimensions Comments
N_SEGMENT unique_identifier_for_each_segment_in_the_

entire_available_timeseries

N_SEGMENT Array of 1 to number of segments into which the data has been separated (Note TIME variable)
Z_DIST distance_from_sensor_along_vertical Z_DIST bin centre distance (in meters) from the transducer along the instrument's vertical axis
N_BEAM unique_identifier_for_each_beam N_BEAM Array of 1 to number of beams (3 to 5 typically)
N_SAMPLE unique_identifier_for_each_sample_within_

the_segment

N_SAMPLE Array of 1 to number of samples in each segment (segment length in seconds multiplied by the sampling rate in Hz)

Variables


Short name Standard name Dimensions Comment
R_VEL_DETRENDED water_radial_velocity_of_scatterers_

towards_instrument_detrended

N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE High-frequency content of the along-beam velocities [m/s], which may include surface wave, motion contamination, in addition to the turbulence signal. Calculated from R_VEL by removing the segment mean or trend. Detrending method should be specified by detrending_method in group attributes (see below). Link to own detrending page. Corresponds to [math]\displaystyle{ b^\prime }[/math] in Nomenclature
TIME TIME N_SEGMENT, N_SAMPLE Units in Days since reference time specified in variable attribute
THETA beam_angle_from_instrument_z_axis N_BEAM Units in degrees, positive (usually ~20-30[math]\displaystyle{ ^\circ }[/math]) except vertical-pointing beams (0[math]\displaystyle{ ^\circ }[/math])
BIN_SIZE instrument_measurement_volume_bin_size N_BEAM Vertical size of the ADCP bins [m]. Usually the same for diverging beams.
Optional variables [math]\displaystyle{ \ddagger }[/math]
PROFILE_NUMBER unique_identifier_for_each_profile N_SEGMENT, N_SAMPLE This variable identifies each record (velocity) profile and is a proxy for TIME
R_VEL_DETRENDED_FLAGS water_radial_velocity_of_scatterers_towards_

instrument_status_flag

N_SEGMENT, Z_DIST, N_BEAM, N_SAMPLE This variable provides flags for outliers in each segment [math]\displaystyle{ \geq 2 }[/math] standard deviations.

[math]\displaystyle{ \ddagger }[/math] One could also re-write for convenience the optional variables in Level 1 after restructuring into segments.

Group attributes (metadata)

This section describes attributes that may provide additional information about how the data was processed and manipulated at this stage.


Attribute name Purpose Suggested content
processing_level Boilerplate about the content of the NetCDF group.

In this group, quality-controlled velocities have been split into smaller segments for processing. The timeseries are also detrended to recover the turbulence, and potentially surface wave and motion contamination.

segment_length Provide length in seconds of each segment Usually about 300 to 600 s dependent on time and length scales of turbulence
segment_overlap_proportion Provide proportion overlap of each segment, i.e., window overlap Often set to 0 or 0.5
detrending_method Specify which filter or technique was used to detrend velocities, which is required for spectral and turbulence analysis. Some examples include

High-pass YY order butterworth filter with XX seconds cutoff frequency on the entire burst/timeseries

Linear detrending on each segment

Optional group attributes
comment (optional) Any additional information pertinent to other users who test their algorithms against the file.

Return to: Level 1 raw

Go to Level 3 structure function