Talk:Level 1 data (velocity profilers) - Revision history

Jmmcmillan: Clarification questions

2022-03-21T19:42:57Z

Clarification questions

← Older revision		Revision as of 19:42, 21 March 2022
Line 31:		Line 31:

	I think we also agreed that the level 1 qaqc R_VEL_FLAGS should also be included as a required variable?		I think we also agreed that the level 1 qaqc R_VEL_FLAGS should also be included as a required variable?

			----
			[[User:Jmmcmillan\|Jmmcmillan]] ([[User talk:Jmmcmillan\|talk]]) 20:42, 21 March 2022 (CET) Can someone clarify the following:

			1. Should PROFILE_NUMBER be N_PROFILE (for consistency with other identifying integers like N_BEAM)? I noticed that in Cynthia's netcdf tools, the variable N_PROFILE exists, but PROFILE_NUMBER does not. The same comment applied to BURST_NUMBER.

			2. ABSIC is in counts, and ABSI is in dB? Should we just require one or the other and not both?

Brian scannell at 10:34, 17 January 2022

2022-01-17T10:34:05Z

← Older revision		Revision as of 10:34, 17 January 2022
Line 28:		Line 28:

	----		----
	[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 11:17, 17 January 2022 (CET) Dimensions for required variable PROFILE_NUMBER should just be TIME whilst the R_DIST dimension for R_VEL, ABSIC and CORR should now be Z_DIST		[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 11:17, 17 January 2022 (CET) Dimensions for required variable PROFILE_NUMBER should just be TIME whilst the R_DIST dimension for R_VEL, ABSIC and CORR should now be Z_DIST.

			I think we also agreed that the level 1 qaqc R_VEL_FLAGS should also be included as a required variable?

Brian scannell at 10:30, 17 January 2022

2022-01-17T10:30:50Z

← Older revision		Revision as of 10:30, 17 January 2022
Line 28:		Line 28:

	----		----
	[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 11:17, 17 January 2022 (CET) Dimensions for required variable PROFILE_NUMBER should just be TIME		[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 11:17, 17 January 2022 (CET) Dimensions for required variable PROFILE_NUMBER should just be TIME whilst the R_DIST dimension for R_VEL, ABSIC and CORR should now be Z_DIST

Brian scannell at 10:17, 17 January 2022

2022-01-17T10:17:21Z

← Older revision		Revision as of 10:17, 17 January 2022
Line 26:		Line 26:
	* RV was changed according TEAMS discussion.		* RV was changed according TEAMS discussion.
	* Burst_number was is not the same as N_SEGMENT. Nortek instrument spits data file into many "bursts" and it's useful to be able to refer back to the rawest dat files. I will let the user decide to add if desired.		* Burst_number was is not the same as N_SEGMENT. Nortek instrument spits data file into many "bursts" and it's useful to be able to refer back to the rawest dat files. I will let the user decide to add if desired.

			----
			[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 11:17, 17 January 2022 (CET) Dimensions for required variable PROFILE_NUMBER should just be TIME

CynthiaBluteau at 01:05, 15 January 2022

2022-01-15T01:05:51Z

← Older revision		Revision as of 01:05, 15 January 2022
Line 24:		Line 24:
	[[User:CynthiaBluteau\|CynthiaBluteau]] ([[User talk:CynthiaBluteau\|talk]]) 02:03, 15 January 2022 (CET) Responses:		[[User:CynthiaBluteau\|CynthiaBluteau]] ([[User talk:CynthiaBluteau\|talk]]) 02:03, 15 January 2022 (CET) Responses:
	* The units for ABSI and CORR are different from ABSIC (counts) and CORRN (%). They should be in the native unit, not converted.		* The units for ABSI and CORR are different from ABSIC (counts) and CORRN (%). They should be in the native unit, not converted.
			* RV was changed according TEAMS discussion.
			* Burst_number was is not the same as N_SEGMENT. Nortek instrument spits data file into many "bursts" and it's useful to be able to refer back to the rawest dat files. I will let the user decide to add if desired.

CynthiaBluteau at 01:03, 15 January 2022

2022-01-15T01:03:59Z

← Older revision		Revision as of 01:03, 15 January 2022
Line 20:		Line 20:

	[[User:Jmmcmillan\|Jmmcmillan]] ([[User talk:Jmmcmillan\|talk]]) 14:17, 7 January 2022 (CET) I agree that BURST_NUMBER shouldn't be required. It is useful for continuous data in the sense that it distinguishes the ensemble that the data are part of, but I think this could be better captured at level 2 in the segmented data. I also agree that we shouldn't limit ourselves to calling the vertical beam #5, and like Brian's suggestion of using a "V" instead.		[[User:Jmmcmillan\|Jmmcmillan]] ([[User talk:Jmmcmillan\|talk]]) 14:17, 7 January 2022 (CET) I agree that BURST_NUMBER shouldn't be required. It is useful for continuous data in the sense that it distinguishes the ensemble that the data are part of, but I think this could be better captured at level 2 in the segmented data. I also agree that we shouldn't limit ourselves to calling the vertical beam #5, and like Brian's suggestion of using a "V" instead.

			----
			[[User:CynthiaBluteau\|CynthiaBluteau]] ([[User talk:CynthiaBluteau\|talk]]) 02:03, 15 January 2022 (CET) Responses:
			* The units for ABSI and CORR are different from ABSIC (counts) and CORRN (%). They should be in the native unit, not converted.

Jmmcmillan: Response to Brian

2022-01-07T13:17:14Z

Response to Brian

← Older revision		Revision as of 13:17, 7 January 2022
Line 18:		Line 18:

	[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 14:00, 28 December 2021 (CET) Optional ancillary variables - what is dimension N_VEL_INSTRUMENT shown for variable PRES? Re note on TEMP - Kinematic viscosity is not required for the SF calculation.		[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 14:00, 28 December 2021 (CET) Optional ancillary variables - what is dimension N_VEL_INSTRUMENT shown for variable PRES? Re note on TEMP - Kinematic viscosity is not required for the SF calculation.

			[[User:Jmmcmillan\|Jmmcmillan]] ([[User talk:Jmmcmillan\|talk]]) 14:17, 7 January 2022 (CET) I agree that BURST_NUMBER shouldn't be required. It is useful for continuous data in the sense that it distinguishes the ensemble that the data are part of, but I think this could be better captured at level 2 in the segmented data. I also agree that we shouldn't limit ourselves to calling the vertical beam #5, and like Brian's suggestion of using a "V" instead.

Brian scannell at 11:45, 29 December 2021

2021-12-29T11:45:13Z

← Older revision		Revision as of 11:45, 29 December 2021
Line 1:		Line 1:
	[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 13:16, 28 December 2021 (CET) I don’t think the definitions work properly for instruments with a vertical beam. We are inconsistent in separating the “regular” and “vertical” beams. ~~Dimension~~ N_BEAM is defined as including the vertical beam, but is used as a dimension for R_VEL which doesn’t include the vertical beam data.		[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 13:16, 28 December 2021 (CET) I don’t think the definitions work properly for instruments with a vertical beam. We are inconsistent in separating the “regular” and “vertical” beams e.g. dimension N_BEAM is defined as including the vertical beam, but is used as a dimension for R_VEL which doesn’t include the vertical beam velocity data.

	There is also the issue identified in the notes that some instruments allow the vertical beam to be defined with a different sampling rate / operating mode / ensemble averaging / burst interval etc and have a different bin size, ~~implying a requirement for~~ a TIME5 dimension as well as ~~different~~ variables ~~where TIME5~~ is a dimension.		There is also the issue identified in the notes that some instruments allow the vertical beam to be defined with a different sampling rate / operating mode / ensemble averaging / burst interval etc and have a different bin size, requiring a suggested TIME5 dimension as well as separate variables wherever TIME is a dimension. I suggest it would be cleaner to always treat the angled and vertical beams separately.

	Should we also allow for the possibility of instruments with three angled beams and a vertical beam ~~- in which case~~ the use of the tag 5 on the variable names becomes somewhat inappropriate. I suggest we use the tag V for vertical (which is also Roman 5).		Should we also allow for the possibility of instruments with three angled beams and a vertical beam? If so, the use of the tag “5" on the variable names becomes somewhat inappropriate. I suggest we use the tag “V" for vertical (which is also Roman 5).

	~~It would seem clearer if the~~ mandatory dimensions remain TIME, R_DIST and N_BEAM but these only relate to the angled beams (so N_BEAM ~~is typically~~ [1 2 3] or [1 2 3 4]). The variables R_VEL, ABSI and CORR use all of these dimensions, whilst BURST_NUMBER uses TIME. THETA and BIN_SIZE are then constants (not sure why THETA is currently shown with dimension TIME).		The mandatory dimensions would remain as TIME, R_DIST and N_BEAM but these only relate to the angled beams (so N_BEAM would be either [1 2 3] or [1 2 3 4]). The variables R_VEL, ABSI and CORR use all of these dimensions, whilst BURST_NUMBER uses TIME. THETA and BIN_SIZE are then constants (not sure why THETA is currently shown with dimension TIME - I think that’s an error).

	We then have optional dimensions TIMEV and R_DISTV and optional variables R_VELV, ABSIV and CORRV with both of these dimensions; BURST_NUMBERV with dimension TIMEV; and BIN_SIZEV is a constant. If the sampling for the vertical beam is the same as for the angled beams, then TIMEV and TIME are simply the same - although I suspect this would be the exception.		We then have optional dimensions TIMEV and R_DISTV and optional variables R_VELV, ABSIV and CORRV with both of these dimensions; BURST_NUMBERV with dimension TIMEV; and BIN_SIZEV is a constant. If the sampling for the vertical beam is the same as for the angled beams, then TIMEV and TIME are simply the same - although I suspect this would be the exception.

	[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 15:39, 28 December 2021 (CET) ~~As an aside~~, ~~why are we defining~~ BURST_NUMBER as a mandatory variable? ~~I think this should be optional~~ if the data is collected on a continuous basis and ~~only needs~~ to ~~be recommended even if~~ the ~~data is collected in bursts~~ as ~~the information~~ is ~~inherent in~~ the TIME ~~dimension~~.		[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 12:45, 29 December 2021 (CET) The more I think about it, the less convinced I become that BURST_NUMBER is actually helpful - I’m certainly not convinced it should be a mandatory variable. It isn’t part of the raw data output from TRDI instruments (not sure about Nortek) and it serves no purpose if the data is collected on a continuous basis. Plus it implicit in the TIME dimension values anyway.

			More fundamentally, BURST_NUMBER replicates what N_SEGMENT does and I don’t think we need the added complexity of having both.

			What I think would be helpful would be to have a variable that is an integer count simply identifying each velocity profile instance (consisting of number of beams x number of bins velocity values) at level 1, this variable (PROFILE_NUMBER or N_PROF or something - I’m unclear on the naming convention) would have the dimension TIME and would simply increment 1, 2, 3 etc. but (with the N_BEAM and R_DIST) provides a unique identifier for each velocity value and as an integer is much easier to work with (e.g. for indexing) than the float TIME. With an equivalent for the vertical beam where appropriate.

	[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 13:49, 28 December 2021 (CET) Suggested units of ABSI are db. For the TRDI instruments the WA configuration setting for the fish rejection algorithm (which is likely to be familiar to any user configuring an instrument for deployment) is based on the echo intensity “count” returned by the instrument not db. So if users are implementing the fish rejection algorithm as part of their QC, I would expect they will use a count difference threshold rather than a db difference. The conversion between counts and db is very simple (1 count = 0.5 db), unless users want to get into distance compensation, but requiring the conversion to db may be unhelpful. Perhaps we should allow the user to specify either count or db?		[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 13:49, 28 December 2021 (CET) Suggested units of ABSI are db. For the TRDI instruments the WA configuration setting for the fish rejection algorithm (which is likely to be familiar to any user configuring an instrument for deployment) is based on the echo intensity “count” returned by the instrument not db. So if users are implementing the fish rejection algorithm as part of their QC, I would expect they will use a count difference threshold rather than a db difference. The conversion between counts and db is very simple (1 count = 0.5 db), unless users want to get into distance compensation, but requiring the conversion to db may be unhelpful. Perhaps we should allow the user to specify either count or db?

	[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 14:00, 28 December 2021 (CET) Optional ancillary variables - what is dimension N_VEL_INSTRUMENT shown for variable PRES? Re note on TEMP - Kinematic viscosity is not required for the SF calculation.		[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 14:00, 28 December 2021 (CET) Optional ancillary variables - what is dimension N_VEL_INSTRUMENT shown for variable PRES? Re note on TEMP - Kinematic viscosity is not required for the SF calculation.

Brian scannell at 14:39, 28 December 2021

2021-12-28T14:39:48Z

← Older revision		Revision as of 14:39, 28 December 2021
Line 8:		Line 8:

	We then have optional dimensions TIMEV and R_DISTV and optional variables R_VELV, ABSIV and CORRV with both of these dimensions; BURST_NUMBERV with dimension TIMEV; and BIN_SIZEV is a constant. If the sampling for the vertical beam is the same as for the angled beams, then TIMEV and TIME are simply the same - although I suspect this would be the exception.		We then have optional dimensions TIMEV and R_DISTV and optional variables R_VELV, ABSIV and CORRV with both of these dimensions; BURST_NUMBERV with dimension TIMEV; and BIN_SIZEV is a constant. If the sampling for the vertical beam is the same as for the angled beams, then TIMEV and TIME are simply the same - although I suspect this would be the exception.

			[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 15:39, 28 December 2021 (CET) As an aside, why are we defining BURST_NUMBER as a mandatory variable? I think this should be optional if the data is collected on a continuous basis and only needs to be recommended even if the data is collected in bursts as the information is inherent in the TIME dimension.

	[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 13:49, 28 December 2021 (CET) Suggested units of ABSI are db. For the TRDI instruments the WA configuration setting for the fish rejection algorithm (which is likely to be familiar to any user configuring an instrument for deployment) is based on the echo intensity “count” returned by the instrument not db. So if users are implementing the fish rejection algorithm as part of their QC, I would expect they will use a count difference threshold rather than a db difference. The conversion between counts and db is very simple (1 count = 0.5 db), unless users want to get into distance compensation, but requiring the conversion to db may be unhelpful. Perhaps we should allow the user to specify either count or db?		[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 13:49, 28 December 2021 (CET) Suggested units of ABSI are db. For the TRDI instruments the WA configuration setting for the fish rejection algorithm (which is likely to be familiar to any user configuring an instrument for deployment) is based on the echo intensity “count” returned by the instrument not db. So if users are implementing the fish rejection algorithm as part of their QC, I would expect they will use a count difference threshold rather than a db difference. The conversion between counts and db is very simple (1 count = 0.5 db), unless users want to get into distance compensation, but requiring the conversion to db may be unhelpful. Perhaps we should allow the user to specify either count or db?

	[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 14:00, 28 December 2021 (CET) Optional ancillary variables - what is dimension N_VEL_INSTRUMENT shown for variable PRES? Re note on TEMP - Kinematic viscosity is not required for the SF calculation.		[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 14:00, 28 December 2021 (CET) Optional ancillary variables - what is dimension N_VEL_INSTRUMENT shown for variable PRES? Re note on TEMP - Kinematic viscosity is not required for the SF calculation.

Brian scannell at 13:00, 28 December 2021

2021-12-28T13:00:19Z

← Older revision		Revision as of 13:00, 28 December 2021
Line 10:		Line 10:

	[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 13:49, 28 December 2021 (CET) Suggested units of ABSI are db. For the TRDI instruments the WA configuration setting for the fish rejection algorithm (which is likely to be familiar to any user configuring an instrument for deployment) is based on the echo intensity “count” returned by the instrument not db. So if users are implementing the fish rejection algorithm as part of their QC, I would expect they will use a count difference threshold rather than a db difference. The conversion between counts and db is very simple (1 count = 0.5 db), unless users want to get into distance compensation, but requiring the conversion to db may be unhelpful. Perhaps we should allow the user to specify either count or db?		[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 13:49, 28 December 2021 (CET) Suggested units of ABSI are db. For the TRDI instruments the WA configuration setting for the fish rejection algorithm (which is likely to be familiar to any user configuring an instrument for deployment) is based on the echo intensity “count” returned by the instrument not db. So if users are implementing the fish rejection algorithm as part of their QC, I would expect they will use a count difference threshold rather than a db difference. The conversion between counts and db is very simple (1 count = 0.5 db), unless users want to get into distance compensation, but requiring the conversion to db may be unhelpful. Perhaps we should allow the user to specify either count or db?

			[[User:Brian scannell\|Brian scannell]] ([[User talk:Brian scannell\|talk]]) 14:00, 28 December 2021 (CET) Optional ancillary variables - what is dimension N_VEL_INSTRUMENT shown for variable PRES? Re note on TEMP - Kinematic viscosity is not required for the SF calculation.