Talk:Processing your ADCP data using structure function techniques

Jmmcmillan (talk) 21:50, 12 November 2021 (CET) Can we change the title of this page to be more specific? Like "Computing the structure functions and dissipation rates"

Yuengdjern (talk) 16:42, 15 November 2021 (CET) It's pretty hard to change page names without actually just creating new ones. we can 'move' pages and then there'll be a redirect shown at the top of the page, because 'move' pretty much creates a new page. Will play around with that.

Jmmcmillan (talk) 21:54, 12 November 2021 (CET) I find the use of $D(n,\delta )$ a little confusing. In reality, for each range bin and each time ensemble, D only depends on $\delta$. Can we drop the n for simplicity?

Djwain (talk) 22:44, 14 November 2021 (CET)I agree here - we don't actually define n on this page, so it is confusing to have it in the definition of v' and D. I think I wrote this originally and was basing it on the way my code is written, but really mean something like D_n(delta) where n denotes which bin we are differencing around or from.

Djwain (talk) 22:44, 14 November 2021 (CET)Having just looked through the differencing examples which use the D(n,delta) nomenclature and are quite clear I think, we probably need to stick with this (otherwise we have to change a bunch of other pages).

Djwain (talk) 22:51, 14 November 2021 (CET)I agree with Justine on the three ways of dealing with the data being somewhat confusing.- I am having a very hard time visualizing what is being explained in item 1 because I don't process my data this way. I think we probably need to be giving some guidance on why you might do a forward vs a center difference and maybe branch based on that. The forward difference example is really nice and helpful I think!

Yuengdjern (talk) 16:42, 15 November 2021 (CET) Right this seems to where the potential for contention arises, given the conversation here and also on Teams. So I think for now we pick the most common and add an aside that there are alternatives. Dumping text here so it is saved:

1. If $D_{ll}(n,\delta )$ was evaluated using a forward-difference scheme, the regression is done for the combined data from all bins in the selected range, hence the maximum number of $D_{ll}(n,\delta )$ values for each separation distance will be the number of bins in the range less 1 for $\delta$ = 1, reducing by 1 for each increment in $\delta$, with the regression ultimately yielding a single $\epsilon$ value for the data segment
2. If $D_{ll}(n,\delta )$ was evaluated using a bin-centred difference scheme, the regression can either be done:
   • for each bin individually, with a single $D(n,\delta )$ for each separation distance, ultimately yielding an $\epsilon$ for each bin; or
   • by combining the data for all of the bins, with each separation distance having a $D_{ll}(n,\delta )$ value for each bin, with the regression again ultimately yielding a single $\epsilon$ value for the data segment.
3. The regression is typically done as a least-squares fit, either as:
   
   $D_{ll}=a_0+a_1(\delta r_0{)}^{2/3}$; or as
   $D_{ll}=a_0+a_1(\delta r_0{)}^{2/3}+a_3((\delta r_0{)}^{2/3}{)}^3$
   
   the former being the canonical method that excludes non-turbulent velocity differences between bins, whereas the latter is a modified method that includes non-turbulent velocity differences between bins due to any oscillatory signal (e.g. surface waves, motion of the ADCP on a mooring).

Djwain (talk) 22:53, 14 November 2021 (CET)I also agree that step 7 should probably be in QA2

Yuengdjern (talk) 16:31, 15 November 2021 (CET) moved Step 7 to top of QA2

Djwain (talk) 22:53, 14 November 2021 (CET)A schematic is definitely needed here - SF is much easier to visualize I think than read! I don't have any schematics that I have made (and thus own for distribution).

Yuengdjern (talk) 16:42, 15 November 2021 (CET) JMM's comment about referring to QA2. I think I second that. Dumping the cut text here in case we want it later, but adding a link to QA2. Pasted text:

Some things to consider are: [JMM: SHOULD WE INCLUDE THESE HERE, OR MAKE A LINK TO A PAGE ABOUT QUALITY CONTROL METRICS TO COMPUTE?]

1. • Including $D(n,\delta )$ for $\delta =1$ may be inappropriate since the velocity estimates from adjacent bins are not wholly independent, therefore the impact of its inclusion should be evaluated
   • Keep a record of the number of instances when the squared velocity difference is evaluated for each bin $n$ and separation distance $\delta r_0$ and their distribution because they are potential quality control metrics
   • The impact of additional quality criteria can also be tested e.g. valid data requirements for all intermediate separation distances, so for a forward-difference scheme with $n=2$ and $\delta =5$, require all data in bins 2 to 7 to meet Level 1 QC requirements for the profile to be included when averaging to calculate $D(n,\delta )$

Yuengdjern (talk) 16:55, 15 November 2021 (CET) making a global change to the wiki page to change D to DLL because that's what our netcdf variable is.