Mathjs: Extend std function for using flexible dimensions

[rwieruch]

The functionality is already there for the mean function:

const a = [[1,2,3], [4,5,6], [7,8,9]]
math.mean(a) // 5
math.mean(a, 1) // [4, 5, 6]
math.mean(a, 2) // [2, 5, 8]

But it is not possible for the std function. Perhaps it can be similar implemented to the mean function.

Ref.: https://github.com/josdejong/mathjs/issues/982#issuecomment-350501299

[bnlcas]

I would like this feature as well, and I am happy to work on this.

Before I start, I would like to discuss how the function API should be modified.
Unlike mean(), the second argument of std/var is already used for the weighting. Thus the function would be changed to std(data, weighting, dimension). In my experience, the default unbiased weighting is by far the most common, so that argument is often extraneous.
In the case where the default unbiased weighting used, which of the following be preferable:

math.std(data, UNBIASED, dimension) // require the user to specify a default
math.std(data, [], dimension) // allow the user to get an empty value to use the default weighting (used in Matlab)
math.std(data, dimension)  // allow dimension to be given without stating the weighting (assume a default weighting)

[josdejong]

@bnlcas I think you're right and the dimension is a more commonly used parameter than the normalization so dimension should be the second argument.

Since dimension is a number and normalization is a string, they should not bite each other and maybe we can simply support all of the following syntaxes:

math.std(data: Array | Matrix, dimension: number | BigNumber, normalization: string)
math.std(data: Array | Matrix, dimension: number | BigNumber)
math.std(data: Array | Matrix, normalization: string)

[bnlcas]

@josdejong I like your suggestion of putting dimension ahead of normalization. As you mentioned, these parameters have different types, so this change should not effect compatibility with existing code. I'll submit a pull request for this in the next few days.

[bnlcas]

@josdejong, I have a branch implementing this feature for var and std, however I had to use a slightly different algorithm to calculate the variance for a multidimensional array. While this might be fine in a single feature, this leads to redundant code, which is harder to maintain and optimize in the future.
In general, it would be better to have a single implementation of a given function for 1d arrays, and apply this function in the multidimensional case using a map like function (like apply in R). This would make it much easier to apply arbitrary functions over the rows or columns of a matrix, and it would simplify the implementation and maintenance of these functions in the future.
I would be happy to implement these utility functions, but please let me know your thoughts on the matter.

[josdejong]

@bnlcas thanks for bringing this up. Now that we have more and more functions supporting operations over rows/columns it makes sense to spend effort in creating util functions to do this in a unified way. So far the reduce util function was suitable. I'm all in favor of improving code re-usability in this regard so if you see need to make changes here I'm all open for it. Maybe we should indeed create a new (public) function like the apply function of R to accommodate these type of operations (we can learn a lot from R, Python, Matlab, etc). I don't have a clear picture on what needs you have right now extending var and std, looking forward to see what you're cooking :).

[bnlcas]

@josdejong I just created a feature request for an apply function. I will work on it when I get some free time over the holidays. Once that is in, I think that it will be very simple to tweak std and var.

[josdejong]

@bnlcas awesome, sounds like a plan!

[bnlcas]

Now that apply has been merged, I will submit a pull request for this feature.