Base Julia provides the rand and rand! functions, which do random sampling over various domains. The StatsBase package provides sample and sample! with partially overlapping functionality. There have been various threads (e.g. this one) where people are discussion additional functionality for one or the other. I'm opening this issue to discuss whether these interfaces should be separate or if they should be merged and if they are to remain separate, to clarify the distinction between rand and sample.
StefanKarpinski
All 27 comments
Looking at the docs, I don't think I see a case where the functions couldn't be combined.
JeffBezanson
on 1 Mar 2014
Among others, there's this:
julia> using StatsBase
julia> rand(1:3)
1
julia> rand([1:3])
ERROR: no method rand(Array{Int64,1})
julia> sample(1:3)
3
julia> sample([1:3])
3
sample(a) where a is an array could just be a method of rand and since we already support rand(3:100) and a range is just an efficiently stored vector, there is precedent for this.
StefanKarpinski
on 1 Mar 2014
Ah, I misread your comment as "could" rather than "couldn't". So you're in agreement :-)
StefanKarpinski
on 1 Mar 2014
I wonder if the idea of a weighted collection shouldn't be generalized. There's a lot of overlap with multisets and dicts where the values are weights. The features that don't appear in Base's rand function are weights, replacement and ordering. It would be nice if we could arrange things so that there doesn't need to be a separate function just to add those features. We can either backports support for those things into Base or figure out a way to layer support onto the base rand functions in a package.
StefanKarpinski
on 1 Mar 2014
cc: @lindahua
andreasnoack
on 1 Mar 2014
I am all for migrating the sample function to base, and merging it with rand. The use of a separate function name in StatsBase is just a stop gap.
cc: @johnmyleswhite
lindahua
on 1 Mar 2014
I wholeheartedly agree. I would love to see sample in Base.
johnmyleswhite
on 1 Mar 2014
+1000000 for sample in Base.
porterjamesj
on 4 Mar 2014
Well, it's more the other direction we're proposing: using rand in stats to sample things.
StefanKarpinski
on 4 Mar 2014
I thought the idea was to merge the names rand and sample.
johnmyleswhite
on 4 Mar 2014
Yes, that's the idea. It remains to be seen which name will be left at the end.
StefanKarpinski
on 4 Mar 2014
Presumably you're going to keep randn, right? That seems sufficient reason to keep rand over sample?
johnmyleswhite
on 4 Mar 2014
I was thinking about that. Samplen is pretty odd.
jiahao
on 4 Mar 2014
Agree to use rand in the place of sample.
lindahua
on 4 Mar 2014
Ok, so the naming issue is decided. It remains to determine what needs to be done to merge them.
StefanKarpinski
on 4 Mar 2014
If you may wait a little bit, I can make a PR for this after the deadline in this weekend. I am also fine if someone else can take a lead to do that.
lindahua
on 4 Mar 2014
No hurry on this. It's not going into 0.3.
StefanKarpinski
on 4 Mar 2014
I have now get to the point to tackle this issue. I am proposing the API below:
The main idea is to reuse rand for sampling from population:
# randomly draw one sample from a
rand(a::AbstractArray)
# randomly draw multiple samples from a, with or without replacement
# this subsumes the rand(a::Range, n) methods
rand(a::AbstractArray, n::Int; replace::Bool=true)
rand(a::AbstractArray, dim::Dims; replace::Bool=true)
The rand function will be extended in StatsBase to accept a weight-vector for weighted sampling.
With this implemented, the sample function in StatsBase will then be deprecated in favor of rand.
lindahua
on 20 Jun 2014
Sounds good. There's a slight potential for confusion with rand((m,n)), which constructs a random array instead of sampling from the tuple. But this doesn't seem like a huge problem.
JeffBezanson
on 20 Jun 2014
@lindahua Is this worth revisiting now in the 0.4 cycle?
ViralBShah
on 22 Nov 2014
I may look into this again next week.
lindahua
on 22 Nov 2014
Did not make it into 0.4, removing from milestone.
tkelman
on 15 Sep 2015
Someone needs to own this to make it happen or it's going to slip again.
StefanKarpinski
on 10 Nov 2016
It's also a bit unclear what still needs to be done at this point.
StefanKarpinski
on 10 Nov 2016
The main difference now is that sample provides keyword arguments for replacement and ordering, e.g.
sample(1:10,8,replace=false)
The only reason we can't combine them is that we can't add more keyword arguments to existing method signatures without getting a method overwritten warning.
simonbyrne
on 10 Nov 2016
and the problem @JeffBezanson mentioned above:
julia> rand([1,2])
1
julia> rand((1,2))
1脳2 Array{Float64,2}:
0.966145 0.590825
(sample throws a MethodError on the tuple one)
simonbyrne
on 10 Nov 2016
After some discussion, the decision was made to leave it as is. rand and sample are indeed subtly different:
randsamples independently (either uniform [0,1], uniform from a discrete set, or from a specific distribution if used withDistributions),sampleallows for non-independent methods (e.g. without replacement), which are only applicable to finite sets.
simonbyrne
on 10 Nov 2016
Related issues
tkoolen
路
3Comments
felixrehren
路
3Comments
wilburtownsend
路
3Comments
omus
路
3Comments
i-apellaniz
路
3Comments
Most helpful comment
After some discussion, the decision was made to leave it as is.
randandsampleare indeed subtly different:randsamples independently (either uniform [0,1], uniform from a discrete set, or from a specific distribution if used withDistributions),sampleallows for non-independent methods (e.g. without replacement), which are only applicable to finite sets.