Hdbscan: Can I do Spatio-temporal clustering with HDBSCAN ?
Hi All,
I've a dataset with latitude, longitude and date-time.
Is there a way to do a spatio-temporal clustering that includes the 3 features?
Havresine is accepting only 2 features (lat lon in radians).
I can scale/normalize the 3 features and use Euclidian, but I believe that I will lose the impact of the physical distance between points.
Any recommendation?
Thanks
Stephane-Thales
>All comments
The hard bit here is to work out how distance in seconds relates to distance in radians. Those are fundamentally two totally different scales. Any density based clustering technique is going to require a consistent distance between your points. If you had few enough points you could compute the Haversine distance between points and the temporal distance between your points and fold those together via a scaling that you felt was an appropriate between radian and seconds. Then you could hand that to hdbscan as a pre-computed distance matrix. Just remember that your scaling will change your clustering results.
If you wanted to go deeper down the rabbit hole of combining data with different concepts of distance on different sets of it's features then I'd suggest starting off by reading The 2016 paper "Generalized Low Rank Models" by Udell, et all. Don't be intimidated by the size of the paper it's a very smooth read with lots of examples.
jc-healy
on 3 Jul 2019
Related issues
Phlya
路
15Comments
mickohara23
路
10Comments
architec997
路
14Comments
thomasht86
路
8Comments
arunmarathe
路
4Comments
Most helpful comment
The hard bit here is to work out how distance in seconds relates to distance in radians. Those are fundamentally two totally different scales. Any density based clustering technique is going to require a consistent distance between your points. If you had few enough points you could compute the Haversine distance between points and the temporal distance between your points and fold those together via a scaling that you felt was an appropriate between radian and seconds. Then you could hand that to hdbscan as a pre-computed distance matrix. Just remember that your scaling will change your clustering results.
If you wanted to go deeper down the rabbit hole of combining data with different concepts of distance on different sets of it's features then I'd suggest starting off by reading The 2016 paper "Generalized Low Rank Models" by Udell, et all. Don't be intimidated by the size of the paper it's a very smooth read with lots of examples.