Tmap: cartogram

[joelgombin]

I'm noo much into js, but I wonder if using http://cran.r-project.org/web/packages/V8/vignettes/v8_intro.html and http://prag.ma/code/d3-cartogram/cartogram.js it would be possible to generate a cartogram? Otherwise I guess we could just generate an html page template?

[mtennekes]

I'd say, let's give it a try!

[mtennekes]

Ideally, we should be looking for a function such as tm_cartogram("X"), with X the aesthetic variable that determines the sizes of the polygons. Alternatively, we could add an argument to tm_shape, e.g. tm_shape(my_spdf, cartogram="X") + tm_borders().

A necessary step is a function like this:

my_distorted_spdf <- create_cartogram(my_spdf, size="X")

If we try the js way, probably topojson is required, isn't it? If we can't avoid this, then we could request the user to install it.

[joelgombin]

I guess so. Seems like an hassle though, so I'm not sure how well it fits the purposes of the package...

[mtennekes]

I still would love to have cartograms in tmap... So can I allure anyone of you (@joelgombin, @edwindj, @timelyportfolio) to make this work, albeit a prototype?

[joelgombin]

News on this front: there now is an R package allowing to create cartograms : https://github.com/chrisbrunsdon/getcartr

However it is not on CRAN and I think it won't be for a long time, since the dependencies are not easy to deal with (see https://github.com/omegahat/Rcartogram).
Also I believe it's much slower than the topojson version, even though one nice thing is that it's smart enough to be able to distort several layers in the same way.

So I guess this is not really a solution, since we can't easily get people to install it. Also it's very easy to use outside of tmap framework, so not sure there'd be real added value for someone who already installed it. What do you think, @mtennekes?

[sjewo]

Maybe one could reimplent the cartogram code in Rcpp with Armadillo, so the dependencys difficulties and unclear licence won't be a problem. I'll hope to find some time to take a look into the math and find out if the fftw library is really needed.

[joelgombin]

That'd be completely awesome (but unfortunately I can't help, I don't do
C++)!

On Thu, Apr 14, 2016 at 8:30 AM, Sebastian Jeworutzki <
[email protected]> wrote:

Maybe one could reimplent the cartogram code in Rcpp with Armadillo, so
the dependencys difficulties and unclear lizenze won't be a problem. I'll
hope to find some time to take a look into the math and find out if the
fftw library is really needed.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/mtennekes/tmap/issues/10#issuecomment-209780268

Joël Gombin

Doctorant en science politique / PhD candidate in political science
CURAPP - Université de Picardie Jules Verne

Parc Bella Vista, bâtiment D
11, allée Sacoman
13016 Marseille

Tel : [phone]
www.joelgombin.fr

[mtennekes]

Just want to keep this thread alive after seeing this nice overview http://www.cs.arizona.edu/~kobourov/star.pdf at EuroVis 2016. It seems like the diffusion algorithm (which is the most popular, since used by cart (and therefore also Rcartogram) and ScapeToad) is much slower than the algorithm from 1985 used by http://prag.ma/code/d3-cartogram/.

There are some new algorithms out there (e.g. http://sunsp.net/download/Reprints/SUN_2013_IJGIS_Carto3F.pdf) that are waiting to be implemented (since Carto3F is Windows only GUI tool, see http://sunsp.net/portfolio.html).

[timelyportfolio]

@mtennekes, thanks for the link to the paper. That is very helpful. I thought d3-cartogram was going to be revived https://github.com/shawnbot/topogram/pull/26, but looks like it is fizzling again.

[sjewo]

I found finally some time some time to start coding on this problem and wrote some code for an R implementation of the cartogram algorithms by Dougenik, Chrisman, Niemeyer (1985). This algorithm is also used by D3.js and the QGIS cartogram plugin (https://github.com/carsonfarmer/cartogram).

The code is actually in this gist:
https://gist.github.com/sjewo/53cd4c47edc4c6803bedb98a0902ae8e

The speed is alright for not too complex shapes. I have not decided about porting to Rcpp, because the most time consuming function are imported from sp and rgeos package.

The function returns a sp object and can be used standalone. One option is to publish a cartogram package, import it to tmap and add an helper function for cartogram aesthetics.

[mtennekes]

Awesome!!! The speed is good; not as fast as cartogram.js (which seems almost real-time), but still as fast or even faster than Scapetoad.

It also worked for the world shape (well, without Antarctica) with 30 iterations:

data(World)
wo <- gSimplify(World, 10000)
wo <- SpatialPolygonsDataFrame(wo, World@data, match.ID=F)
wo$pop_est[is.na(wo$pop_est)] <- quantile(wo$pop_est, probs=0.25, na.rm=T)
wo <- wo[!wo$continent %in% c("Antarctica"),]
wo.carto <- cartogram(wo, "pop_est", 30)
qtm(wo.carto)

However, when I increased the number of iterations I get an error after iteration 54:
rgeos_PolyCreateComment: orphaned hole, cannot find containing polygon for hole at index 4
Did you take polygon holes into account in this algorithm?

How would you like to publish it? As you wrote, you (or we) can publish a stand-alone cartogram package. Another option would be to just include it in tmap. The first seems the most transparent. However, if it's just this single function without additional imports, we could just add it to tmap.

[sjewo]

Thanks for testing!

The sp-polygon objects are recreated after every iteration. One polygon was marked as hole (maybe this was guessed by the ring direction), but was moved outside the surrounding polygon. I now use checkPolygonsHoles from maptools to fix the hole value (update in gist).

That's not a really good solution, because the hole will now act as country border. Maybe it's better to delete this type of polygons.

I tend to publish a seperate package, because its useful on its own and I hope to add additional algorithms. I'll make an upload to CRAN in the next days.

Maybe I should also add a tm_cartogram function?

[joelgombin]

Wow, that's great news, many thanks! I haven't tested it, but an obvious way to speed things up (although it introduces new dependencies) would be to parallelize the for-loops, wouldn't it? (for one approach for parallelising nested loops, see https://cran.r-project.org/web/packages/foreach/vignettes/nested.pdf)

[timelyportfolio]

Looks like the week for cartography with new progress on https://github.com/shawnbot/topogram/pull/25#issuecomment-243291262. Thanks everybody.

[sjewo]

I tested a parallel version, but it introduced some overhead and there is only a speed improvement for shapes with many features (maybe 500 and more).

Neverless I plan to make this an user option.

[mdsumner]

I may be able to improve some sp object handling with spbabel. Decomposition to tables makes it easier to filter and modify sp stuff, and spbabel::sp() can recompose - so long as the mods are sane. Also r-gris::rangl shows some more interesting transforms, though in devel branch

[mtennekes]

We can indeed add a tm_cartogram. Maybe something like this?

tm_shape(Europe) + 
tm_cartogram("pop_est", itermax=10) + 
tm_polygons("HPI")

tm_cartogram is not really a layer function, but a shaping function. However, ideally, the order should matter. For instance:

tm_shape(Europe) + 
tm_fill("HPI") +
tm_cartogram("pop_est", itermax=10) + 
tm_borders()

Process-wise it's for more easier to apply the polygon distortion right after tm_shape. We can tackle this by interpreting this code as:

tm_shape(Europe) + 
tm_fill("HPI") +
tm_shape(Europe) + 
tm_cartogram("pop_est", itermax=10) + 
tm_borders()

[mdsumner]

I mention this since it seems there could be a more general transformation engine while maintaining topology, and cartogram a particular family of those. May not be a helpful suggestion, but one I hope to have a look at!

[sjewo]

@mdsumner spbabel looks really interesting! But I don't think the cartogram algorithm will benefit from it, because it needs some area and centroid calculations, which are easier in rgeos.

I put up an repo with the code for the r-package: https://github.com/sjewo/cartogram

There are actually some limitations in respect to the weighting variables and a big number of features (data outlier, e.g. features with very low values will raise the mean error and the algorithm needs a lot of iterations or doesn't converge at all) .

[mtennekes]

Great! Also a pleasant surprise that cartogram isn't occupied yet:)

The limitations call for some automatic and pragmatic data cleaning methods. For instance:

• Remove features with values lower than some threshold, which is save to do, since you don't expect them to appear in the cartogram anyway.
• More general: calculate the original area / distorted area ratio. If this ratio is very low or very high for some features, adjust the corresponding data values. One option is to use cut-off values. Another option would be to adjust all data values, so that the area sizes are a little less distorted.

Maybe good to issue this on your github repo page.

[sjewo]

Sounds like a good idea! It's on my todo list for the second release.

What would be the best starting point for the tm_cartogram function? Or would it be easier to prepare the shape before plotting, similar to smooth_map.

[mtennekes]

Good question, also from a conceptual point of view. Maybe we should go for both options.
However, a stand-alone function would be pretty identical to your cartogram function, isn't it?

I can implement tm_cartogram (the variant which I described above), since the internal processing could be quite hard to figure out, except for me;-).

[timelyportfolio]

Have a look at this very recent work on hexagon cartograms https://pitchinteractiveinc.github.io/hexagon-cartograms/ with github repo https://github.com/PitchInteractiveInc/hexagon-cartograms. I'll watch this very closely to see how we might tie into this discussion.

[timelyportfolio]

also discovered this R package also called "cartogram" https://github.com/chxy/cartogram

[mtennekes]

@timelyportfolio Thanks for the urls. I really like the hexagon maps. About the cartogram package you mentioned: it is a front end for the old Rcartogram package, which is hard to install.

@sjewo I think a tm_cartogram function as part of the plot call is not very handy, since cartograms are not created instantly. I propose to go with your base cartogram function, which does the job nicely, and re-export it into tmap. In that sense, it is similar to functions like smooth_map. They are functions to create/modify shape files, which might take a couple of seconds, so users will have to call them prior to the plotting call in order to set the parameters properly.

[sjewo]

@mtennekes I think you're right. Should the cartogram function give any additional output/attributes for use with tmap?

[mtennekes]

I cannot think of any.