Numba: Dict type in nopython mode

[seibert]

Hash tables are very useful, and would be nice in nopython mode.

[fabioz]

Just wanted to say that this would be really nice (I have some code right now which could be made considerably faster if there was some API I could use for caching to skip some computation -- I even tried creating an int->int hashtable for my use case and couldn't really find a good way to do it in nopython mode as it's not possible to create something as [[(int, int)]]) for the buckets).

[carlosvega]

This feature would be awesome.

[garyrob]

+1.

[synapticarbors]

While I would also love support for typed dicts in nopython mode, I threw together this proof-of-concept using khash (which pandas uses under-the-hood) and cffi to get a int->int hashmap that is callable in nopython mode:

https://github.com/synapticarbors/khash_numba

Maybe this will be useful to some people in the interim. Depending on the number of unique elements that are being mapped, it looks like it is often faster to use a numpy array (trading speed for increased memory consumption), but there are definitely cases where this solution looks to be better.

[garyrob]

Very nice interim measure! Thanks!

[tamasgal]

Is there any news on this? I have tons of lookup tables which I want to use in my jitted functions... This features would be huge!

[hameerabbasi]

We just ran into needing this for our custom-rolled DOK implementation over in pydata/sparse#126. It'd be a big help if it was added.

[ericmjl]

I would also like to register my vote for dictionary support in Numba.

My use case is as follows: I have been writing graph neural network models using numpy + autograd, and I would like to be able to see JIT compilation of my for-loop intensive code. (Long story cut short, I am looping over pairs of matrices to get around an inability to use block sparse matrices with autograd.)

The parameters that I optimize are stored in a dictionary, so that I can access them by name in each function layer of the neural network.

[tamasgal]

@ericmjl meanwhile you can try the Julia language (sorry for advertising) ;) It's JIT, python-like syntax and C-speed. Talks to Python without significant overhead.

[seibert]

Dictionary support is on our roadmap (just being reviewed now in #3617) for first half of 2019.

[ericmjl]

@seibert thank you for letting us know! :smile: 🎉

[caiofcm]

Cool, included in 0.43 as experimental feature :tada:, thank you!
https://numba.pydata.org/numba-doc/dev/reference/pysupported.html#dict

[stuartarchibald]

@sklam think you should have the honour of closing this :)

[sklam]

Closing this. We now have a typed dictionary since 0.43.

[ericmjl]

:+1: :smile: 😸 !

[Avik1993]

This is so cool. However, is there anyway I can get around nested dictionaries?

[stuartarchibald]

This is so cool. However, is there anyway I can get around nested dictionaries?

@Avik1993 Please could you clarify what you mean and open a new issue with an example of what you want to do but does not work at present, thanks.