Deeplearning4j: 3D/Volumetric Convolutional Neural Networks support

[archenroot]

Issue Description

Is there any kind of support for 3D/Volumetric Convolutional Neural Networks in deeplearning4j? If yes, any example available?

Something like:
https://github.com/dimatura/voxnet

[AlexDBlack]

No 3d CNN support at present. We don't currently have 3d CNN layers, plus we would also need to add some supporting functionality (such as preprocessors to handle 3d CNN -> dense, etc; plus dataset merging, etc) for this case.
It's not on the immediate roadmap for us at present.

[agibsonccc]

https://github.com/deeplearning4j/deeplearning4j/issues/3053 Flagging this as well.

[turambar]

Overlaps https://github.com/deeplearning4j/deeplearning4j/issues/2447. Should we consolidate and/or close one of these?

[agibsonccc]

Close it in favor of this one. Reason being: we have to implement 3d convolutions first. We can do keras after that.

[jadedfire]

looking at the code, it appears that assumptions concerning multidimensional shape were made all the way down into ND4J as i find "row" and "column" all over the place (i.e. always assume 2D). i suspect 3D support will require a major overhaul if my limited research is accurate.

[agibsonccc]

@jadedfire could you clarify? Nd4j itself is a tensor library we have ndarrays everywhere (you don't have to look far - even our current cnn implementation is 4d o_0) Don't judge a library by looking at it for 5 minutes. We have rows/columns for specialized calls. Our docs even make it obvious we support multiple dimensions. Your research was horribly off.

[raver119]

Fortunately your research isn't accurate. Nd4j supports up to 32 dimensions :)

[jadedfire]

my apologies, i will dig deeper
[update] i get it now, 2D is common so there are specializations. i see now where shape is used

[archenroot]

I am from Czech republic and there is widely used search engine is www.seznam.cz, which provides as well now 3D maps view generated by engine which is developed by https://www.melown.com. In general this engine is open source. I know it is not Java, but in general this could be component which will take any image and convert it into 3d model... I am not sure about all the capabilities in the moment, but maybe worth to investigate available mechanisms and possible compatibility bridge with deeplearning4j.

I think they will be more than interested in cooperation with neural network engine if they can see some possible business use cases in future. Even now, they are still experimental startup company producing money some money because of the only account (www.seznam.cz)

https://www.melown.com
https://github.com/Melown

[agibsonccc]

@archenroot We'll implement it when we get time. We usually don't expect external contributions for layers. The barrier is a bit high for 99.9% of folks out there ;/

[archenroot]

@agibsonccc I see, I just red about them and see that they produce engine for transforming images into 3D models, so just got idea about some integration, etc. But in general I understand that it will be more consistent to have it fully integrated inside deeplearning stack rather than reusing some "unknown" technology.

[archenroot]

Maybe this could be possible project to adopt: https://github.com/chrischoy/3D-R2N2/

[archenroot]

@agibsonccc may I ask you if you can provide any estimation of this feature request to be implemented? even gross estimation is ok, thx.

[agibsonccc]

No. If skymind doesnt need it, its not being built. We only build things
for customers.

On May 15, 2017 22:58, "archenroot" [e-mail] wrote:

@agibsonccc https://github.com/agibsonccc may I ask you if you can
provide any estimation of this feature request to be implemented? even
gross estimation is ok, thx.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/deeplearning4j/deeplearning4j/issues/3174#issuecomment-301482900,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABF18qBfGGWpZ1LUEodF0tRVW1-Bkzc_ks5r6FndgaJpZM4Mwge3
.

[archenroot]

@agibsonccc I see and understand, is there any docs, or example to follow how I can try to adapt/implement it by myself?

[turambar]

Reminder: we have to implement this in DL4J before we can import it from Keras so perhaps not @maxpumperla's purview?

[raver119]

Implemented in libnd4j quite a while ago.

[maxpumperla]

@raver119 there's an open PR for this here #4927. let's not just close things like this.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.