Openpilot: Source code of visiond

[luciocorrea]

If I got it right, visiond binary is responsible for the vision part of the project, right?

So, the most important part of the project is not open source? Really?

[geohot]

At comma.ai, our goal is to be as open as possible without killing the company. For now, for business model purposes, we have decided to keep visiond closed. Just like how parts of Android are closed. This may change in the future. At least it's free.

[mslavescu]

I assume you are not interested in getting help from open source community in the visiond space.

The features shown in the Youtube video is what attracted me the most, and I hoped that I could modify and contribute also on those.

Too bad :-(

[geohot]

Hmm, if you mean what's on the screen, a lot of what you are looking for is probably in ui.c

[peterwilli]

Yeah this is quite a bummer but I'm not going to doubt the decisionmaking of someone with such a background. @geohot I've just got one question. This visiond, is it a compiled version of a machinelearning model? And what language is this binary file written in?

I'm asking because I'm deeply interested in AI since the past 2 years. I only ran my models on Python/Jupyter and Tensorflow/Keras.

[mslavescu]

I understand the decision, I don't see how this project can be extended in open source way, if a main component, the one that decides where the car should go is not open source.

@geohot I'm talking about the computer vision and machine learning algos embeded in visiond, I belive that is the brain of this project, the rest is just plumbing, which is nice also, but most people will probably not use it.

The UI is nice, but is not something very hard to do, for example with STEMCA Inventor a 5th grader could probably build a UI with similar functionality in a few hours.

[slivingston]

re https://github.com/commaai/openpilot/issues/4#issuecomment-264013103
Is it not the case that the license of openpilot also applies to the visiond blob?

If so, anyone is free to reverse it and modify it, redistribute the results, etc. Certainly that is not as useful as having the original source code, but this provides more freedom than some driver blobs.

[mslavescu]

@slivingston I don't think so.
If you don't have the source you cannot realy modifiy it, regardless of license.
The visiond is not the only thing offered only in binary form, NEOS the modified Android version is in binary form also, looks like they are too lazy to release the source for that :-)

[peterwilli]

@mslavescu it's much harder but it can be done (I've modified binaries myself as well back in my 'darker ages' / when I was a a kid xD)

[paramaggarwal]

@mslavescu I think a more fair criticism would be that one should know the API for visiond - so that other open-source implementations/alternatives could be built and plugged in. Given that everything outside visiond is open, the API by extension is available for anyone interested in working on it.

This is like Android OS, the core is open source, but shipped versions may include Google Play Services which is closed and can and has been substituted by others like Amazon for their own variants. This is the beauty of OSS - let's appreciate it.

[espes]

The API for visiond: broadcast ModelData packets at 20hz. That's pretty much it controls wise. See https://github.com/commaai/openpilot/blob/master/common/services.py#L46

You are welcome to replace visiond with an open source version.

[geohot]

The same license applies to visiond. We encourage binary mods :-)

[edvin1983]

Reproducing comma's visiond is trivial from deep learning point of view, use the existing system by driving around under different conditions and generate ground truth data from 20hz I/O already provided by ModelData. Than, train another neural network model offline or on the cloud with visiondground truthing reference from ModelData braodcast packet, at least this will get a new neural network very close to visiond behavior, than share the new neural network model for open source inference.

[mslavescu]

@edvin1983 I would love to use such an implementation (on Android, optimized for Snapdragon 820) if you get a chance to build one.

[NeoFromMatrix]

Slowly digging through openpilot a bit, curious how hard it is to get it running.

Also think it's kinda sad that openpilot is sold to the media as open source but visiond is not ?yet? open sourced.
I can understand that everyone is responsible for himself, and there are actually a couple of people out there which look on the code. (e.g. some found out that the CAN commands don't seem to be sanity checked at some point).

But what should we with that? Just hope that visiond works as expected without any chance to do a code analysis?

[paramaggarwal]

I don't think they pitched it as "open", they pitched it as, "hey, you can build your very own" - which is true.

[petterreinholdtsen]

Well, the name is 'openpilot', which emphasizes 'open' quite a bit. :)