Octoprint: [Request] Non-MJPEG cam support

[Thulinma]

It looks like OctoPrint takes whatever webcam URL you insert, and embeds it inside an IMG HTML tag.
That is all perfectly fine and reasonable, assuming your webcam is MJPEG-based.
However, my setup actually makes the camera feed available as a high-quality low-bitrate progressive MP4 feed (using the RPi hardware encoder to encode to H.264), at "/cam.mp4". This type of link would have to be embedded inside a VIDEO HTML tag, instead.

Even more flexible would be to have a text field where you can simply enter the HTML/JS code to embed your video feed, which would work for literally any type of camera in existence (including funky plugin-requiring ones and the like), and allow for users adding their own custom controls and such as well.

PS: Love the software, regardless of any response to this request!

[ntoff]

However, my setup actually makes the camera feed available as a high-quality low-bitrate progressive MP4 feed

Got any info on how to set that up? My wifi is pretty terrible, lowering the bandwidth on the cam stream would be great (and more details on how it works might help someone fulfil your request if they can replicate your setup).

[Thulinma]

The method I used isn't currently publicly available (though it should become so, soon!).

It's part of a private development branch of the MistServer project ( https://github.com/DDVTECH/mistserver/ ), which supports receiving raw H264 on standard input and then transmuxes that to various consumer-grade protocols (such as progressive MP4, HLS, RTSP, RTMP, etc) for easy watching. Specifically the H264 raw input is still private, but will soon (timeline ~2-3 weeks, most likely) make it into the open source edition of the software.
With that installed, it's a matter of running something along the lines of this as an auto-restarting system service: raspivid -t 0 -pf high -lev 4.2 -g 10 -ih -qp 35 -o - | MistInH264 -s cam

As an example, the stream of my printer can be accessed here: http://patchy.ddvtech.com:8080/cam.html (autodetects best playback method, might go down anytime, or not be interesting, but it's up and actually printing as of right now).
The direct MP4 link would be: http://patchy.ddvtech.com:8080/cam.mp4 (not all browsers always support the direct format, but Chrome does, for example).

Does that help? I can definitely leave a note when all this is publicly available as well.

[lf-]

Hm. How would you implement something like this to be as extensible as possible? Add a field for a custom template for the webcam viewer? Extensions for the different formats?

[Kunsi]

Easiest way would be a selectbox ["MJPEG-Streamer", "Youtube", "Stream", "custom"], with corresponding input fields, all creating a matching HTML string, except for "custom", which would allow you to enter any kind of HTML code you'd want.

[Thulinma]

That, yes. To go for a perfectly customizable implementation, a "Custom JS" option could be added, with two fields for JavaScript code blobs: one that is ran when the video should start displaying, and another that is ran when it should stop. Many custom players require some unloading code to make sure browsers don't keep the connection open in the background, so the stop field is quite useful for those cases.

For the brave: the latest code on the "development" branch of MistServer will now support streams being set up with a source of (for example - most of these params can be changed to suit your mood/wishes) h264-exec:/opt/vc/bin/raspivid -t 0 -pf high -lev 4.2 -g 10 -ih -qp 35 -o - which will automatically start/stop the stream depending on it being viewed or not, for optimum power usage. It requires no further external scripts or services.
For the less brave: we'll probably post a full tutorial on how to set this up in a month or two. I'll drop a note here when it's been posted.

[daktylus]

Hey, any news on the project??

I would love to see that video stream bandwidth cut down.

Currently 480x640 @ 10 fps needs 5Mbit/s , that is crazy and doesn't allow me to watch the stream properly from remote :-(

[ntoff]

If this turns out to be a giant help and works amazingly well, you might want to also post over in the octopi issue tracker to see if they're interested in it. OctoPrint is just the part that controls the printer and doesn't include mjpeg streamer by itself, OctoPi is the precompiled raspberry pi image https://github.com/guysoft/OctoPi/ I'm sure they'd love to have a pre-packaged low bandwidth streamer, although if it only supports the raspberry pi camera I don't know if they'd be willing to throw out mjpeg streamer just yet (since some people may be using cheap webcams).

[Thulinma]

There is news! The post on how to set this up was posted a bit ago: https://news.mistserver.org/news/69/Raw+H.264+from+Raspberry+Pi+camera+to+MistServer
The most recent release (v 2.12) of MistServer contains the functionality - even in the OS/Free edition. Do note that the OS edition does not have pure HTML5 output (the one mentioned in my first message), but the next version will likely include MKV/WebM format streaming in the OS edition as well for (some) native HTML5 support. Until then, the HLS/TS/RTMP/FLV outputs are all usable and at least HLS will work in most browsers and most mobile devices.

This is not necessarily RPi-only, as we've implemented a generic raw H264 input over stdin, but it does require a hardware encoder of some type to be available... so it doesn't apply to all use cases OctoPi handles. But it does work for most people, I think.

[gappleto97]

You could do this over VLC fairly easily too.

cvlc v4l2:///dev/video0 :v4l2-standard= :input-slave=alsa://hw:0,0 :live-caching=300 --sout '#tnscode{vcodec=VP80,vb=2000,scale=0,channels=2,samplerate=44100}:std{access=http{mime=video/webm},mux=webm,dst=:8080/}'

Only difference is that it doesn't allow for snapshots, though I think that could be done without too much difficulty.

[buzztiaan]

Maybe this issue could be helped with https://github.com/flok99/img2jpeg

[johnboiles]

I think last time I tried to use cvlc it tried to pull in tons of x11 dependencies. Not a huge issue but def a bummer for octoprint users who otherwise don’t need the cruft of a desktop environment

[gappleto97]

I'm not saying that it's necessarily ideal, but it is available right at this moment. If I recall, you can do a similar thing with ffmpeg.

[bgamari]

For what it's worth, I have hacked together a small Python hack which serves WebM or H264 produced via gstreamer over HTTP. It is also capable of serving still frames. The implementation isn't particularly robust (e.g. it requires some help identifying parameters which the video source can provide) but it works for me; improvements and refinements are certainly welcome.

One interesting possibility is to incorporate support for WebRTC using this rather recently-introduced GStreamer plugin.

[jneilliii]

You could tweak the ffmpeg settings within my OctoPrint-YouTubeLive plugin which uses docker to stream direct to YouTube Live. It's not as real-time because it has to re-encode the stream, but it would work. You would just need to edit the entry.sh file for the correct parameters for your source.

[drewcassidy]

+1 to this. most browsers support mpeg4 streams now so I dont see much point in using mjpeg, and I'd like to be easily able to access the webcam feed through other things (I'm currently trying to get it to also work over Homekit/homebridge and I'm having to do a lot of re-encoding)

[foosel]

I dont see much point in using mjpeg

Ooooh, that's fairly simple to explain: Native camera support even among the cheapest of them. Yes, you could just transcode the whole stream of any source format, be it mjpg, single jpgs or h264 and push that out as an mp4 http stream. But then that whole work has to be done by the system's CPU. Which - in case of OctoPrint's common runtime scenario - is severely limited. Transcoding some full-HD MP4 stream is simply a whole lot of computation that needs to be done while also trying to do everything else. So just taking the MJPG stream from the camera that it supports natively and piping that through 1:1 without having to think about it any way is a huge benefit.

[daktylus]

Some people use the Logitech C920 which offers a hardware encoded h.264 stream.

[foosel]

Some. And there lies the problem.

[foosel]

To add a bit to this: I'm not saying "I don't want MP4 support" here. The question above was why MJPG was chosen in the first place, and I outlined the reasons above. I'd be happy to offer to consume a stream via anything supported by the HTML5 <video> tag as another option next to the <img> based MJPG default present today. Getting around to that might take some time for me though. But if someone wants this now, feel free to send a PR against maintenance that allows switching the webcam stream to the video tag via the settings and adjusts the embedding accordingly. This is an open source project after all :)

Now, what @bgamari posted there looks really promising. I've been eyeing GStreamer for a while now, but so far it seemed to lack a well supported way to produce MP4 streams or WebRTC. If something can be built up around it however that would put an equally low resource impact on the system as mjpg-streamer does today, it would definitely be a valuable alternative and I'd even see that we can get it included in OctoPi then.

[Thulinma]

I'll see if I can find some time to make a PR for generic video tag support. 👍

[bgamari]

I should mention that with the proper hardware acceleration video encoding can be quite cheap. Streams which my two-core/four-thread Skylake laptop can't feasibly encode via software encoding can be encoded with ~5% CPU usage via VA-API using the Intel GMA accelerator hardware. This hardware is ubiquitous in modern amd64 and ARM platforms. Unfortunately, in the case of ARM platforms driver support tends to be extremely spotty.

[foosel]

2493 and #3020 have now also asked for iframe support instead of img. Adding this to this request.

So what we want here is an option to select between img for mjpg, video for anything that supports, and iframe for the cases where neither of the other two fits.

[jneilliii]

Here's a quick plugin for iframe support in an extremely beta version. Note, it will break keyboard control on mouseover of the camera, but I've been using OctoPrint for a real long time now and have never used that feature. Update the webcam stream url in OctoPrint's settings to the url you want to load in the iframe (ie for twitch https://player.twitch.tv/?channel=jneilliii&muted=true) and then install the plugin using the url below in plugin manager.

https://github.com/jneilliii/OctoPrint-WebcamIframe/archive/master.zip

When the page reloads the default webcam_container will be replaced with an iframe pointing to the webcam stream url in OctoPrint settings.

[pazooki]

I tried @bgamari 's code for webm to use it with video tag https://github.com/bgamari/cam-stream after fixing some issues in that code but no result and no error on Raspbian. do you kknow a better way of streaming to web?

[Baenwort]

I tried @bgamari 's code for webm to use it with video tag https://github.com/bgamari/cam-stream after fixing some issues in that code but no result and no error on Raspbian. do you kknow a better way of streaming to web?

Are you trying to replace mjpeg or send the feed out to somewhere? Either way unless your going to offer to code this up you may find more luck asking for help at https://discourse.octoprint.org/

[yufanyufan]

I use android phone and ip webcam app with octoprint. The app starts a http web server, and use h264 streaming.
It works perfectly with octoprint.

So I think this feature is already supported.

[mikeperalta1]

You could do this over VLC fairly easily too.

cvlc v4l2:///dev/video0 :v4l2-standard= :input-slave=alsa://hw:0,0 :live-caching=300 --sout '#tnscode{vcodec=VP80,vb=2000,scale=0,channels=2,samplerate=44100}:std{access=http{mime=video/webm},mux=webm,dst=:8080/}'

Only difference is that it doesn't allow for snapshots, though I think that could be done without too much difficulty.

Just to add: You can easily have ffmpeg take a snapshot of any compatible stream. I just wrote a script to do this for my own custom webcam bouncy thing yesterday (Raspivid -> ncat -> Relay Server -> ncat -> cvlc).

Here's an example from my recent logs (outputs to stdout):

$ ffmpeg -i http://[redacted]:9935/ -vframes 1 -c:v mjpeg -f image2pipe -

From there you only have to wrap the command in a script that gets called by your webserver. On my apache, I'm calling it with php.