Octoprint: [Brainstorming] Add advanced timelapse functionality; add plugin capability

[youngiif]

Below is a dump of my thoughts/plans on extending the timelapse functionality.

I know right now the plugin system doesn't support extensibility here, so I'm wondering if that should be added or if it would be preferred to add this to the core code. I believe a large number of users would use these features so I could see it in core, but if it's not too much trouble to add the plugin capability that would be nice for the long term.

1. Full specification of the ffmpeg command
   
   
   
   • I would expect if a user does this they are OK with ignoring the filter string created for flip/rotate/watermark; a warning would be given
   
   
   • Would be good to show user the dynamically built ffmpeg command for comparison and/or tweaking
   
   
   • Should expose a Test button to test the ffmpeg command with error reporting
   
   
   • This would require some test folder of images?
   
   
2. User specified post-create commands
   
   
   
   • This would pass the filepath of the created timelapse to any user command
   
   
   • For example, could upload timelapse to YouTube, move to network share, turn off printer
   
   

[gordyf]

I was about to open my own timelapse brainstorming issue when I saw this one, so forgive me for piling on, but I would like to be able to specify a target timelapse video length and have Octoprint calculate the framerate required to hit that video length. Perhaps there can also be a minimum framerate to avoid short prints turning into a slide show.

If the timelapse functionality were more easily extendable, I could write this as a plugin or extension.

[youngiif]

@gordyf No problem. There are other timelapse issues/brainstorms, but I tried to keep mine in an area that it would make sense or be possible to include in the core functionality if it might be a while (version 2.0 for example) before plugin extendability is added for timelapse.

Your idea is a good one and could likely be handled with my item #1 above since ffmpeg takes a duration parameter. I think you would be able to just specify the length you want and it would adjust framerate automatically.

[gordyf]

Yes, you're right, that would be great!

[ntoff]

Full specification of the ffmpeg command

Do you mean the full list of variables that octoprint uses to invoke the ffmpeg command? Because ffmpeg's own internal commands are pretty well documented all over the internet.

There was a script a while ago to use a more efficient GPU encoder https://github.com/guysoft/OctoPi/issues/184 would be nice if that became the default way under octopi (though that's a discussion for the octopi tracker).

[youngiif]

@ntoff , yes I'm referring to the full set of commands available to ffmpeg, just as if you were going to use it stand-alone from the command line or your own application. You are right the ffmpeg options are very well documented with a lot of examples for many different scenarios, but there would need to be some macros/variables available by octoprint that would be parsed to include such things as the filename pattern.

[ntoff]

I'm referring to the full set of commands available to ffmpeg, just as if you were going to use it stand-alone from the command line or your own application

Then go read one of the many ffmpeg documents from around the internet, the version of ffmpeg octoprint uses isn't a custom version. Just read the regular ffmpeg documentation on how to create a movie out of a directory of images.

https://trac.ffmpeg.org/wiki/Slideshow

unless I'm misunderstanding what it is you're asking for, all octoprint is doing is using the command line options for ffmpeg, using the current gcode name as a template, and spitting out a video with the same name.

[foosel]

@youngiif @ntoff I think you two are misunderstanding each other ;) I think @ntoff understood you to wonder about what parameters the ffmpeg command supports and to request a documentation or something like that built in. But I think instead you meant to just basically put a text box somewhere with the full ffmpeg command line that will be used to generate the movie, with some placeholders, allowing the (pro) user to adjust and tweak it.

I agree that that (or something like it) would be handy. I've thought about this too in the past but it's a bit tricky since it's not just one possible command line but quite a number of them (various flag combinations with regards to watermark, rotation, flipping). So far my personal brainstorming always ended at the question of how to do a proper UI for something like that :/

The second point is somewhat already possible since there's an event fired after movie creation that contains all that data and could be used for any post processing. In fact, there's an example in the event handler cookbook on the wiki on how to do an automatic youtube upload.

[youngiif]

@foosel You are exactly right on the misunderstanding.

I think I handled the tricky part you mentioned in my original post, the easiest way possible :rofl:

I would expect if a user does this they are OK with ignoring the filter string created for flip/rotate/watermark; a warning would be given

I saw the callback on movie creation, but I had not seen the events documentation to really understand how to use that. So that definitely answers part 2 of my original post.

On the ffmpeg command, what if we take a step back and go about this a different route (instead of putting a textbox in UI for users to modify.)

What if there was an event added for, 'before_timelapse_render_start' or 'on_timelapse_command_created', or something similar, that fires after octoprint has completely built the ffmpeg command (watermark/flipping filters applied and all) and right before it is executed. Then a plugin could take that ffmpeg command and either completely replace it or tweak it as desired.
I'm not yet familiar enough with plugins to know if the event handler payload can be modified or if it's a one way street, but either way I think this could work. If event handler payload is read-only then perhaps octoprint could save the ffmpeg command to disk, pass that filename to the event handler, and then plugins could modify the ffmpeg command on disk. Finally octoprint would reload the file from disk and execute the command.

Does this sound like a good/bad idea?

[ntoff]

I see, sorry, my bad.

What about a simple tickbox "don't render movie" and just archive the images? Users could download said archive, add print info overlays, fancy music, adjust brightness and contrast, etc all in their favourite video editor.

Or it could also serve the function of delaying the video renders so you don't have to wait for a render to complete before starting a new print, then once all the prints are done, you could go back and render the videos all in one go.

[youngiif]

@ntoff, that's a good idea.

I think with the event I mentioned above you could also handle this. Basically you could blank out the ffmpeg command in a plug-in so it is not called and move/copy the images to a location of your choice if you want to process later. Or you could even run your own program/command instead of ffmpeg at this time. In this instance you would, blank out the ffmpeg command returned to octoprint so it doesn't run it.

This approach might also keep a user from accidentally clicking don't render timelapse and filling their sd-card/hard drive up.

[ntoff]

I think it could tie in with the feature request posted somewhere on here about job queues. You'd queue up all the prints, then add a timelapse render job to the queue as the very last job. Adding a print could automatically add that timelapse render job, but adding subsequent print jobs would add them as the "last" job, but before the render job. That way a user would have to manually remove the render job.

That way it'd capture all the images, move them to a safe location (or leave them where they are since I don't think they're overwritten now) and just keep on printing until that render job comes up, at which point it would render all the movies, or archive the images for later download unless the user manually removed the render job.

[youngiif]

@ntoff I can see the benefit of a job queue, and a single render for all the prints in a job set, if I'm understanding you correctly. (Especially if you implement this as well https://youtu.be/Viy928RLsdU)

I think if there is enough thought up front, then there can be enough information included in the proposed timelapse events and/or settings to support just what you are mentioning.

This may deviate slightly from your proposal as it would rely on a plug-in for the render job at the end of a job queue (I'm assuming you were expecting it to be built into the job queue), but I think this would give ultimate flexibility since you could actually do what you want without a job queue.

Basically I can see a plug-in allowing you to capture the images without rendering a video, as you suggest, but then later be able to see a list of all image sets generated and pick the ones you want to combine into a single render. It would be a bit more manual than being automatically added at the end of a job-queue, but without a job-queue existing, this should suffice I hope. Then when/if a job-queue feature is added, the plug-in could communicate with and remove that manual step for you.

[ntoff]

I wasn't necessarily making a proposal, just throwing out random ideas.

I say just turn the timelapse functionality into its own plugin, let end users run wild with it. I guess we currently already can though, right? I mean you could leave ffmpeg's command line blank and octoprint would still send out the camera feed, it just wouldn't capture images. I'm betting a plugin could already be written under the current system that takes over the timelapse functions without modifying octoprint at all, right?

[foosel]

I'm betting a plugin could already be written under the current system that takes over the timelapse functions without modifying octoprint at all, right?

Sadly no, at least not without some heavy monkey patching (that I would advise against, at least long term). The timelapse stuff is still pretty hard coded and also one of the oldest bits I think. Not as trivial to hook into. But considering that a lot of people are looking for more flexibility there in general, tackling that in general would be a good idea.

I like the idea of uncoupling the rendering from the capturing mechanism since that would already allow for a lot of stuff to be overridden by plugins.

(Warning, a bit of brainstorming on my part follows ;))

Another thing I have in mind (although not really fitting this particular train of thought here) is making the timelapse types pluggable. That will require some mechanism though to tell the frontend what kind of parameters are needed to be configured by the user, so that the timelapse configuration page can be generated dynamically (e.g. timed timelapse needs the interval, z-change needs the z-hop, something triggered by some specific GCODE sent to the printer might need that GCODE, something with a fixed target length would need said length, all timelapses in general might need stuff like post-roll, frame rate, etc).

In general, looking at what kind of functionality has been requested for in the past for timelapses fall into these categories:

• Overlays with e.g. temperatures, print progress and such (huge favourite among my patrons it seems)
• Additional modes (e.g. different triggers for the snapshots, automatic snapshot interval calculation based on target size and such)
• Additional formats and/or post processing (e.g. mp4 instead of mpeg2, youtube etc)

The first two points would be part of the capture phase (and also the configuration). The latter I'd see as the render step, but with input provided by the capture phase.

I could imagine something like

• creating a configuration parameter system for the timelapse parameters itself with which to model what is needed and also which to use for UI representation
• providing a hook to plugins to register additional timelapse types
• defining some capture type that will be called during timelapse capturing, taking care of the snapshots but also any overlays on top (maybe already as image? if so we'd need to pull in pillow as dependency, no problem though) and allowing to compose the timelapse output from these base image and overlay layers
• defining some render type that takes a bunch of layered images as input plus some configuration as well (accepted parameters can be specified to be mirrored in UI as well, see above) and allows chaining - stock mpeg2 timelapse render process gets moved there, additional variants (mp4, youtube upload, gif, ...) can then be added via an additional plugin hook
• probably also adjusting the UI heavily, because all that new found freedom most likely requires restructuring thing quite a bit to keep stuff managable (e.g. for defining overlays and such)

A first approach could stick to porting the existing functionality in such a structure to allow later expansion with overlays, plugin provided variants etc.

Now, that's gotten a bit long winded and possibly confuse. Sorry for that 😊

[bagobones]

Personally I would like to have an overlay indicating the REAL print time Hours:Min during the video and at the end a summary of things octoprint knows like the final print time, how much filament was used, temp, average speed etc.

[JohnOCFII]

Personally I would like to have an overlay indicating the REAL print time Hours:Min during the video and at the end a summary of things octoprint knows like the final print time, how much filament was used, temp, average speed etc.

@bagobones - Some of your requests are handled by the History Plug-in: http://plugins.octoprint.org/plugins/printhistory/

[bagobones]

@JohnOCFII I am specifically saying it would be cool if it could be inserted into the end of the video..

I know I can track the info through various other ways externally

[JohnOCFII]

@JohnOCFII I am specifically saying it would be cool if it could be inserted into the end of the video..

I know I can track the info through various other ways externally

Oh! I get it now. Yes, I agree, that would be cool.

[FormerLurker]

I'm working on a plug-in that does a lot of this (fixed length video, mp4), etc, but you all are giving me ideas in other areas. I wish I had found this issue months ago.

I used to have a file template option, but it seemed like a lot of trouble. I do like the idea of tacking info (print time/failed) to the end of the filename. Maybe I should revisit my template idea.

What about custom scripts for rendering? That could be a catch all for odd cases.

[ntoff]

@FormerLurker what about just having a filename template field like OBS-studio has? You can give it %variablenames% that equate to dates. If you gathered info about print time or something, and let users enter it as a variable, and another field for direct ffmpeg command line control, I'm sure that'd probably satisfy 90% of the requests for more rendering control.

variables like:

• start time
• end time
• pass/fail (whether the print was cancelled, or printer errored out)
• printer profile name
• user who started the print (seen people ask for this)

I'd be worried about the file name length becoming too long though, I don't know about linux but as OctoPrint aims to be platform agnostic, I do know windows has limits on file path lengths (ridiculously long but you'd still want to account for it so a file won't error out if it's too long, but would truncate itself).

[FormerLurker]

Not just that, but validations are difficult, especially on a web client.
My templates worked pretty much as you described. I still use them in the
back end, just no ui for it :(

On Thu, Mar 8, 2018, 8:55 PM ntoff [e-mail] wrote:

@FormerLurker https://github.com/formerlurker what about just having a
filename template field like OBS-studio has? You can give it
%variablenames% that equate to dates. If you gathered info about print time
or something, and let users enter it as a variable, and another field for
direct ffmpeg command line control, I'm sure that'd probably satisfy 90% of
the requests for more rendering control.

variables like:

• start time
• end time
• pass/fail (whether the print was cancelled, or printer errored out)
• printer profile name
• user who started the print (seen people ask for this)

I'd be worried about the file name length becoming too long though, I
don't know about linux but as OctoPrint aims to be platform agnostic, I do
know windows has limits on file path lengths (ridiculously long but you'd
still want to account for it so a file won't error out if it's too long,
but would truncate itself).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/foosel/OctoPrint/issues/1980#issuecomment-371697618,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Af0UuCE2PFguwV6uqldTPl-WMiRnoY_8ks5tce8-gaJpZM4OCyQU
.

[FormerLurker]

I added one of the things requested here to my plugin, which is now in the repository.

Output Filename Template - This includes the failed flag that was mentioned here. Thanks for mentioning that, it was worth the effort. The failed flag is awesome. I believe the validations are working, and there is a fail-safe in case the template won't parse.

I'm considering allowing a custom rendering string or bash/bat script in a future release. I've created an issue here for further discussion so as to not pollute this thread too much with non-OctoPrint talk.