Olive: Feature Request: extra, standard plugin APIs (frei0r, LADSPA, OpenFX, et al.)

[oc1024]

I understand that by design, Olive uses a completely different base from most FOSS video editors, and that it's more GPU than CPU demanding. However, for two reasons, I think this should be implemented

• Interoperability between video editors.
• Take advantage of the huge number of plugins and documentation that already exists for these platforms.
• Provide APIs that people might be more comfortable with than writing code specifically for the program.

The APIs I cited above are the two most common open-source multimedia plugin APIs, but this could be extended to other open APIs (LMMS Instrument, LV2) or even closed standards (VST - I think I found a reference for this in the effects source code).

[itsmattkc]

Yes, this is already planned. Olive supports VSTs already on Windows (see #14) and LV2 support is planned soon. I've been looking at frei0r too, primarily as an alternative for users with old GPUs (see #284) but also as you say to take advantage of existing work out there.

[itsmattkc]

Frei0r support added in d00ad46f268a87ca3bbc989b62a71bea85584d75

[oc1024]

I just download the latest AppImage and I noticed some weird behavior. Frei0r only works properly with .mp4 movies. PNGs become black images and most other formats trigger segfault.

[itsmattkc]

63bd1ac597715f09c9a19b2f34a3e517dc8a3b6f should fix this

[itsmattkc]

As stated in #14, VSTs are now available on Windows, Mac, and Linux.

[frink]

VSTs are good. LV2 is standard on Linux. frei0r is an old standard that never quite took root. LADSPA and DSSI are too old and not worth anyone's time. OpenFX seems to be an emerging standard that is worth supporting. GLSL is also another pretty good standard.

However, with plugin's you have to ask what can be done with them. Audio processing in a video editor is difficult with anything accept the most basic of projects. If you're doing anything with a great amount of foley work you will find quickly that the NLE is NOT the place for audio editing. This is the idea behind Davinci Resolve's Audio tab. But I think it may be a better use of resources to rely on something like Ardour to produce a refined soundtrack.

One thing that is not discussed much about audio is the need to produce multiple dialog languages for Blu-Ray or DVD release of truly professional works. If we are really trying to be a professional video editor then those audio needs should be considered along with the audio workflow.

In the end I think that you will need a different layout and approach for great audio editing. One of the downfalls of most NLEs out there is the sync with the audio workflow which can be quite different from the video. As deadlines get tighter and color grading, sound design and soundtracks are being produced simultaneously the need to wrangle all of these pieces in a flexible flow is ever increasing. My hope is to see Olive adopt more of the multi-person workflows which have always been ignored by open source video editors up to this point. (With the 1 known exception of Blender - Which does allow some limited multi-seat workflows...)

[unfa]

I am very excited to see LV2 support, as it's a great, modern format!
Inter-operability with audio-dedicated software like Ardour could be an interesting idea.

[frink]

@unfa Hey dude. Love your vids on Adrour and all things open source music. We share a lot of common goals to help make open source media production more accessible for the common musician - non-tech-geek. Would love to chat more directly on that topic sometime...

[frink]

One of the main things that video editing people need in sound is "normalized" audio output that sounds good for every clip from different sources so they don't have to jank with the sound. When editors say "normalize" they really mean automatic compressor/limiter without need to set ratios knees or makeup gain. Basically a one button compressor like the ones that are starting to trend in the DIYRE Colour format, but they want it in software.

They say, "It would be amazing to have sweetening button or two that magically makes everything sound better without effort." Most video directors don't know what can be done with the original audio to avoid requiring expensive ADR. We know the code that's out there and how good it sounds. Several LV2 plugins come to mind for this action.

There are two ways to do this. One is processing every sound source separately and allowing fine tuning audio stacks. (which will sound better but take more processor time) The other possibility would be a single audio chain on the master out. If we create favorite audio chains with sane defaults that can be turned on and off with the flick of a button but also allow us to dig deeper into the chain that may be the best of both world...

When exporting stuff to Ardour most pro-engineers are going to want the untouched unprocessed audio, while the hobby and prosumer video dudes will want to export everything pre processed so that they don't have so much rope to hang themself in Ardour. So really we have several different scenarios that have to be thought thorough...

At first I was against using LV2 or VSTs at all in the NLE... But after some consideration, I think this may be the quickest way to get that audio sweetening buttons that every editor wishes was there. It also allows for us to piggyback on all the amazing software that has already been developed.

However, there is a problem with this approach too. VSTs are mostly only on Windows - even though they work in theory everywhere most developers use platform specific GUIs. LV2 has the same problem on Linux. Although I love the format, it's not standard enough to be embraced by the cross platform community that Olive is trying to form. AudioUnit has some of the "BEST" most advanced plugins. But getting AU working on open source is pure voodoo from what I understand. (Ardour has it kinda working on MacOSX...) So no mater what standard we choose we are limiting our audience...

Still another possibility would be to bring some of the best plugins inhouse and offer sane defaults the way that Ardour has begun to do with it's compressor and equalizer. This is sadly all very political in ecosystems surrounding OSes that we don't really control. So we need to think long and hard about how to sidestep as much politics as possible.

I've been mulling this around for some time now. I wonder if its better to have a few sane defaults with great export to an interchangeable format if that is better than trying to bring the whole plugin ecosystem under our roof. Not sure there is an easy answer for this. All I know is pretty much every NLE - commercial or open source - does audio totally sub par.

There is a lot of room for innovation here!

[Symbian9]

Rack 1.0 coming soon with VCV Rack v1 plugin C++ interface

• https://twitter.com/vcvrack/status/1129454065094725632

The VCV Rack v1 plugin C++ interface is now stable. Developers may begin porting their plugins to Rack v1 See https://vcvrack.com/manual/Migrate1.html for a migration guide. More details and development builds at https://community.vcvrack.com/t/rack-v1-development-blog/1149/501

[unfa]

Honesly, I do not think a "make it sounds better" button is a good idea.
As an audio guy I cringe at that thought.

I am all for LV2 plug-in support though.

As if somone makes a "Magic Red Button" plug-in - I'm all for that. I will never use it, but the choice is there.

But having such a function built-in in Olive seems incredibly amature to me.

I am afraid giving users a "make things right" button will only promote stupidity and cluelessness among creators. That's why I so dislike the concept of "Soundgoodizer" or "Sausage Fattener" type plug-ins.

Nothing will make your work good, if you have no idea what you're doing. You need to learn the craft, or employ an expert, there is no thinrd good way IMHO.

I want to understand what is going on with my signals, not do random stuff that makes it "better" - if I don't know what is going on I migth be making it sound much worse and I can't really tell.

I don't say experimentation si bad - but it's working best if you can analyse what you've done and why you've got the final result. With a "back box" processor you can't learn that. It's like a drug - it may make things seem easier and better in the short term, but it'll really hurt you in the long term.

If a plug-in at shows you what it's doing and provides sane presets - fine. You can learn from that.
I'm all for such a plug-in. But I don't think we have an open-source solution like that yet.
We have many great plug-ins that a skilled user can do wonders with.

[unfa]

@frink - feel free to contact me via e-mail:

[frink]

@unfa I'm in a crunch for the next week but I'll try to reach out soon...

Olive is not and should never be a DAW even at the most basic level. The mixer interface and audio tools need to be focused on video editing instead of audio production. I think Ardour's a-compressor would be one of the better suggestions to borrow code from for this purpose...

Personally, I'd like to see us remove VST and LV2 and leave that to bigger guns in a dedicated DAW. You'll get a cleaner interface for both audio and video and provide a better overall workflow for everyone. But there does need to be some basic audio capabilities in the video editor. Mixing, soloing, muting, level and panning automation and probably stem grouping for export. Plus there needs to be a way to quickly and temporarily "normalize" with basic compression/limiting so that editors can better here poorly recorded audio when cutting dialog scenes.

As far as audio plugins in the video editor, without a mixer interface this is not useful at all. (And the sentiment is highly against a mixer window... #944) Audio editing and video editing workflows are very different. The workflow for audio in a video editor really needs to be focused on clean video editing. Complicated audio work should be relegated to a dedicated editor in a DAW.

The ultimate goal for Olive is to support the professional workflow which puts audio design outside the landscape of the video editor. (See #944 for most recent details...)