Ffmpeg-python: create a new filter with a specific signature selecting a subset of actual ffmpeg filter args?

[ljwobker]

Apologies for what might be an incredibly dumb or obvious question: I'm new here. I was thinking about trying to add a 'fade' filter -- as much to learn the interface and repo as anything else... then my usecase morphed a little bit into something more like "What if I wanted to implement a filter that exposed a subset of the arguments or mangled them in a specific way?". A couple of illustrative examples might be:

• implement a fadeByTime filter, where the signature was something like:
  def fadeByTime(stream, direction=in, offset=0, fade_len=0, **kwargs) -- but in my contrived example, the offset and duration would be interpreted in seconds rather than in frames... In the code I would then map the inputs of this function to the actual ffmpeg fade filter with the arguments {t=direction, st=offset, d=fade_len}

In much more general terms, I guess I'm asking how this library knows or decides that some parameters for some filters are position and some appear to be defined by keywords.

https://ffmpeg.org/ffmpeg-filters.html#fade

[153957]

The following should be possible:

ffmpeg.input(…).filter('fade', type='in', start_time=2, duration=1).output(…).run()

To create a fade in after 2 seconds lasting 1 second.

This just uses the simple custom filter option in ffmpeg-python: https://github.com/kkroening/ffmpeg-python/blob/master/ffmpeg/_filters.py#L28

All args and kwargs are just directly passed to ffmpeg 'verbatim', i.e. exactly as you provide them.

[ljwobker]

Sorry - I should have made it more clear that I was interested more in learning how the code in this particular repo works and is structured (so I can do additional filters or whatever) rather than just solving the example "fade" problem. I figured if I could learn how to implement 'fade' I'd be close enough to anything else I wanted to do. ;-)

[153957]

If you want to implement your own function to apply the fade filter to an input you can do it like this:

def fade_by_time(stream, type, start_time, duration):
    return stream.filter('fade', type=type, start_time=start_time, duration=duration)

But maybe I misunderstand and you really want to add your own filter operators, in which case I would suggest your just look at other implemented filters in the filter module, for example: https://github.com/kkroening/ffmpeg-python/blob/master/ffmpeg/_filters.py#L216

[kkroening]

Yeah, I'm not sure exactly what you're asking, @ljwobker. Does @153957's comment help you? I'm going to close this for now, but if you have further questions feel free to comment here.

[ljwobker]

The comment does help - I'm trying to understand how the arguments are "aligned" in the python definitions versus the actual filter syntax. I don't totally understand how the actual ffmpeg filter syntax (with respect to arguments) is built, and in turn how the local bindings map to that...

[kkroening]

In short:

• Every kind of operator (input, output, filter) produces a certain kind of Node instance.
• A node refers to a set of upstream parent nodes along with any additional metadata. For example, an input node has no parents, and a filter node can have one or more parents (e.g. an input node or other filter nodes).
• Nodes are immutable: once created, they contain all the metadata and reference all the other upstream nodes.
• Adding additional nodes to the graph simply consists of making more node(s) that reference other upstream node(s), similar to how a git repository keeps track of history with commits that point to parent commit(s) (which in turn point to other parent commits and so on, maintaining the entire history of the repo).
• Together, a set of nodes represents an ffmpeg signal graph as a directed acyclic graph (DAG). (Again, think of git and how it forms a DAG from immutable commits)
• Once the entire signal graph is built (using a combination of ffmpeg.input / .filter / .output / etc), it can be _compiled_ into the corresponding ffmpeg command-line arguments using ffmpeg.compile (which ffmpeg.run uses internally).
• Compilation consists of a topological sort of the DAG. The graph is traversed in a topologically sorted order and sequentially converted into a linear sequence of command-line snippets, in the same way that a programming language compiler converts an abstract syntax tree (AST) into machine code. One can think of the ffmpeg-python signal graph / DAG as an AST and most of the concepts map accordingly.
• Care is taken during compilation to allocate unique stream identifiers to be used in the -filter_complex command-line argument (e.g. the things that show up as e.g. [s0], [s1], etc.
• The end result is that the signal graph DAG/AST is compiled into a set of ffmpeg command-line arguments.

A key thing to note is that everything in this design is functional/immutable: constructing a graph consists of creating nodes that refer to other nodes, and then compiling the graph into ffmpeg command-line arguments is a pure function that takes a graph (downstream leafnode references) and produces a set of strings.

Useful background info might be to look at simple compiler/AST examples on the web to understand some of the compilation concepts.

Does this explanation help?

(At some point this info/explanation should probably be copied into some actual docs somewhere, so feedback on the above explanation is helpful)