Universalmediaserver: Ability to test FFmpegVideo

[SubJunk]

We could detect and prevent bugs like https://github.com/UniversalMediaServer/UniversalMediaServer/issues/1648 if we had tests for our FFmpeg transcoding that sends the instructions we generate to FFmpeg and detects if it gives an error.

This is part of my vision for the future of our testing suite, which when combined with a library of real files and MediaInfo tests, could give us some nice coverage to detect regressions in those external libraries. Right now there is a lot of uncertainty and manual testing needed when we update those libraries and it prevents us from updating them (our FFmpeg version is over 2 years old) so this would give me more confidence in those updates when we do them. Particularly MediaInfo since they frequently change the way they report certain things which results in us needing to do code changes, but FFmpeg will be really useful too.

[stasinos]

I looked around a bit, and found mediainfo being indirectly used through ffmpeg in net.pms.dlna.DLNAMediaInfo::parseFFmpegInfo()

Are there other places as well, or would the mediainfo part of this Issue be testing that ffmpeg output is parsed correctly?

[stasinos]

There are also a bunch of projects that could considerably alleviate the ffmpeg/mediainfo pains. These two are the most actively maintained:

https://github.com/filebot/filebot
https://github.com/bramp/ffmpeg-cli-wrapper

Have these been considered and rejected for some reason?

[SubJunk]

@stasinos I don't understand how filebot could be used for this, can you explain?
I haven't seen ffmpeg-cli-wrapper before but it could be good. However it might not be relevant for what I meant here, I'll try to explain better since I didn't give enough detail in the initial post :)

We have a lot of different commands that we pass to FFmpeg for transcoding, it is very complicated in there with all the different user and renderer configurations that affect those commands. This exposes us to a lot of edge-case bugs like #1648 where that would only happen on renderers with KeepAspectRatio=true, and since most renderers do not set that, the bug can go unnoticed by even those of us who use UMS every day.

So it would be cool to have automated tests that start FFmpeg transcoding within various contexts and verify that it started to transcode and did not report an error.

A short way to say it is that we should have tests that verify that we are sending correct parameters to FFmpeg under a lot of contexts.

[stasinos]

@SubJunk filebot implements a Java API that wraps ffmpeg for its own internal use. We can use the Java API ffmpeg, just as we would with ffmpeg-cli-wrapper.

Both wrappers are actively maintained, so (to some extent) they abstract and insulate against minute changes in ffmpeg/mediainfo. I haven't looked at either API in much detail, but one would expect that UMS would at least be able to get rid of maintaining a parser for console output to get ffmepg/media reporting.

Naturally, the ffmpeg execution parameters for the different renderers will have to be maintained by UMS. But the code can get significantly simpler if it is using a Java API.

[stasinos]

But the above are just food for thought for a future issue. Coming back to the current issue, I have started a branch where I am struggling to figure out how to start a headless UMS and successfully call FFMpegVideo::launchTranscode(). Tracing execution flow from DLNAMediaInfo::parseFFmpegInfo() upwards, FFMpegVideo::launchTranscode() seemed the right depth to test as a unit.

I have a some preliminary explorations at
https://github.com/stasinos/UniversalMediaServer/tree/1651_mediainfo_ffmpeg_tests
but it is still in a very sorry state. Is it ok to PR although it's nowhere near usable just so that you guys can comment and suggest, or would it be bad manners?

[SubJunk]

I like having incomplete PRs, a lot of ours are in that state :) Sometimes we are explicit with the Not ready label to make it clear. Also if you install the ZenHub browser extension you can put it into the In Progress pipeline to indicate it is a WIP

[Nadahar]

@stasinos Even though I'm no longer a part of this project, I've been a quite active developer for 2-3 years.

I don't know your working environment, and I'm not sure why you need to debug UMS in headless mode. You'd need to use a renderer to play something that made UMS call launchTranscode(). That said, I don't know why you'd want to do debugging from a headless state, but you might be one of those that does everything in a terminal :wink:

Personally I use Eclipse for everything and would really recommend it. Even though I know there are many people that dislike Eclipse for all kind of reasons, I must say that I think the powerful refactoring functions and reference tracking are unparalleled. Whenever I have to use a different IDE for some reason (for other languages), I constantly find myself looking for things that simply aren't there.

For debugging I would absolutely fire UMS up from Eclipse, put a breakpoint where you want to start debugging and use a software renderer to initiate things. I use mainly use Kodi for this, which I run in another window making it easy to control "both sides" form the same desktop.

Kodi has its shortcomings when it comes to UPnP AV/DLNA, but it's the software renderer that works best as a general test tool. I've had some issues with v17, so I'm sticking to v16 for now. In Kodi 16 everything works except viewing images. Browsing images works fine though. One thing to be aware is that Kodi caches a lot, and in particular it caches all thumbnails. That means that if you want to troubleshoot thumbnails, you'll have to keep manually clearing Kodi's cache folder.

VLC 3.x has finally got their UPnP AV/DLNA support up and working, so it's a possibility to use as well. I don't have much experience using it though, since there have been so many problems with their implementation in the past that I've largely given up using it.

For audio media Foobar2000 and MediaMonkey are other possible software renderers. Personally I think MediaMonkey is confusing and inconsistent, but Foobar2000 has a very good UPnP AV/DLNA implementation. Since it's limited to audio media and browsing, I don't use it that much though.

I hope some of these tips can make debugging easier for you.

[stasinos]

@Nadahar thanks for the pointers.
For what it's worth, I use the trace log and emacs when that is more convenient, and I also use eclipse (with emacs bindings, of course) when stepping through the code is more convenient.

[stasinos]

Headless mode is, I believe, more appropriate for these test cases. I see no reason starting the GUI to test ffmpeg outputs.

[stasinos]

@SubJunk I don't see anythink to click to apply labels to the PR. Maybe it's a members-only thing?

[Nadahar]

Headless mode is, I believe, more appropriate for these test cases. I see no reason starting the GUI to test ffmpeg outputs.

What mode you use doesn't really matter, UMS behaves exactly the same. I simply don't know how to debug from headless mode :wink:

I don't see anythink to click to apply labels to the PR. Maybe it's a members-only thing?

It has to do with permissions. I used to be able to set labels here, but isn't anymore since my permissions have been revoked.

[Nadahar]

This is a digression, but I thought the emacs crowd had only died out. I haven't used it since 1995, and these days I mostly hear praise for VI(M) from those that are eager to "spread the word" regarding a text editor.

I have some vague memories about Ctrl X + V or Ctrl X + C and that nothing generally behaved as expected, but that's where it stops :smile: I've never liked VI(M) either, and use Nano or Pico if I have to edit text files from a non-GUI environment. If I can choose, I'll use things like Notepad++ or Eclipse every time though.

[SubJunk]

@stasinos if you'd like labels applied please let me know and I can apply them. I have added the not ready label to your PR since I guess that's what you wanted?
I hadn't realised before that labels are member-only. I would be interested to see how ZenHub behaves for you if you get time to try it, whether you are able to change pipelines

[SubJunk]

We have this ability now. There is a lot of room for improvement but I think this issue has been done.