Universalmediaserver: Videos don't work in Safari on iOS and macOS

[SubJunk]

Videos don't play for me

[SubJunk]

Confirmed still a problem in 7.4.0

[Nadahar]

I don't know if the cause is the same, but #1351 might be related. #1351 seems to be a simple problem with the UI scaling that doesn't work properly on macOS, and should be very easy to fix for anyone with some web knowledge.

If the scaling issue was fixed, it might be that video playback shows the same symptoms on desktop versions.

[Nadahar]

It seems like the issue is the same for the macOS version of Safari. When testing with #1591, video playback only works when using the Flash player. The HTML5 player doesn't seem to do anything. I assume that Flash isn't available on iOS, and as such the issue is probably the same.

[SubJunk]

Thanks for the details, I will be looking into this soon

[SubJunk]

I noticed that our sample web feed for TED Talks works in Safari HTML5 mode so we can possibly learn from that

[SubJunk]

Well the problem is pretty clear after looking at the browser support tables at https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats
We are transcoding to OGG and Safari has no support for it. So the fix here will be to transcode to MP4 for Safari probably

[SubJunk]

Ok so it doesn't like our MP4s either, from:

ffmpeg -re -y -loglevel fatal -i inputfile.mkv -filter_complex scale=720:404 -g 52 -c:v libx264 -preset ultrafast -c:a aac -ab 16k -movflags frag_keyframe+empty_moov -f mp4 

[SubJunk]

Tried and failed with the Chrome trick we use too:

ffmpeg -y -loglevel fatal -i inputfile.mkv -filter_complex scale=720:404 -c:v libx264 -profile:v high -level:v 3.1 -c:a libmp3lame -ac 2 -pix_fmt yuv420p -preset ultrafast -f matroska 

with video/webm and video/mp4 MIME types, so it seems our existing tricks don't work on Safari, will need to find new ones

[ExSport]

Hi. Not sure if there is not some new parameter in ffmpeg but in past it was possible to move index from end of file to the start.
ffmpeg -i input.mp4 -codec copy -movflags +faststart output.mp4
Or check here: https://github.com/danielgtaylor/qtfaststart

[SubJunk]

@ExSport thanks, and good to see you :)
Do you remember why we don't always use MP4 by default? Since it seems to have the best support across all browsers

[ExSport]

Hi :)
I am still around...
Not sure but I suppose it had something to do with moov atom. By default mp4 index is at the end of file but such approach is not suitable for streamed videos. Maybe it is root cause why mp4 files are not playable in browsers? I know there was/were special parameters for moving it to the beginning of file in ffmpeg so it can be played, like ogg and other web formats. But my memory is very short so maybe I am wrong😎

[SubJunk]

@ExSport I'm going to experiment with this now if I have enough time. Also did you see your mention on our SourceForge interview this month? You are in the first answer https://sourceforge.net/blog/october-2018-staff-pick-project-month-universal-media-server/ :)

[ExSport]

Long time when I did some code last time 😎 Btw. missed that interview. Now reading first answer:
Thanks to our talented past and present developers like valib, Nadahar, SharkHunter, Sami32, skeptical, and many more!
Good I am not there by name as definitely I am not Java talented guy with my almost zero knowledge and no support for last year(s)🤗

[SubJunk]

Damn I was sure I added you to that list... Well I appreciate all your contributions, you have done some great ones

[ghost]

So after some preliminary testing i determined the issue to be that Safari requires byte-range support (where the current one just streams all the data directly after sending some appropriate http headers), i then proceeded to attempt to implement this support into the RemoteMediaHandler.java, but ultimately failed to make it work properly, with only a few seconds playing sometimes.

Where chrome/firefox sends header "Content-Range: bytes 0-".
But Safari sends header "Content-Range: bytes 0-1" initially and expects a range it can request with a 206 code.

I made changes to WebRender.java

    public String getVideoMimeType() {
        if (browser == CHROME) {
            return HTTPResource.WEBM_TYPEMIME;
        } else if (browser == FIREFOX || browser == SAFARI) {
            return HTTPResource.MP4_TYPEMIME;
        }
        return defaultMime;
    }

And to RemoteMediaHandler.java

            if ((long)range.getEnd() == resource.length()) {
                OG Implementation here
            } else {
                //Inform the debugger of the data request type
                LOGGER.debug("Browser wants byte range!");

                //Client wants byte-range
                code = 206;

                //Add headers
                Headers headers = httpExchange.getResponseHeaders();
                headers.add("Content-Type", mimeType);
                headers.add("Accept-Ranges", "bytes");

                //Get available byte-range
                long start = range.getStart();
                long end = range.getEnd();
                int availlength = in.available();
                long length = end-start;

                //Client wants too much data
                if (length >= availlength || start>=end) {
                    //Range not satisfyable.
                    code = 416;
                    //Limit range
                    headers.add("Content-Range", "bytes */" + availlength);
                    length = availlength;
                    end = end > length ? length : end;
                    start = start > end ? end : start;
                } else {
                    //Send available byte-range
                    String rStr = start + "-" + end + "/" + availlength;
                    headers.add("Content-Range", "bytes " + rStr);
                    headers.add("Content-Length", "" + length);
                }
                LOGGER.debug("Sending range {} to {}",start,end);

                headers.add("Server", PMS.get().getServerName());
                httpExchange.sendResponseHeaders(code, 0);
                OutputStream os = httpExchange.getResponseBody();
                if (renderer != null) {
                    renderer.start(resource);
                }
                if (sid != null) {
                    resource.setMediaSubtitle(sid);
                }

                //Append data to http response
                RemoteUtil.dumpLimit(in, os, renderer, (int)length);
            }

Where dumpLimit is just dump but with os.write(in.readNBytes(length)); instead of the write loop.
But again, this doesn't work consistently or well, and only sometimes displays some 20 seconds.

[SubJunk]

@outnos good start. It could be good to make a branch for that so we can use that as a base to improve upon