Mpv: Subtitles no longer autoloaded

[tomba4]

So, the 2017-02-12 changelog says: "sub: remove .txt as text subtitle extension", which leaves me with no subtitles support. Is there a way to readd .txt as subtitle extension? I couldn't find anything in the documentation.

[ghost]

Why do your subtitles have a .txt extension?

[tomba4]

That's the default where I come from. Almost all subtitles produced in the last ~20 years are in .txt.

[ghost]

And where is that?

[tomba4]

Poland.

[wiiaboo]

Could you upload a sample? I've never seen a .txt subtitle before.

[tomba4]

.txt is used as a container for various formats. Some examples:

1) time-based (rarely used anymore)
00:01:54:Pomóż mi! Nie!
00:01:57:Akcja rozgrywa się między 09:00,|a 10:00 wieczorem
00:02:00:w dniu prawyborów prezydenckich|w stanie Kalifornia.

2) frame based (old)

[2453][2482]Poprosiłem fotografa.
[2484][2526]Najstarsze zdjęcia 506. Pułku.
[2528][2554]Pochodzą z Toccoa.

3) frame based (currently in wide use):
{5390}{5414}Czas na nas.
{5414}{5486}- Co z Sheldonem?|- Nie idzie.
{5488}{5543}Co powiemy Siebertowi?
{5543}{5735}Powiedzcie, że dr Cooper woli|zedrzeć maskę natury

[wiiaboo]

That's horrible. Why would they choose an extension that's used everywhere to indicate a simple text file as something to be interpreted as subtitles?

[tomba4]

Well, it's a text :) and I'm not sure what other choices were available ~20 years ago when the subtitles scene started. In any case, it has worked quite well all those years.

[wiiaboo]

I can't think of a way to fix this completely. We either revert to accepting .txt again and screw people with .txt on the video's directory or use an heuristic and only allow .txt if prefixed by 'pl' or 'pol'?

What do you use to get these subtitles, subliminal?

[kkkrackpot]

Maybe rename subs.txt to subs.pol and make mpv ttreat that .pol file as text subs?

No idea if it can work out or not, just couldn't help showing myself off. Sorry...

[tomba4]

Just wondering: how is .txt a problem? I have many videos with .txt that's not subtitles and never had any problems, the file is just not loaded (txt "[file] Stream opened successfully", followed by "Can not open external file file.txt").

I've never used subliminal. For a very long time there have been large polish-only sites that dealt with subtitles and although recently they started switching to .srt, the vast majority is .txt.

Renaming is not really an option, since people's collections often are on DVDs.

Would it be a big problem to add supported extensions in the config file?

[wiiaboo]

@fhlfibh same effort as ffmpeg-converting the text subs to srt before playing.

[dragons4life]

Do you mind uploading a sample file with the .txt subs?
Maybe try https://www.subtitle-converter.com/

Though I think the idea of choosing the files extensions mpv should load would be better, have a default set for obvious ones .srt .ass .idx .sub etc... and let people add whichever extensions they want to load?

[termenti]

I reverted mpv to previous version when I find out that it don't support *.txt subtitles.
I propose two workarounds which are not mutually exclusive:

Workaround:
Usually filename of .txt subtitles is the same as video file, so you could accept and auto-load *.txt subtitles if video and subtitles name are the same, for example:
some video.mp4
some video.txt
subtitles auto-loaded, and if filename mismatches .txt subtitles will be not loaded.

Additional workaround:
You could add small button in lower right corner of window and shortcut key to open directory browse window for user, so that he could choose another file with subtitles.

[wiiaboo]

Workaround:
Usually filename of .txt subtitles is the same as video file, so you could accept and auto-load *.txt subtitles if video and subtitles name are the same, for example:
some video.mp4
some video.txt
subtitles auto-loaded, and if filename mismatches .txt subtitles will be not loaded.

That's --sub-auto=exact/fuzzy behavior exactly, not a workaround.

[ghost]

You can just rename the .txt files to .sub or something else random.

[tomba4]

Renaming is not an option as I mentioned previously. I guess it all boils down to a question whether there are other mpv users who lost subtitle support with this change and if there's enough of them for this update to be reverted...

[termenti]

Workaround:
Usually filename of .txt subtitles is the same as video file, so you could accept and auto-load *.txt subtitles if video and subtitles name are the same, for example:
some video.mp4
some video.txt
subtitles auto-loaded, and if filename mismatches .txt subtitles will be not loaded.

That's --sub-auto=exact/fuzzy behavior exactly, not a workaround.

--sub-auto=exact/fuzzy/all is not working, subtitles are not loading. Maybe because .txt subtitles are not supported...

And if I would want to change all my *.txt subtitles, what extension/format should I choose to not have problems? What subtitle extensions/formats are most popular/trouble-less?

[wiiaboo]

.sub, .srt, .vtt, etc. It just matters that it's an extension that'll get autoloaded. You don't even need to convert them.

[mgorny]

Maybe it's time to abandon the DOS age and start recognizing files by contents rather than filename suffix?

[ghost]

So you want mpv to open and probe _all_ other files that happen to be in the same directory?

[CounterPillow]

As an outside observer with no horse in this race as I neither use external subtitles nor maintain any mpv code that will suffer as a result from special workarounds, I'd like to raise some points in this now somewhat lengthy issue.

Maybe it's time to abandon the DOS age

Implying that one side's arguments are plagued from a bad case of being stuck in the past is an annoying rhetoric tool, @mgorny, and is likely going to piss people off unnecessarily. We all have our bad days, but simply because your interests don't align with those of the developers, there's no need to be cheeky. You can't just use libmagic to get the mime-type of a file that has no mime-type because it piggybacks on an extension, is extremely obscure and also just some special formatted plain-text. The amount of heuristics and string-parsing involved would be not worth the damage to the codebase's maintainability as I see it.

As far as I remember, the reason why .txt was dropped as an autoloaded subtitle extension was because people were having issues with mpv trying to load random txt files as subtitles, and since people not from Poland use other subtitle formats or god forbid even mux them into their containers, I believe the mpv developers had good reason to assume that keeping .txt as an autoloaded subtitle format was not worth the problems it caused.

Going back to awkward heuristics and string-parsing, that's the kind of thing where C is probably not going to make a pretty solution, and in addition to the relative niche use here, a heuristic .txt subtitle autoloader might find a good home in a Lua user-script. That is, if someone who wants this functionality is willing to step up and write that script.

[tomba4]

As far as I remember, the reason why .txt was dropped as an autoloaded subtitle extension was because people were having issues with mpv trying to load random txt files as subtitles

Is it really the case though? I mentioned earlier that I never had such problem and I'm genuinely interested why and how would a random .txt be used as a subtitle.

[kkkrackpot]

@tomba4 The reason for dropping txt subs is explained here https://github.com/mpv-player/mpv/commit/50991fac81eb56bc0985af2c9d31cd503302dcfe

[tomba4]

@fhlfibh It doesn't explain why .txt was dropped, it only explains that --log-file=something.txt will break mpv. That's not a subtitle problem.

If used with fuzzy matching, the player tends to pick up random text
files, sometimes with interesting results.

This is the part I'd like to see happening, because I have never seen such effects on .txt files which are not subtitles.

[wiiaboo]

@fhlfibh It doesn't explain why .txt was dropped, it only explains that --log-file=something.txt will break mpv. That's not a subtitle problem.

You're right. It's not a subtitle problem. It's autoloading .txt files as subtitles that's the problem. So it was fixed and you're asking to revert it to considering any .txt file as subtitles.

What's so hard to understand about this? Have you tried opening the --log-file resulting file? Do you not see how that would confuse the FFmpeg parser of .txt subtitles?

[ghost]

The problem with --log-file is actually that it is being opened and probed while it's still appended. Since the prober doesn't find a format, and the normal action is to feed it with more data, it tries to read more data from the file, which for some reason makes the probing process take a lot of time.

[wiiaboo]

So maybe use some arbitrary limit to how much FFmpeg tries to probe subtitle files? ASS files with riced typeset can be whole MBs, though, so it's a poor fix.

[ghost]

FFmpeg doesn't indicate whether it needs more data to make a sure guess, or if the probe score is just low. So the default behavior is to give it more data until it reaches a maximum (2MB), or returns success.

[wiiaboo]

Suggest people to use --log-file only with ".log" instead of ".txt"? :)

[mgorny]

You could detect that you've opened that file for writing already. Possibly fstat() it after opening and store dev+ino no somewhere, then fstat() the potential subtitle file and compare it with previous log file.

In a more lazy manner, I think you might be able to do with file locks (i.e. locking the log file for writing, and refusing to process subtitle files that are locked) but I don't recall which of the locking mechanism on Linux works properly for the same file being open multiple times in the same process.

[ghost]

These ideas are absolutely terrible.

[mgorny]

Preventing mpv from using logfile as input by recognizing it's a logfile is an absolutely terrible idea. Making random assumption that people will use .txt for logfiles and not for subtitles is a great idea. Sure. That makes a lot of sense.

[garoto]

In a more lazy manner, I think you might be able to do with file locks (i.e. locking the log file for writing, and refusing to process subtitle files that are locked)

Like a mutex?

[floppym]

I don't recall which of the locking mechanism on Linux works properly for the same file being open multiple times in the same process.

I think that would be "open file description locks". See F_OFD_SETLK and F_OFD_GETLK in fcntl(2).

[mgorny]

Or even simpler, flock(2). Basically, after opening the file for writing log:

flock(log_fd, LOCK_EX);

...which is a good idea independently of the issue at hand. It sets an exclusive (write) lock on the file. Note that if another process has already set a lock on the file (e.g. another instance of mpd is writing to the same log file) this will block the program until that lock is released (e.g. by the other instance exiting). If you want to avoid that, you can add |LOCK_NB to avoid it return with EWOULDBLOCK instead.

Then, in the code opening files for subtitle search:

flock(sub_fd, LOCK_SH|LOCK_NB);

Which simply obtains a shared (read) lock on the file, in non-blocking mode. If the file is locked for writing, it will return -1 with EWOULDBLOCK. This not only solves the immediate log problem but it is a generic good idea to avoid using files that are being locked for writing (i.e. are still being written).

[ghost]

That idea is rejected, because it's dumb.

[Amynka]

Do you have any specific reason why exactly you think it is dumb idea?

[ghost]

File locks have never been a good idea. They have a chaotic history on POSIX and Linux and never were sane. I don't even know why you'd want to block read access on a log file of all things. Or why you'd use something so intrusive and unportable to fix some corner case.

[mgorny]

So is it better to discriminate a whole nation for a corner case of few people deciding to output a log into .txt in the movie directory?

[ghost]

Yes, we did that on purpose to piss off the polish people.

[tomba4]

I guess we shouldn't be surprised that mplayer arrogance was forked together with the code...

[ghost]

I guess you shouldn't be surprised if I won't bother implementing a fix after all this.

[tomba4]

You mean about the disrespecting comments from mpv contributors, despite me (the issue creator) being nothing but reasonable with explaining the situation and answering all questions?

Yes, I'm not surprised. Please go ahead and close this ticket.

[ghost]

You mean about the disrespecting comments from mpv contributors,

Where?

Please go ahead and close this ticket.

No, it's a semi-legitimate issue. We're used to multimedia being terrible.

[wiiaboo]

@tomba4 try this lua script hacked from autoload.lua: https://gist.github.com/wiiaboo/a8f355905aaa2da5f11f5202d9a0d099

[tomba4]

@wm4 Well, since mgorny's first reply was considered offensive (and it was), wiiaboo's reply that autoloading of .txt was the problem and the problem was fixed should be considered that as well. And things went completely downhill since mrgorny's fstat() idea...

Anyway, personally I can hexedit the binary and restore the .txt support (which is what I did), but what I was hoping for was to bring the functionality back to the rest of mpv users. In my initial post I even gave an idea how to do it in a way that would make everyone happy (i.e., make the extensions configurable).

[kmikolaj]

Are there any plans to revert this insanity?

[tomba4]

:) Probably not. I guess Poland might be the only country using .txt for subs...

[wiiaboo]

There's no need to revert it. Just use https://github.com/mpv-player/mpv/issues/4144#issuecomment-282491560 for your .txt subs.

[Glymbol]

Ah, so I finally know why subtitles don't auto load anymore... Thanks for the script wiiaboo.

[haasn]

As a newcomer to this issue, I'm honestly surprised nobody seems to have suggested the (imo) simplest fix:

1. Distinguish between “safe” subtitle suffixes and “risky” subtitle suffixes
2. Don't “fuzzy match” risky suffixes, but require them to match the video name exactly.

Sounds like the intended use case (e.g. OP) has .txt files that exactly match the video filename, and the broken case (e.g. 50991fa) describes it potentially blowing up when using fuzzy matching.

So don't use fuzzy matching for potentially risky extensions like .txt that could mean anything?

[haasn]

Do you have any specific reason why exactly you think it is dumb idea?

1. It doesn't solve the issue in general, only one specific corner case.
2. It's an extreme hack and layering violation.
3. It has insane consequences like not being able to tail -f your own log. (If I understand wm4's comment correctly)

[pink-mist]

It has insane consequences like not being able to tail -f your own log. (If I understand wm4's comment correctly)

I've no idea if that's what wm4 meant or not, but it's not true. A file lock like the one suggested is only advisory, and so it only applies to processes checking for the lock, and tail does not.

[konserw]

It doesn't solve the issue in general, only one specific corner case.

Your general issue is by itself pretty corner case - i.e. it's rather rare case in my opinion that you have text file (with any extension) that has the same name as video you want to play, in the same same directory, and it is not your subtitles.
Only valid case in my opinion would be when you have "fuzzy" or even "all" policy for subtitles set, but then
"fix" that removes "*.txt" from subtitles extension doesn't work for files like "some.ass" or even that bug with reading your own log if you provide for instance log name as 'log.utf' for whatever reason.

It's an extreme hack and layering violation.

What's extreme about it?

It has insane consequences like not being able to tail -f your own log. (If I understand wm4's comment correctly)

Nope - from what I understand @wm4 proposal is to:

1. Lock log file for writing - so tail (which is reading, or even don't bother about locks as per @pink-mist comment) work just fine with log file. That would also probably mean that 2 mpv instances won't be able to log to the same file (if implemented checking).
2. When opening file as subtitle check if it is locked for writing - so you won't open your own log file (see 1.) or file that is in the middle of transfer/unpacking/whatever (not sure about that)

What I call insane consequences is disabling subtitle file format that is most popular in 38 000 000 people country (I know it's not so big, but still not negligible I think)

All in all most reasonable solution in my opinion would be adding some configuration option to enable .txt subtitles, or even enumerate all subtitle formats that user want to be loaded - that is pretty future-proof. Waht would you say about that?

[wiiaboo]

What I call insane consequences is disabling subtitle file format that is most popular in 38 000 000 people country (I know it's not so big, but still not negligible I think)

What about a general file format that's used by the rest of the world to represent a simple text file that's not supposed to be specially interpreted? Why isn't it called .iso or .exe?

[konserw]

What about a general file format that's used by the rest of the world to represent a simple text file that's not supposed to be specially interpreted? Why isn't it called .iso or .exe?

Don't be mistaken - I'm not defending .txt as subtitles extension - but after all subtitles are (specific for of) text - I also think it is terrible idea, then again I can't just batch rename thousands of subtitle files floating around internet.

Also I'm not trying to force you to create or use subtitles in .txt format, I'm marely asking for possibility to do so without rebuilding entire mpv or relying on 85-line third party lua script I don't even know where to paste.

[tomba4]

I have already suggested to have an option instead of hardcoded list of extensions, but the discussion went in another direction, so don't count on it.

[marcin-gryszkalis]

What about simple --additional-subtitle-ext=txt that would extend sub_exts[]?

I have similar case where neither rename nor manually loading subtitle is not an option (computer illiterate elder user with movies on dvds)...

[marcin-gryszkalis]

@konserw regarding lua script, you can put in into ~/.config/mpv/scripts (works with raw mpv) or load with --script (this is the way I could make it work with smplayer).

[konserw]

@jerkstore369 - this is exactly what I was asking for.

[Jj0YzL5nvJ]

Someone has to make a pull request to add support to increment the hardcoded extension list with custom extensions by user request, that’s all.

Let's not mention non-standard extensions and let "libass" or whoever to handle the worst scenario possible. Let's not force the cannibals to use fork and knife in name of "progress".

[akrasuski1]

I was annoyed by the same problem (.txt not autoloading, forcing to use command line options). For future reference: if you rename the subtitle file to .sub, everything works fine.

[Akemi]

stale and a workaround exists.