Streamlink: Twitch.tv embedded ads (2020-09-29)

[bastimeyer]

Since we're seeing reports of new changes on Twitch regarding the embedded ads situation (once again), I feel the need of opening a new thread before it gets too messy again with the issues on the issue tracker here. I will update this post once we've got new informations, so that the current state can be found and read easily without digging through issue threads.

---

Edit 9 (2020-11-28)

Added a new access token request workaround in #3373 which again prevents ads from being embedded into the stream. (additional note: the access token requests have been moved to the GQL API already in #3344).

Like usual when there's a new change, you can install Streamlink from the master branch, or wait for the next nightly installer to be built if you're on Windows, or wait for the next release.

Edit 8 (2020-11-05)

The ad prevention request parameter workaround has stopped working again, as predicted, and the ad "placeholder" screen seems to have been replaced with real ads now :boom:

Filtering out ads via --twitch-disable-ads is however still working.

Edit 7 (2020-10-28)

A new ad prevention has been discovered (#3301) and has been merged into master on 2020-10-28.
If you don't want to wait until the next release (2.0.0 - release date unknown), you can install Streamlink from the master branch, or apply the small change yourself (cf978c5), or as a Windows user, wait for the next nightly build of the installer (>= 2020-10-29).

If you're using the Twitch GUI, you will have to use a debug / pre-release due to other breaking Streamlink changes in regards to the upcoming 2.0.0 release:
streamlink/streamlink-twitch-gui#757

Edit 6 (2020-10-18)

1.7.0 has been released on 2020-10-18.

Edit 5 (2020-10-03)

3220 has been merged as well, which should prevent the embedded ads again (for now (edit: broken again)).

Edit 4 (2020-10-03)

3213 with the new ad filtering implementation has been merged into the master branch. You can install Streamlink from there, as described in the docs, or if you're on Windows without a Python environment, wait until the 2020-10-04 nightly installer has been built.

Edit 3 (2020-10-03)

Removing the platform=_ parameter again from the stream access token API request once again results in no ads being embedded into the stream. This is a game of cat and mouse at this point, which is ridiculous.
https://github.com/streamlink/streamlink/issues/3210#issuecomment-703022351

Edit 2 (2020-09-30)

I've opened a pull request (see #3213) with the new filtering implementation. Feedback is highly appreciated, since I still don't get any ads myself and therefore can't verify it.

If you want to test this and give feedback, install it via python-pip like this (more details in the install docs), but beware that there will be more changes on that branch. Alternatively, install it in a virtual-env, so that it doesn't interfere with your main install.

pip install --user 'git+https://github.com/bastimeyer/streamlink.git@plugins/twitch/embedded-ads-2020-09-29'

Edit 1 (2020-09-29)

Twitch has changed the annotated titles of the ads, and Streamlink doesn't expect this and thus can't filter them out. The issue can be fixed with a trivial change (see post below), but this may introduce other issues in the future. A better filtering approach is being worked on. This trivial change won't prevent embedded ads from happening and only fixes the filtering.

---

As you can read in the changelog, we have recently (re-)added changes in the latest release (1.6.0) to prevent the embedded ads on Twitch from happening. This workaround basically sets the platform=_ request parameter when acquiring the stream URL and that made Twitch not embed pre-roll and mid-stream ads into the HLS stream. I've already said in #3173 that this probably won't last long, just like last time we had it implemented, and now here we are and it's broken again after Twitch has pushed another set of changes.

Another issue seems to be that the ads themselves are now annotated differently and this breaks the current ad-filtering implementation, which exists in addition to the workaround request parameter. We have gone through multiple iterations here as well, because the way ads were embedded has changed quite a lot.

It appears though that Twitch is only rolling out new embedded ads in certain regions right now, because it's not reproducible for me at the time (with a German IP address).

---

Checking the HLS playlist content of the new ads (DONE, thanks for the posts)

Because of that, I'm interested in the content of HLS playlists with embedded ads, so we can debug this. Please share a link to a Github gist post with the playlist content. You can check this by running

curl -s "$(streamlink --stream-url twitch.tv/CHANNEL best)"

The content should NOT look like this (notice the EXTINF tags and their "live" attribute):

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:1234
#EXT-X-TWITCH-ELAPSED-SECS:1337.000
#EXT-X-TWITCH-TOTAL-SECS:1338.000
#EXT-X-DATERANGE:ID="source-1234567890",CLASS="twitch-stream-source",START-DATE="2020-09-29T00:00:00.000Z",END-ON-NEXT=YES,X-TV-TWITCH-STREAM-SOURCE="live"
#EXT-X-DATERANGE:ID="trigger-1234567890",CLASS="twitch-trigger",START-DATE="2020-09-29T00:00:00.000Z",END-ON-NEXT=YES,X-TV-TWITCH-TRIGGER-URL="https://video-weaver.XXXXX.hls.ttvnw.net/trigger/some-long-base64-string"
#EXT-X-PROGRAM-DATE-TIME:2020-09-29T00:00:00.000Z
#EXTINF:2.000,live
https://video-edge-XXXXXX.XXXXX.abs.hls.ttvnw.net/v1/segment/some-long-base64-string.ts
[... list of remaining segments and program date metadata]
#EXT-X-TWITCH-PREFETCH:https://video-edge-XXXXXX.XXXXX.abs.hls.ttvnw.net/v1/segment/some-long-base64-string.ts
#EXT-X-TWITCH-PREFETCH:https://video-edge-XXXXXX.XXXXX.abs.hls.ttvnw.net/v1/segment/some-long-base64-string.ts

[gravyboat]

On windows 10 you need to directly invoke curl.exe in the command otherwise it will run the alias. Your command should look like this: curl.exe -s "$(streamlink --stream-url twitch.tv/CHANNEL best)"

[fronkr56]

https://gist.github.com/fronkr56/75c54414a5aef583bf2326c914e4e1df mine looks slightly different

[bastimeyer]

These are all playlists with regular "live" content. As said, see the #EXTINF tags and their title.
Also, a playlist with ads must include tags like #EXT-X-DISCONTINUITY.

[drmthrowaway]

Note that I could not reproduce unless I set Firefox headers.
curl -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:52.0) Gecko/20100101 Firefox/52.0" -s "$(streamlink --stream-url twitch.tv/$CHANNEL best)"

https://gist.github.com/drmthrowaway/99ddc2308ba431e76945e456da837b75

[bastimeyer]

@drmthrowaway
Thank you. I already suspected it, because it's one of the differences between the HTTP requests which curl (without params) and Streamlink sends (Streamlink sets its UA to Firefox by default).

The TwitchM3U8Parser currently only finds ads which are annotated with Amazon at the beginning of the segment title. Changing this to != "live" should fix the filtering.

Completely preventing ads however is a different topic.

[bastimeyer]

Btw, I don't know if title and title != "live" is the correct solution here, because I'm not sure if this could cause issues with VODs and streams in the future. The VODs I've checked (I have no experience with Twitch's VODs because Streamlink doesn't work well with non-live content and I'm usually not interested in it) currently have empty segment titles, but if Twitch decides to change this, or if I've missed something here, or if regular live streams will have a different annotation in the future for whatever reason, this will break and everything will get detected as an ad incorrectly.

A better fix would be checking for an existing EXT-X-DATERANGE tag with an ID attribute value beginning with stitched-ad and getting its START-DATE and DURATION attribute values. These attribute values can then be used for properly filtering out segments without reading their titles, because each segment is always annotated with a timestamp and duration.

[tp0]

https://gist.github.com/tp0/f4e6798301fb8361c192e17734a0ca16

[quocka]

Any guess why I would not be seeing the pre-roll ads on 1.4? I do get them on 1.6.

[pflanzenandi]

Any guess why I would not be seeing the pre-roll ads on 1.4? I do get them on 1.6.

I am on 1.5 and I do not even have to wait for the pre-roll ads to finish anymore. Each stream starts immediately.

[bastimeyer]

As usual, Twitch is enabling and disabling embedded ads temporarily for specific events, for testing purposes or whatever their plans are. And that often times regionally, which means not everybody will get embedded ads all the time. 1.4.0 had the same filtering logic as 1.6.0, so there's no difference here other than the request parameter I mentioned in the issue post which prevents ads (unless it doesn't).

Regarding the better filtering implementation, I have done some work on my local repo and it's working fine so far, but I'm not ready to open a pull request yet. Also because of another PR that needs to be merged first which this work relies on.

[mlvzk]

Is there a diff/patch available for this that I can try?

[bastimeyer]

As mentioned above, for now, it's just that:

diff --git a/src/streamlink/plugins/twitch.py b/src/streamlink/plugins/twitch.py
index 6bdfd452..e17f0475 100644
--- a/src/streamlink/plugins/twitch.py
+++ b/src/streamlink/plugins/twitch.py
@@ -128,7 +128,7 @@ LOW_LATENCY_MAX_LIVE_EDGE = 2
 class TwitchM3U8Parser(M3U8Parser):
     def parse_extinf(self, value):
         duration, title = super(TwitchM3U8Parser, self).parse_extinf(value)
-        if title and str(title).startswith("Amazon"):
+        if title and title != "live":
             self.state["ad"] = True

         return duration, title

[expiation]

changing the above ^ does not stop ads for me

[bastimeyer]

Please read the thread. This doesn't prevent/stop ads, this fixes the filtering when having --twitch-disable-ads set.

[h1z1]

@bastimeyer - I get the feeling Twitch is reading very carfully what people are trying fwiw. The are still using that name or are rotating it.

[Terra-B-Welch]

So I have noticed inconsistencies in the ad skipping, sometimes it skips, sometimes it doesn't. I've been using the same code every time.

"streamlink --loglevel debug --twitch-disable-ads twitch.tv/(insertnamehere) 360p"

EDIT: Never mind it straight up no longer works. What a bummer.

[bastimeyer]

Btw, there's a PR open with the updated filtering logic and I would appreciate it if I could get some more feedback from users getting ads on Twitch, because I don't and can't verify that it's working correctly 100%.
https://github.com/streamlink/streamlink/pull/3213
See the updated OP from yesterday for install instructions, in case you don't know.

And just as a reminder, this doesn't prevent ads, and it is still unknown if there's another workaround for that.

[drmthrowaway]

It's been working well on my end - the pre-roll ads have been correctly identified and stream starts when they're over. Fortunately (or unfortunately), over the past day and a half, I've only gotten ONE mid-roll ad, which did get filtered correctly.

[h1z1]

FWIW in the last hour or so Twitch has gone full retard wrt ads. Seeing 2minute + prerolls and mid. So while you may not be watching an ad, your player will likely appear hung and eventually timeout.

Was trying to watch the rocket launch on NASA whom have a channel on Twitch. Thankfully they also have other platforms.

[martinsstuff]

Embedded ads should be out for you as well now, just a couple hours ago ads started embedding for me in Germany.

Edit: Using the new uBlock Origin Development version works for me in Chrome though, maybe their fix can be used here as well?

[bastimeyer]

@martinsstuff
This looks like a back-and-forth battle between setting the platform=_ stream token request parameter and removing it. We've already done it once, and removing it a second time is just ridiculous.
https://github.com/gorhill/uBlock/commit/75c58ec7af969141296371aca2b91bea9ae91a6a#diff-d7616deaf3a39e3f19f8d70d098bfe7aR1337

It seems to work though when I test it on shroud's stream. With platform=_ => pre-roll, without, no ads.

This requires a decision how/when this parameter should be applied. We can't just add and remove it every couple of days/weeks and play a game of cat and mouse with Twitch.

---

For now, this is the diff to remove the parameter once again:

diff --git a/src/streamlink/plugins/twitch.py b/src/streamlink/plugins/twitch.py
index 6bdfd452..a327d405 100644
--- a/src/streamlink/plugins/twitch.py
+++ b/src/streamlink/plugins/twitch.py
@@ -317,7 +317,7 @@ class TwitchAPI(object):
     # Private API calls

     def access_token(self, endpoint, asset, **params):
-        return self.call("/api/{0}/{1}/access_token".format(endpoint, asset), private=True, **dict(platform="_", **params))
+        return self.call("/api/{0}/{1}/access_token".format(endpoint, asset), private=True, **params)

     def hosted_channel(self, **params):
         return self.call_subdomain("tmi", "/hosts", format="", **params)

[gravyboat]

@bastimeyer What if we simply define a second access token, one with platform="_" and one without, then we thread the plugin and whichever one doesn't get hit with pre-roll ads or an error is the one that we use?

[bastimeyer]

That doesn't solve the problem at all, because Twitch can simply make arbitrary changes to their backend and serve embedded ads no matter what parameter+value combination is set or is not set. We don't know why the platform=_parameter currently en-/disables embedded ads and why it changes over time. What's for sure though is that _ is just a placeholder and that the value is meant to differentiate between (official) apps on various platforms because of certain policies, contracts, etc. This system can change any time. And Twitch will also always be able to quickly block workarounds which ad blockers or tools like Streamlink implement.

Also, making multiple requests and loading multiple HLS streams, which first need to be resolved from the master playlists and then parsed and analyzed for embedded ads at the beginning, just slows down the initial stream loading by a lot. That nonsense is not worth the time implementing it, not even remotely close.

What we should do here is not hardcode it, because of the rapid changes, as everyone should know by now. Only applying the request parameter to the --twitch-disable-ads Streamlink parameter however doesn't make sense, as the --twitch-disable-ads parameter is meant for changing the HLS logic and it should always be possible to set it without affecting the access token acquirement logic. So what are the options here? Add another Streamlink parameter? Parse the request parameters from an env var? I don't know. Or we keep changing the parameter every time Twitch makes new changes and then publish a new release afterwards. Any volunteers for this work? Because I am already annoyed.

[gravyboat]

Fair points. I commented over in #3220.

[h1z1]

Twitch is not as smart as they think they are and I'll leave it at that. Take the target off your back. It could be argued streamlink is doing what it should and leave the implementation of ad handling to another project. Presumably the one embedding streamlink.

The eventual end goal of this will be full drm by them.

[superfreeman1989]

Any possible solution to the ads issue in the foreseeable future?

[fragande]

Is there any player out there that can handle the new ad segment filtering? I've tried latest MPV, MPC-HC and VLC and they all freeze and won't resume (or resume after a very long time (couple of minutes) but out of sync / stuttering badly) after the following line:

[stream.hls_filtered][info] Filtering out segments and pausing stream output

Streamlink is resuming Reloading playlist, Adding segment x to queue etc. as normal without errors, but none of the players above seem to be able to resume playback.

[bastimeyer]

What are you talking about? That is the whole point. This is embedded ad filtering, and not an embedded ads workaround, because there currently is none. Read the thread before posting.

If your player has any kind of synchronization issues after the playback resumes, then it's because of the stream discontinuity and the offset in stream timestamps. This can't be avoided, as Twitch simply hasn't served you the necessary data, either when watching an ad, or when filtering it out. Use a player that can handle this properly, like MPV for example. There is nothing that can be done here and I'm sick of repeating myself. If Twitch is embedding ads into the HLS stream, then you'll have to deal with it by either watching it, or filtering it out. And since your player is reading a progressive stream from Streamlink's output, watching ads is more prone to errors, since it's a different stream with different encoding parameters and will thus cause more issues with the player's demuxer and decoder than a simple timestamp offset when filtering out ads.

[fragande]

If it wasn't clear I do understand that it's filtering and not a workaround. I'm also aware that the issue isn't with streamlink but the player, but was simply wondering if there was a known working player with this solution. Neither latest MPV git build or stable version resumes playback (on my end anyway) after the filtering pause, it hard freezes and the process has to be killed manually. But if MPV is supposed to be able to handle it properly I'll look into it, thank you.

My apologies, clearly this wasn't the place to ask.

EDIT:

For anyone else having the same issue and reading this: disabling hardware decoding for MPV seems to have worked for me, but YRMW.

[bastimeyer]

Apparently, ads can be prevented again by adding player_type=frontpage to the access token request parameters, as shown here:
https://gist.githubusercontent.com/odensc/764900d841cbdd8aa400796001e189f1/raw/ads.js

That's the current diff if you want to try it out:

diff --git a/src/streamlink/plugins/twitch.py b/src/streamlink/plugins/twitch.py
index b534b003..d4bf53ee 100644
--- a/src/streamlink/plugins/twitch.py
+++ b/src/streamlink/plugins/twitch.py
@@ -297,7 +297,7 @@ class TwitchAPI(object):
                 validate.get("sig"),
                 validate.get("token")
             ))
-        ))
+        ), player_type="frontpage")

     def token(self, tokenstr):
         return parse_json(tokenstr, schema=validate.Schema(

It's the same situation as the platform=_ request parameter, and it might get changed quickly again by Twitch if they see it being (ab)used by lots of users, so I don't know if it makes sense hardcoding such a workaround again and having to remove it later on because it's broken and causes other issues.

[bastimeyer]

The header post has been updated again...

[myhowbou]

I would love to edit the hard code, but How do I access the U Block Orgin script to add that line of code?

[myhowbou]

Nvm I set my userResourcesLocation to https://gist.githubusercontent.com/odensc/764900d841cbdd8aa400796001e189f1/raw/ads.js
and that seems to work for now. Not sure sometimes twitch ads disappear for a bit and then come back. But if Twitch changes the parameter again how can I check to see what that new parameter is so I can add it?

[gravyboat]

@bastimeyer I think we're a bit blocked by the remaining Python 3 only stuff, otherwise I'd say we should just do another full release to try and head this off.

[myhowbou]

Nvm it didnt work lol

[myhowbou]

hmm well it only works half the time. Not sure why it's inconsistent.

[bastimeyer]

@myhowbou can you stop spamming this thread with unrelated posts? This is about Streamlink, not about any browser related ad blocking stuff. The request parameter is working fine here, and if it doesn't for you in another region or so, then be absolutely clear about that with an example log post. I'm tempted to mark this as offtopic, because I really don't like bloat in such megathreads, and everyone seeing nonsense notifications probably neither.

@gravyboat What do you mean? 2.0.0 is technically ready, as every mandatory change has been merged. Everything else listed in the checklist is optional. I don't see the need of rushing another release right now though just because of this change. The workaround is very simple, and I have provided a diff and also instructions for installing the latest development version, and that should be good for now until we're all confident with publishing the 2.0.0 release in a good shape.
Making another 1.x release would mean that we'd have to branch off from an earlier commit, which doesn't make sense considering that we explicitly chose not to do the py2 removal on a secondary branch (now the situation is in reverse). If that's a problem now, then we should have switched to a different git flow instead of having just one main branch.

[myhowbou]

@bastimeyer Hmm. Okay sorry. I thought this was a patch for the new twitch thing for Ublock.

[tarkov2213]

I did a pip upgrade and suddenly twitch displays those blank 10 sec moments for ads again? What happened?

[gravyboat]

@bastimeyer Sorry I should have been clear, I agree with you that the workaround is simple but it was simple last time too when people needed to install the nightly and we still had tons of comments. I do not want to make another 1.x release.

[bastimeyer]

@tarkov2213 You have either not installed the latest dev version, or you are using an older version in parallel by accident. As I've said in my previous comment, one should post a debug log (with --twitch-disable-ads set, so it'll include logs for ad segments) if they think that it's not working correctly, because non-meaningful statements like this without any information at all are a waste of time for every developer, regardless the project.

---

@gravyboat If there's going to be spam or more repetitive questions of people who haven't read anything before posting, then we're going to lock it. This should not make us rush a new release, because there are some changes left which I think should be done/finished first. I hope that's somewhat understandable.

So far, it's been rather tame though. But I also want to mention that due to being the Twitch GUI dev, I've been answering questions left and right each day on various other channels as well, and this gets very annoying over time, especially since I don't get paid for "playing unofficial Twitch support staff" while working on an unrelated project, so I understand that you want to push this and be done with it. But remember that the workaround can stop working at any moment, and if we rush things because of it, it might backfire.

Once again, the current solution for the enduser is very simple, and if some people are unwilling to read or don't understand, or just want to complain, then it's not my problem.

[S0und]

Offtopic:

Interesting, they've moved the HLS request from api.twitch.tv behind graphql

https://i.imgur.com/DteEeoC.png

[arrowgent]

i put this twitch.py patch and seems to work fine (linux)
no need to push a release

cat & mouse either way
btw, the "commercial break" screen is satisfactory...atleast its not an actual ad
thanks for your work and that "frontpage" discovery lol

[Darushin]

Just wanted to add some odd behavior I am observing. I am using the precompiled windows binaries for twitch (latest stable) and am noticing for channels I am subscribed to I am seeing ads. This happens regardless of tier level (I have tier 1-3 subs) and I verified the ads are not being ran by the streamers. I patched in the python above and I am seeing ads for streamers I am subscribed to still (before I was seeing the commercial break screen). I checked streamers that I am not subscribed to and saw ads randomly (5 saw ads 1 did not). If there is any logs you would like me to provide just let me know what you need and how to produce them and I will send them.

[bastimeyer]

@S0und I don't see anything about that on Twitch. They are still using the old private API for the access token. If they had rolled out those changes, then we'd seen tons of posts here, because it'd break the Twitch plugin. It is possible though that this change will come at some point.

---

@Darushin And how exactly do you think Streamlink knows about your subscriptions? Any kind of authentication has been removed from previous Streamlink versions (see https://github.com/streamlink/streamlink/issues/2680#issuecomment-557605851).
https://github.com/streamlink/streamlink/blob/master/CHANGELOG.md
Also, the last stable release is pretty much offtopic at this point since #3301 got merged.

[tarkov2213]

@bastimeyer well excuse me for assuming the package you have in the pip repository would contain important fixes like no ads. uninstalled it and pip installed straight from git+github and the problem is fixed.

[bastimeyer]

If this change was included in the latest stable release, then we wouldn't be discussing it here.

---

I've just noticed that the new ads prevention request parameter is not working on every Twitch channel, unfortunately. It's working fine on most, but on esl_dota2 for example, you'll still see preroll ads. Not sure why.

[bastimeyer]

In addition to player_type=frontpage, it seems that player_type=dashboard is another way of preventing embedded ads.

[S0und]

I had to switch from frontpage to embed yesterday. Seemingly worked for a lot of people, but someone reported ads today even with that.

https://gist.githubusercontent.com/odensc/764900d841cbdd8aa400796001e189f1/raw/ads.js

dashboard is also a variant, but i haven't tried it yet.

[pixeltris]

When mid-roll ads play a mini-player is created so that the viewer can still see the stream whilst ads play. Whilst it's limited to 360p it might be an idea to use this in cases where ads cannot be blocked by streamlink. There's some additional discussion on that here https://gist.github.com/odensc/764900d841cbdd8aa400796001e189f1#gistcomment-3510566

[f0rmatme]

https://gist.github.com/odensc/764900d841cbdd8aa400796001e189f1#gistcomment-3511057
works well for me so far

[dusty-dusty]

https://gist.github.com/odensc/764900d841cbdd8aa400796001e189f1#gistcomment-3511057
works well for me so far

how do I go about adding this twitch.py source code?

[gravyboat]

@dusty-dusty This has nothing to do with Streamlink, it's for uBlock. @f0rmatme why even mention this here? It's completely unrelated to Streamlink.

[Nil-00]

Today i started to run into embedded ads again from a German ip while using streamlink-1.7.0_46.g9384f3d

[RomeuG]

I confirm the post above.

When I ran a Twitch stream, I got an ad...

Streamlink version: streamlink 1.7.0+46.g9384f3d

[www2000]

I have the same problems with this whene i look in to the m3u file i found an intresting domine name on the fist line https://video-weaver..hls.ttvnw.net/trigger/ when i block this string to a webserver i get no ads in streamlink

[bastimeyer]

That URL is part of an EXT-X-DATERANGE tag, and it gets discarded by Streamlink, because it's only looking for those EXT-X-DATERANGE tags which indicate advertisement time ranges:
https://github.com/streamlink/streamlink/blob/bc78360c51c95e1052fc387c6e19c071f4a7b5bd/src/streamlink/plugins/twitch.py#L43-L45

If you take a look at the attributes, they look like this:

#EXT-X-DATERANGE:ID="trigger-%TIMESTAMP%",CLASS="twitch-trigger",START-DATE="2020-11-05T05:21:33.704Z",END-ON-NEXT=YES,X-TV-TWITCH-TRIGGER-URL="https://video-weaver.%LOCATION%.hls.ttvnw.net/trigger/%LONG_BASE64_STRING%"

and the URL is the value of the X-TV-TWITCH-TRIGGER-URL attribute, which means it's irrelevant to Streamlink.

The only HTTPS requests which are made by Streamlink are the Twitch API ones, the HLS master playlist, the HLS stream playlist, the regular HLS stream segments and the HLS prefetch segments. There is nothing to block here, and if you're taking about your web browser, then you're at the wrong place here.

[oldLucke]

I started getting pre and mid roll ADs a couple of day ago while using the fix in the 1.7 streamlink versions. Now I'm using 065cb82794cb80dd5d2abf3f11034ae40e2308df with a german IP and the twitch-disable-ads flag, I no longer get mid roll ADs, but I'm pretty sure I get pre roll ADs, cause the stream takes a much longer to open.

[cliocaev]

--twitch-disable-ads

what do i have to do with it? i use streamlink twitch gui

[bastimeyer]

@cliocaev
Nothing. Just check the checkbox in
Settings -> Streaming -> Advertisements

[nicolas-martin]

I have the same problems with this whene i look in to the m3u file i found an intresting domine name on the fist line https://video-weaver..hls.ttvnw.net/trigger/ when i block this string to a webserver i get no ads in streamlink

Sorry to side track the conversation here. I thought this would work, however I just get There was a network error. Please try again. (Error #2000) when loading a stream. I think it's because they serve their ads with the same servers as the streams. So you end up blocking both.

[tophercullen]

Up until last night, I'd been avoiding this issue by using version 1.2.0. For what ever reason, it is/was innately immune to all these ad changes, including pre-rolls. I don't see any mention of this behavior anywhere here.

1.2.0 not longer works because of 410 API Gone error for /accesss_token.json. But is it possible to replicate its functionally in a newer version?

[gravyboat]

@nicolas-martin There is nothing we can do about that, if you google it you can see this is a Twitch error due to a lack of a secure connection. I'm running the same options that the GUI enables via the CLI and I'm not having any problems. If it's still occurring for you try the CLI with streamlink -l debug --twitch-disable-ads twitch.tv/streamer for more output.

@tophercullen Nope. Old versions like that are dead and we can't replicate the functionality. The API they used for this was never publicly available, it just happened to be publicly accessible.

Let's stop derailing this thread, it's not a catch all for various Twitch related questions and I'm going to start deleting off-topic comments.

[tophercullen]

@gravyboat It wasn't dead. Thats my point. At a high level, I understand what happened with the API calls included in that version. Yes, certain functions long stopped working when those APIs were disabled. I never expected it to keep working forever. It was a stop gap. However, that doesn't change the fact that it is unaffected by ads.

Now if you're saying you acknowledge it was ad free all this time, know exactly why this is, and are certain that it cannot be replicated in the current API, I can accept that. As I said, I don't see anyone bringing up how legacy code is more functional in this regard.

But the way you have phrased it, it seems like offhandedly ignoring a potential workaround.

Myself, I briefly looked through the code and couldn't tell if the cause of API gone error is relevant or not to why it was still bypassing ads in 1.2.0. From what I saw, the failing API(s) are still in the current version.

[gravyboat]

@tophercullen Sorry if my comment wasn't clear. I specifically meant the old API access we cannot replicate. Twitch has had multiple APIs, both public and private, undocumented, but externally accessible. We had to remove the undocumented but externally accessible API back with the 1.3 release due to issues with user login. I had no idea the old release was ad free this entire time as I don't run old releases. If you were logged in via 1.2 and watching streamers you are subscribed to it would make sense as to why you weren't seeing ads. There were a lot of changes we had to make after 1.2 and I'm pretty sure we were using a problematic key at that time as well, it's hard to remember. Without more information about your specific setup I can't say, and even if something was working with that specific release we wouldn't be able to go back to how the Twitch plugin was setup for that release.

Edit: Yeah here is the PR: https://github.com/streamlink/streamlink/issues/2680 it was specifically to address the error you've been seeing. Bastimeyer has a good comment that sums everything up. Seems like they finally EoL'd the API instead of just saying it was gone for the URL since the message has changed.

[tophercullen]

@gravyboat Subscription didn't affect the ad status.

I understand completely not running old versions. Literally the only reason I found this out, soon after they started rolling out these changes, one computer was strangely unaffected. Further troubleshooting lead me to find it was the version it was running.

As far as replication, simply uninstalling current streamlink (and twitch-gui) and then installing streamlink 1.2.0 (twitch-gui 1.8.1) replicated the behavior on all computers.

Thanks for the background comment/issue number. Further troubleshooting shows this is indeed likely the root cause of why it worked. There are really two errors I'm seeing in 1.2.0:

• With Streamlinks clientid: 410 API Gone calling the private api (/api/access_token). Most certainly the same issue you linked above
• With Twitch's client-id: 404 client not found calling the public api (/kraken/users). Likely indicating its getting passed the private API call, but they changed something in their with their oauth scope or something, and it fails on the public calls.

In either case, 1.2.0 is not functional at a basic level. Switching to twitch's clientid in 1.7.0 does not replicate the ad exempt behavior seen in 1.2.0

This seems to indicate some magic with tokens returned by twitch's private API was the reason it was being exempted. As this is all on twitch's side, I would now agree with your assessment that the functional behavior cannot be replicated, even juggling clientids.

[Lord-Nightmare]

Offtopic:

Interesting, they've moved the HLS request from api.twitch.tv behind graphql

https://i.imgur.com/DteEeoC.png

This just got rolled out today, I think, and the api.twitch.tv/kraken/ portal now gives a "404 Client Error: Not Found for url: https://api.twitch.tv/kraken/user.json?as3=t&oauth_token=secretsymbols0123456789abcdefg" error. We should probably open a separate ticket for this if it breaks the plugin, but I don't think the latest top of git plugin depends on this api at all anymore, as it no longer allows defining the user oauth token since v1.2-ish?
(This is a bit off-topic, but if the oauth token is allowed to be user-defined, then ads do not display on channels a user is subscribed to, which avoids this whole cat-and-mouse game with twitch over blocking embedded ads and doesn't build bad blood between streamlink and twitch. I personally think that blocking embedded ads should not be part of the plugin at all, or we invite twitch's ire, they could even do something as draconic as enabling widevine/EME l3 globally for all streams (instead of just NFL and similar events as they do now) which would mean streamlink is completely useless for twitch from then on due to anti-circumvention laws)

[gravyboat]

@Lord-Nightmare This behavior was expected. We've known they were switching to GraphQL for a while, and the V5 API (Kraken) has been deprecated but somewhat functional for quite a while,

but I don't think the latest top of git plugin depends on this api at all anymore, as it no longer allows defining the user oauth token since v1.2-ish?

Correct.

Take a look at the comment I linked above from Bastimeyer. We used to let people use their tokens to log in so they wouldn't get ads, we can't do that any more because that endpoint is no longer exposed to us. Twitch has been fine for years with people not watching ads. We aren't doing anything that isn't possible to do in a browser. You don't have to block the ads, feel free to watch them if you want to. I'm not going to watch ads (especially for channels I subscribe to) simply because Twitch took away our ability to log in and I prefer to use a better and more efficient video player than what Twitch offers directly.

Let's please try to keep it very on-topic in this issue, it's already a nightmare notification wise.

[Lord-Nightmare]

Which issue (or other communication means) should I use, if I want to comment on the token thing? (because, as it turns out, the old kraken api did still work and even allowed ads to be avoided if subscribed, with some caveats, until about 12 hours ago. This required using the plugin version 1.1(!) with some fixes from 1.2 backported.)

[gravyboat]

@Lord-Nightmare For the old login issue from 1.2 with the user token login? The plugin is working fine on the latest Streamlink release, I'm running it right now. If you're on the latest version and getting an error then please open a new issue, don't bump that old one. If this is an issue on an old release there is no need to open an issue as we'll just say upgrade Streamlink then close the issue.

[bastimeyer]

I'm going to lock this thread now, as the past posts have all been off topic. There is nothing to discuss here in terms of embedded ads, because there's currently no workaround known. While writing this post, the master branch is still setting the player_type=frontpage access token request parameter. This doesn't make a difference anymore though and it should be removed again.

To explain it again, embedded ads don't occur all the time, and Twitch is in full control over this. If you're not seeing ads for a while and then start seeing them, it doesn't mean that something has changed or that Streamlink's Twitch plugin broke.

What's changed in Streamlink in the past releases in regards to embedded ads on Twitch are just simple request parameter workarounds while acquiring a stream access token. These parameters were used by certain players on Twitch's website to circumvent ads from being embedded into the streams. For example platform=_, which made Twitch think that the viewer was on a platform where no ads are being shown, in contrast to platform=web, or the most recent workaround with player_type=frontpage or player_type=dashboard, which made Twitch not embed ads into the streams, as they don't do this on their front page player or on a streamer's dashboard. This however kept working only for a couple of days/weeks until Twitch noticed that behavior among popular browser ad-blockers and applications like Streamlink, so they've changed it and/or ignored those parameters completely.

Applying workarounds in Streamlink is a game of cat and mouse and there's no solution to this. If Twitch wants to embed ads into the HLS streams, they can, and you'll have to accept that. The only way of not seeing ads is filtering them out. --twitch-disable-ads has seen multiple revisions over the past releases and the most recent one should properly cover all cases now until Twitch stops adding time annotations to the HLS segments or if they change their ad annotation scheme, which could happen, but that would be rather simple to fix.

If your player is having issues with embedded ads, like high-pitched sound or other weird things, you'll have to filter out ads, as the stream-discontinuity with the ads is causing this and the player can't handle differently encoded streams stitched together. The stream-discontinuity while filtering out ads is still a problem, but since it's only a gap in the same stream, this doesn't seem to cause any problems with most modern players / decoders.

---

@nicolas-martin
Anything "video-weaver" related is irrelevant, as I've already explained here
https://github.com/streamlink/streamlink/issues/3210#issuecomment-722152105
And There was a network error. Please try again. (Error #2000) is not a Streamlink error message, so you're using a different piece of software and that's off-topic here.

---

@tophercullen @Lord-Nightmare
Regarding older versions of Streamlink, this is completely irrelevant as well, as Twitch has made changes which prevent 3rd party client IDs (eg. Streamlink) or users authenticated with an OAuth token that's tied to a 3rd party client ID (eg. Streamlink) to acquire an access token for watching streams. This happened last November over a period of several weeks with a couple of days still working and then breaking again, multiple times, which made it difficult to make a decision fast. If it has been working for a while again in the past few months while using older versions of Streamlink, then good for you, but the latest version of Streamlink won't be able to revert this, as Twitch will simply break it again.

Read my post from last year here:
https://github.com/streamlink/streamlink/issues/2680#issuecomment-557605851

Streamlink is still using the same private API endpoint for that access token, and it's still working. The public kraken or helix APIs have nothing to do with that. Once Twitch changes the access token API endpoint from their old private API to their new private GQL-based API and shuts down the old private API, then it'll break the Twitch plugin, but so far this hasn't happened yet, otherwise we'd seen a ton of posts already.

While they were only sporadically using the GQL API for the access tokens in the past weeks, this seems to have changed now. I am not sure though if this justifies changing it here yet, as they could still be testing it and make further changes at any time, as it's not a public API endpoint and thus not deemed stable. Comparing the current GQL request with the one mentioned earlier in this thread, there have already been changes in the way the access token is requested. I'm going to open a new issue thread for that later today, so that the status can be checked there.

[bastimeyer]

Commenting here, so people who have subscribed the thread can see it.

There's a new workaround added by #3373 which again prevents ads from being embedded into the stream. I've updated the OP accordingly.