Is it possible to make access to Browser API from the User-Script code? At least for Chrome-like browser.
Sometimes this really is really not enough and I do not want to write everytime extra extensions for the sake of a couple of lines of code.
_Similar to:_
var filter = {urls: ["http://*/*", "https://*/*"], tabId: currentTabId };
var opt_extraInfoSpec = ['blocking'];
GM_webRequest.onBeforeRequest.addListener(
callback, filter, opt_extraInfoSpec);
The thing is, blocking code is synchronous so it must be inside the background page of Tampermonkey and you surely understand it would be insane to allow arbitrary userscripts to run there.
A viable solution might be declarative syntax e.g. provide an array of rules that specify what to do.
On the other hand, maybe Tampermonkey's sandbox can be applied to userscript code in the background page too...
@tophf I understand it and certainly agree with you what is required declarative syntax for rules. If I knew exactly how Tampermonkey works, I would try to implement it.
Agree that this option would give a strong advantage before similar extensions?
Ok, I've created a preview that shows how GM_webRequest and @webRequest could possibly work.
After downloading and drag and dropping it to the extension page you can install this script:
// ==UserScript==
// @name GM_webRequest testing
// @namespace http://tampermonkey.net/
// @version 0.1
// @description try to take over the world!
// @author You
// @match *://*/*
// @include http://new_static.url/*
// @include *//redirected.to/*
// @grant GM_webRequest
// @webRequest [{"selector":"*cancel.me/*","action":"cancel"},{"selector":{"include":"*","exclude":"http://exclude.me/*"},"action":{"redirect":"http://new_static.url"}},{"selector":{"match":"*://match.me/*"},"action":{"redirect":{"from":"([^:]+)://match.me/(.*)","to":"$1://redirected.to/$2"}}}]
// ==/UserScript==
var currently_active_webrequest_rule = JSON.stringify(GM_info.script.webRequest); // == @webRequst header from above
GM_webRequest([
{ selector: '*cancel.me/*', action: 'cancel' },
{ selector: { include: '*', exclude: 'http://exclude.me/*' }, action: { redirect: 'http://new_static.url' } },
{ selector: { match: '*://match.me/*' }, action: { redirect: { from: '([^:]+)://match.me/(.*)', to: '$1://redirected.to/$2' } } }
], function(info, message, details) {
console.log(info, message, details);
});
/*
Notes:
Action:
The final redirect: URL needs to be included into the scripts @match or @include header
Selector:
If just a string is given, it is interpreted as include: property.
include:, exclude: and match: properties are supported and can have an array or a string value - the syntax equals @include, @match and @exclude
@webRequest may have a human readable format in the future, but for now it's the stringified first argument of GM_webRequest and allow request
manipulation even if the script didn't run yet. The only downside atm is that you need to re-register the rules (which overrides all previous ones)
in order to get manipulation events. GM_webRequest(currently_active_webrequest_rule, function...)
*/
This all is not heavily tested so expect bugs here and there. :)
@derjanb
Thanks! This is going to help a lot!
If I understood right, the callback function will only receive information about the WebRequest once it is done and is not able to manipulate it.
In this case, I think being able to redirect to a generated string as the response content would be helpful.
I know it is not a good idea to let random function to run in privileged extension context, but there can be times when that would be handy. What is your opinion on this? Should I just create another extension to do this or we can have a special header for it?
The redirection and blocking that the design above allows can mostly be done by uBO, Adguard, or some other ad blockers, what they cannot do is run complex function to make dynamic decision.
Also, as a side thought, shouldn't it be named TM_webRequest? Since this functionality is obviously not in Greasemonkey.
@derjanb
Wow, thank you for such a prompt reply! :zap:
I try to play with https://ssl.gstatic.com/gb/images/v1_b3735dd8.png, but not one of the rules does not work for me.
What am I doing wrong?
// ==UserScript==
// @name GM_webRequest testing
// @namespace http://tampermonkey.net/
// @version 0.1
// @description try to take over the world!
// @author You
// @match *://*/*
// @include *gstatic.com/*
// @include https://google.com/*
// @include *//google.com/*
// @grant GM_webRequest
// @webRequest [{"selector":"*cancel.me/*","action":"cancel"},{"selector":{"include":"*","exclude":"http://exclude.me/*"},"action":{"redirect":"http://new_static.url"}},{"selector":{"match":"*://match.me/*"},"action":{"redirect":{"from":"([^:]+)://match.me/(.*)","to":"$1://redirected.to/$2"}}}]
// ==/UserScript==
console.log("GM_webRequest start");
var currently_active_webrequest_rule = JSON.stringify(GM_info.script.webRequest); // == @webRequst header from above
GM_webRequest([
//{ selector: '*gstatic.com/*', action: 'cancel' },
{ selector: { include: '*gstatic.com/*', exclude: 'http://exclude.me/*' }, action: { redirect: 'https://google.com/' } },
//{ selector: { match: '*://*.gstatic.com*' }, action: { redirect: { from: '([^:]+)://(.+)gstatic.com/(.*)', to: '$1://google.com/$2' } } }
], function(info, message, details) {
console.log(info, message, details); // This also does not work
});
Ah sorry, I forgot to add this information. Since intercepting requests makes things slower Tampermonkey only handles the following request types at the moment: 'sub_frame', 'script', 'xmlhttprequest' and 'websocket'.
All other are considered to be replaceable by a userscript, even after they were loaded, but that's discussable.
Hm, can be done something like?
/* here, any calculations of javasscript are not related to the page (setting vars, functions) */
GM_webRequest([
{ selector: '*gstatic.com/*', action: 'replace' },
], function(info, message, details) {
console.log(info, message, details);
details.url = details.url.replace('gstatic', 'gstatic-2');
return details;
});
input >>> https://gstatic.com/index.php?foo=bar...
output >>> https://gstatic-2.com/index.php?foo=bar...
We change url or body of the request and query data from the site.
With support for GET and POST requests.
Of course, according to the rules, the script code will be executed once for each request.
We change url or body of the request and query data from the site.
Atm only the URL via:
{ selector: { match: '*://match.me/*' }, action: { redirect: { from: '([^:]+)://match.me/(.*)', to: '$1://redirected.to/$2' } } }
With support for GET and POST requests.
Shouldn't make a difference.
details.url = details.url.replace('gstatic', 'gstatic-2');
This can't work because all communication is asynchronous. This means when the request is inspected by the background page, then there is no synchronous way to contact the userscript to make a decision. That's why all rules need to be defined when the requests happens.
@derjanb
You did not understand me. I meant, this scheme:
So will it work?
I don't understand webRequest's work well, but I'm afraid to give up hope for success 😃
Loading a script (string) from the storage(array) and executing into callback of webRequest
The background page has all extension permissions. It's not possible to run foreign JavaScript code there without leaking these permissions. Sorry. That's why a defined rule set is needed to tell the background page what needs to be done with a request.
@derjanb
Oh, thanks now everything is clear. But the parameters can still be read?
There can be such a parser?
var rules = [
{ event: 'request', key: 'url', action: replace: { from: '([^:]+)://match.me/(.*)', to: '$1://redirected.to/$2'}, },
]; // Execution in order
GM_webRequest(rules, function(info, message, events) {
console.log(info, message, events); // ex. events.request.url or events.response.body
}); // Reload background page and load rules with GM_webRequest into array rules at background page.
Help:
action)event)key)type, default - all types)method, default - all methods)UPDATE 03.06.17
I don't think only intercepting certain request types is a good idea. There might be instances where one wouldn't want some request to even be made in the first place. Also in general replacing things that already have been loaded by something that too has to first be loaded seems pretty slow as well.
How about being able to configure which requests types will be intercepted?
@LennyPenny You mean, XHR and Location types of query handlers, as well as the query types like POST, GET and others?
Give an example please, if it is not difficult for you 😃
I'm just responding to @derjanb and this https://github.com/Tampermonkey/tampermonkey/issues/397#issuecomment-300132157
Isn't it still the case that the soonest Tampermonkey can run scripts is asynchronously after document-start? If so, then wouldn't it be possible to miss resources (scripts, etc) loaded in the head? Don't get me wrong, I appreciate the effort, but it seems kind of silly to me to have an API that can 1. not dynamically handle requests, 2. can only catch certain resource types, and 3. may or may not actually work.
@Couchy
Isn't it still the case that the soonest Tampermonkey can run scripts is asynchronously after document-start?
That's true. (Even though it's much less likely if the experimental "Inject Mode" is set to "Fast".) That's why you can use @webRequest to allow Tampermonkey to process your rules even though the script doesn't run yet.
can only catch certain resource types
For now. As I said, that's discussable and may be subject to change.
@LennyPenny
How about being able to configure which requests types will be intercepted?
That's doable, but adds a lot of complexity. So I'll implment it if really needed by will try to go without as long as possible.
I was able to use this to cancel a request for a single static javascript file, however refreshing the page repeatedly, both with CTRL+R and also CTRL+SHIFT+R (clear cache), debug console sometimes doesn't show log output from: console.log(info, message, details);
In Chromium, I see the red text line for "GET file.js net::ERR_BLOCKED_BY_CLIENT" but sometimes the console.log text is missing. Why is this?
@jasonkhanlar
Because the script was not injected/executed yet, but Tampermonkey blocked the request based on the @webRequest header. See #211 regarding document-start reliability.
@derjanb i need block script loading only from "script" type request, allow loading from XHR. Is not possible to do this?
@AugustoResende You can initially block all by using @webRequest and once your script runs reconfigure the rules via GM_webRequest to allow it being downloaded.
@derjanb webRequest is not human readable. You can easily fix this allowing multiple @webRequest, like multiple @include
```
// @include ://example1.com/
// @include ://example2.com/
// @include ://example3.com/
// @webRequest [{"selector":"://example1.com/","action":"cancel"}]
// @webRequest [{"selector":"://example2.com/","action":"cancel"}]
// @webRequest [{"selector":"://example3.com/","action":"cancel"}]
And... A suggestion:
webRequest Header/Body content intercept and editor
chrome.webRequest.onBeforeSendHeaders.addListener(
function(details) {
injectHeader(
'Referer',
'https://www.google.com.br/',
details.requestHeaders
);
return {requestHeaders: details.requestHeaders};
},
{
urls: [
"*://www.example.com/*"
],
types: ["main_frame"]
},
["blocking", "requestHeaders"]
);
GM_xmlhttpRequest already have header request manipulation, is like that:
GM_xmlhttpRequest({
method: 'GET',
url: window.location.href,
headers: {
'Referer': 'https://www.google.com.br/'
},
anonymous: true,
onload: function(response) {
}
});
like xhook, but xhook only works with XHR and fetch. And... include "main_frame", is not replaceable by a userscript
@derjanb BUG:
webRequest not blocking script if it is in the browser "memory" cache :(
Chrome
@derjanb ?
You can easily fix this allowing multiple @webrequest, like multiple @include
Yes, this would be one way to make it better readable.
webRequest Header/Body content intercept and editor
This is AFAIK only possible at Firefox:
https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/webRequest/StreamFilter
https://bugs.chromium.org/p/chromium/issues/detail?id=104058
and therefore not supported by Tampermonkey.
For the moment you can only cancel requests, download the resource via GM_xhr, modify it and then forward it to the page.
webRequest not blocking script if it is in the browser "memory" cache :(
I know. All you get is that it's loaded from cache at the onResponseStarted listener:
https://developer.chrome.com/extensions/webRequest#property-details-fromCache
but it's not possible to block or modify things at this point.
header intercept working on chrome:
```
chrome.webRequest.onBeforeSendHeaders.addListener(
function(details) {
injectHeader(
'Referer',
'https://www.google.com.br/',
details.requestHeaders
);
return {requestHeaders: details.requestHeaders};
},
{
urls: [
"://www.example.com/"
],
types: ["main_frame"]
},
["blocking", "requestHeaders"]
);
block url without this bug (cache):
```
chrome.webRequest.onBeforeRequest.addListener(
function(details) {
return {cancel: true};
},
{
urls: [
"://example.com/"
],
types: ["script"]
},
["blocking"]
);
@derjanb Doesn't work with the userscript below
// ==UserScript==
// @name NoPaywall
// @namespace NoPaywall
// @version 1.0
// @description Bypass all paywalls!
// @author You
// @match *://*/*
// @grant GM_webRequest
// @run-at document-start
// ==/UserScript==
const websites = [
"*://*.adelaidenow.com.au/*",
"*://*.baltimoresun.com/*",
"*://*.barrons.com/*",
"*://*.chicagobusiness.com/*",
"*://*.chicagotribune.com/*",
"*://*.chip.de/*",
"*://*.clarin.com/*",
"*://*.courant.com/*",
"*://*.couriermail.com.au/*",
"*://*.cricketarchive.com/*",
"*://*.dailypress.com/*",
"*://*.dailytelegraph.com.au/*",
"*://*.durangoherald.com/*",
"*://*.economist.com/*",
"*://*.fd.nl/*",
"*://*.forbes.com/*",
"*://*.ft.com/*",
"*://*.geelongadvertiser.com.au/*",
"*://*.goldcoastbulletin.com.au/*",
"*://*.haaretz.co.il/*",
"*://*.haaretz.com/*",
"*://*.hbr.org/*",
"*://*.heraldsun.com.au/*",
"*://*.inc.com/*",
"*://*.independent.co.uk/*",
"*://*.investingdaily.com/*",
"*://*.irishtimes.com/*",
"*://*.kansas.com/*",
"*://*.kansascity.com/*",
"*://*.latimes.com/*",
"*://*.lanacion.com.ar/*",
"*://*.letemps.ch/*",
"*://*.mcall.com/*",
"*://*.medscape.com/*",
"*://*.medium.com/*",
"*://*.nationalpost.com/*",
"*://*.newsweek.com/*",
"*://*.newyorker.com/*",
"*://*.nikkei.com/*",
"*://*.nrc.nl/*",
"*://*.nyt.com/*",
"*://*.nytimes.com/*",
"*://*.ocregister.com/*",
"*://*.orlandosentinel.com/*",
"*://*.quora.com/*",
"*://*.scmp.com/*",
"*://*.seattletimes.com/*",
"*://*.slashdot.org/*",
"*://*.smh.com.au/*",
"*://*.sun-sentinel.com/*",
"*://*.technologyreview.com/*",
"*://*.theage.com.au/*",
"*://*.theaustralian.com.au/*",
"*://*.thenation.com/*",
"*://*.thestreet.com/*",
"*://*.thesundaytimes.co.uk/*",
"*://*.thetimes.co.uk/*",
"*://*.washingtonpost.com/*",
"*://*.wsj.com/*",
"*://*.wsj.net/*"
];
const cookies = ([
// theaustralian.com.au
'open_token=anonymous',
'sr=true',
'FreedomCookie=true',
'n_regis=123456789'
]).join('; ').trim();
// from https://support.google.com/webmasters/answer/1061943
const UA_Desktop = "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)";
const UA_Mobile = "Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible ; Googlebot/2.1 ; +http://www.google.com/bot.html)";
function evadePaywalls(details) {
const shouldDropUA = !details.url.includes("medium.com");
var useMobileUA = false;
var reqHeaders = details.requestHeaders.filter(function(header) {
// drop cookies, referer and UA
switch(header.name) {
case "User-Agent":
useMobileUA = header.value.toLowerCase().includes("mobile");
return !shouldDropUA;
case "Cookie":
case "Referer":
return false;
default:
return true;
}
});
// Add the spoofed ones back
reqHeaders.push({
"name": "Referer",
"value": "https://www.google.com/"
});
if (shouldDropUA) {
reqHeaders.push({
"name": "User-Agent",
"value": useMobileUA ? UA_Mobile : UA_Desktop
});
}
reqHeaders.push({
"name": "Cookie",
"value": cookies
});
reqHeaders.push({
"name": "X-Forwarded-For",
"value": "66.249.66.1"
});
return {requestHeaders: reqHeaders};
}
function blockCookies(details) {
var responseHeaders = details.responseHeaders.filter(function(header) {
if (header.name === "Cookie") {
return false;
}
return true;
});
return {responseHeaders: responseHeaders};
}
chrome.webRequest.onBeforeSendHeaders.addListener(evadePaywalls, {
urls: [...websites],
types: ["main_frame", "script"],
}, ["requestHeaders", "blocking"]);
chrome.webRequest.onHeadersReceived.addListener(blockCookies, {
urls: [...websites],
types: ["main_frame", "script"],
}, ["responseHeaders", "blocking"]);
Console reports : Cannot read property 'onBeforeSendHeaders' of undefined
@rjkpa
Console reports : Cannot read property 'onBeforeSendHeaders' of undefined
Please check this post in order to learn how this API is designed and what is possible and what is not: https://github.com/Tampermonkey/tampermonkey/issues/397#issuecomment-299949037
The script you've posted will never work as userscript.
Is it possible to do x-frame bypassing using this?
So it is impossible to modify response body with GM_webRequest?
So it is impossible to modify response body with GM_webRequest?
At the moment: yes.
How to use this API?
Editor said “’GM_webRequest‘ is not defined.”
Is @webrequest available on Safari?
This feature is lacking documentation on https://tampermonkey.net/documentation.php
Hey @derjanb, is there any resource where I can find information about my last question? I need to know if @webRequest is available on Safari. Thanks!
@rodorgas the entire WebRequest API is not available on Safari, so no.
This feature is lacking documentation on https://tampermonkey.net/documentation.php
@rodorgas This feature is still in alpha state and this comment is the only documentation. ;)
@nakibmark is right, Safari does not support a web request intercepting API like WebRequest.
Syntax:
action: { redirect: { from: ..., to: ..., } }
doesn't work at all
redirect error {description: "unable to determine the redirect URL", rule: {…}, url: "..."}
@kghost Thanks. Will be fixed at the next beta version.
The expected changes to the WebRequest API in Chrome might break this feature :/ The dev team is asking for feedback here.
The expected changes to the WebRequest API in Chrome might break this feature :/ The dev team is asking for feedback here.
Unfortunately this is only a very minor issue: https://groups.google.com/a/chromium.org/forum/#!topic/chromium-extensions/hQeJzPbG-js
@webRequest now can be set multiple times. This means the example script header can also be written like this:
// ==UserScript==
// @name GM_webRequest testing
// @namespace http://tampermonkey.net/
// @version 0.1
// @description try to take over the world!
// @author You
// @match *://*/*
// @include http://new_static.url/*
// @include *//redirected.to/*
// @grant GM_webRequest
// @webRequest {"selector":"*cancel.me/*","action":"cancel"}
// @webRequest {"selector":{"include":"*","exclude":"http://exclude.me/*"},"action":{"redirect":"http://new_static.url"}}
// @webRequest {"selector":{"match":"*://match.me/*"},"action":{"redirect":{"from":"([^:]+)://match.me/(.*)","to":"$1://redirected.to/$2"}}}
// ==/UserScript==
@derjanb Does multi line @webRequest work for latest stable? or beta only?
Edit: It seems multi line only works in beta not stable 4.8.41
Edit2: Any plan to make this to stable?
@derjanb does multi line @webRequest work for latest stable? or beta only?
Tampermonkey 4.9+ is required to use it. This means current stable version (4.8.41) doesn't support it (yet).
The download link for the "preview" returns a CRX with invalid header. Could we get a new version? Is it possible to use GM_webRequest without installing extra exstensions?
@SilverBeamx I can use GM_webRequest in stable 4.8.41. see https://github.com/Tampermonkey/tampermonkey/issues/397#issuecomment-299949037 for more detail. Only multi line definition here https://github.com/Tampermonkey/tampermonkey/issues/397#issuecomment-471446386 requires 4.9 beta (can be download on google store)
@jc3213 I see. I indeed can get it to work, but only sometimes. I don't know what i'm doing wrong. Still, i have a couple questions:
var open = window.XMLHttpRequest.prototype.open;
var send = window.XMLHttpRequest.prototype.send;
function openReplacement(method, url, async, user, password) {
this._url = url;
return open.apply(this, arguments);
}
function sendReplacement(data) {
if(this.onreadystatechange) {
this._onreadystatechange = this.onreadystatechange;
}
// PLACE HERE YOUR CODE WHEN REQUEST IS SENT
this.onreadystatechange = onReadyStateChangeReplacement;
return send.apply(this, arguments);
}
function onReadyStateChangeReplacement() {
// PLACE HERE YOUR CODE FOR READYSTATECHANGE
if(this._onreadystatechange) {
return this._onreadystatechange.apply(this, arguments);
}
}
window.XMLHttpRequest.prototype.open = openReplacement;
window.XMLHttpRequest.prototype.send = sendReplacement;
hey all,
hope you are doing fine.
Is there maybe somebody who found already a way to bypass CSP with Tampermonkey?
I have tried to find something but I dont have any success.
I read a lot about webRequest but I dont know how it works because I am not a developer and more try and error.
Thanks,
Sebastian
I am wondering whether it's possible to take action based on the contents of a POST request. Unconditional actions like cancelling work, but I want to analyze the request itself and log it at least.
@youk, as was mentioned above the communication between userscript and background page is asynchronous so there is no way to dynamically analyze the request and then take different actions. You only can define static rules and listeners for rule triggering.
@derjanb, looks like GM_webRequest returns object with abort method that does nothing.
action property isn't provided, error on the background page happens;action is neither cancel nor redirect ({}), no errors but the listener is never called;redirect has both static and dynamic redirections then only static redirect happens. Maybe it'd be better at first try dynamic one and if it fails then redirection to a static link;details.redirect_url maybe it should be in camel case details.redirectUrl?// ==UserScript==
// @name Test GM.webRequest
// @namespace http://tampermonkey.net/
// @version 0.1
// @description try to take over the world!
// @author 7nik
// @match https://httpbin.org/*
// @grant GM.webRequest
// @connect httpbin.org
// ==/UserScript==
(async () => {
const rules = [
// error on the background page
{ selector: "https://httpbin.org/anything?key=1" },
// works but never calls the listener
{ selector: "https://httpbin.org/anything?key=2", action: {} },
// always redirects to the static URL
{ selector: "https://httpbin.org/anything?key=3", action: {
redirect: {
from: "https://httpbin.org/anything\\?(.*)",
to: "https://httpbin.org/anything/redirect?mode=dynamic&$1",
url: "https://httpbin.org/anything/redirect?mode=static",
},
} },
];
console.log("start");
for (let i = 0; i < rules.length; i++) {
try {
let [action, message, details] = await new Promise((resolve, reject) => {
GM.webRequest([rules[i]], (...args) => resolve(args))
.then(() => fetch(`https://httpbin.org/anything?key=${i+1}`))
.finally(() => setTimeout(reject, 500, "timeout"));
});
console.log(i+1, details.description || message, details.redirect_url || details.url);
} catch (ex) {
console.log(i+1, ex, rules[i]);
}
}
console.log("done");
})();
I've stumbled upon this ticket while figuring out how to intercept request headers (specifically, auth token) in a user script. I tried a simple match-and-forward rule with GM.webRequest but the listener did not seem to receive request headers; apparently, they are not exposed.
For posterity, the workaround I ended up with relies on overriding window.XMLHttpRequest.prototype.setRequestHeader. The downside of this approach is that it is hard to map headers to request URLs, which can be an issue if the page makes requests with distinct tokens. The script lives here.
@rukletsov, the GM.webRequest doesn't provide any info about the body or headers of the request. It allows only redirect and cancel requests and get notified about it.
You can use Map/WeakMap to store XHRs and their URLs on open and then retrieve the link in the setRequsetHeader via XHR aka this.
Most helpful comment
Ok, I've created a preview that shows how GM_webRequest and @webRequest could possibly work.
After downloading and drag and dropping it to the extension page you can install this script:
This all is not heavily tested so expect bugs here and there. :)