I am getting the same thing, just started happening an hour or so ago. Other methods are working.

ryantbrown on 7 Apr 2018

Same here, work only if login with credentials

rhcarlosweb on 7 Apr 2018

Same here.

fattony80 on 7 Apr 2018

+1 Same here

gsound on 7 Apr 2018

Hey, according last news about private data leak, Insta to close some anonymous api.

eastygh on 7 Apr 2018

oh, sorry. this news https://www.instagram.com/developer/changelog/ clarified the situation?

eastygh on 7 Apr 2018

+1 . But If use login with credentials, sometimes I get this exception 'InstagramScraper\Exception\InstagramAuthException' with message 'Something went wrong. Please report issue.'

Epimetheus84 on 7 Apr 2018

Fatal error: Uncaught exception 'InstagramScraper\Exception\InstagramException' with message 'Response code is 403. Body: message => forbidden; status => fail; Something went wrong. Please report issue.' in /InstagramScraper/Instagram.php:315 Stack trace: #0 /InstagramScraper/Instagram.php(272): InstagramScraper\Instagram->getMediasByUserId(4919194635, 3, '') #1

zaivst on 8 Apr 2018

https://www.instagram.com/vasiliizaikovskii/?__a=1 this works!

zaivst on 8 Apr 2018

@zaivst I tried to get the next page of medias but without success 😕

I think query_id, or query_hash is one of thinks to make this work again, but i'm not a developer just a layman 😔

rhcarlosweb on 8 Apr 2018

I made a solution for this one, but in python using automated browser to retrieve cookies and new URL.
Really don't know how PHP implementation would look like, but this are the steps to do:

Get cookies with automated browser
Make request with this cookies, and new URL:
'https://www.instagram.com/graphql/query/?query_hash=42323d64886122307be10013ad2dcc44&variables={"id":"<user_id>","first":<items_to_retrieve>,"after":"<end_cursor>"}'
where is either blank or end_cursor from previous request
- instagram web uses 12. I tested successfully with 20.

_Disclaimer 1: no authorization needed!_
_Disclaimer 2: actually I reused the same cookies several times and it worked. The expiry seams to be set in one year. But I don't know if Instagram will catch the usage of cookies from many different clients if hardcoded to this scapper!_

Python implementation:

# ! Error handling is omitted for clarity
import requests
from selenium import webdriver

media_url = 'https://www.instagram.com/graphql/query/?query_hash=42323d64886122307be10013ad2dcc44&variables={"id":"%s","first":20,"after":"%s"}'
browser = webdriver.Chrome()

# first get https://instagram.com to obtain cookies
browser.get('https://instagram.com')
browser_cookies = browser.get_cookies()

# set a session with cookies
session = requests.Session()
for cookie in browser_cookies:
    c = {cookie['name']: cookie['value']}
    session.cookies.update(c)

# get response as JSON
# > using id '25025320' - profile of Instagram for this example
response = session.get(media_url % ('25025320', ''), verify=False).json()

myrs on 8 Apr 2018

👍2

https://www.instagram.com/vasiliizaikovskii/?__a=1 this works!

@zaivst this one works and always worked, but, unfortunately, only for first 12 records of the profile. It worked before to retrieve next chunk of media, adding max_id parameter, but now it is just ignored.

myrs on 8 Apr 2018

this one works and always worked

All other features works too until 4 apr :)

zaivst on 8 Apr 2018

Same problem here.

luanrox on 8 Apr 2018

@raiym @rhcarlosweb @gthedev hi! I really don't know PHP to help with this one, but maybe the quick hotfix would be:

change the actual URL in https://github.com/postaddictme/instagram-php-scraper/blob/master/src/InstagramScraper/Endpoints.php:
ACCOUNT_MEDIAS = 'https://instagram.com/graphql/query/?query_id=17888483320059182&id={user_id}&first={count}&after={max_id}';
to:
ACCOUNT_MEDIAS = https://www.instagram.com/graphql/query/?query_hash=42323d64886122307be10013ad2dcc44&variables={"id":"{user_id}","first":{count},"after":"{max_id}"}
Send the following cookies with request. I just checked - the cookies I retrieved yesterday still work (one day now) and from different clients, without need to get the new ones before each request. So automated browser part _might_ be omitted for now:

[
    {
        "domain": "www.instagram.com",
        "httpOnly": false,
        "name": "rur",
        "path": "/",
        "secure": false,
        "value": "PRN"
    },
    {
        "domain": "www.instagram.com",
        "httpOnly": false,
        "name": "ig_vw",
        "path": "/",
        "secure": false,
        "value": "1038"
    },
    {
        "domain": "www.instagram.com",
        "expiry": 1554672942.248612,
        "httpOnly": false,
        "name": "csrftoken",
        "path": "/",
        "secure": true,
        "value": "ObRXje2ByOUmAnxqPaoFsD0CHvBEK8dQ"
    },
    {
        "domain": "www.instagram.com",
        "expiry": 2153943342.248646,
        "httpOnly": false,
        "name": "mid",
        "path": "/",
        "secure": false,
        "value": "WsqLMgALAAFkkaMz9rbL568BCU5N"
    },
    {
        "domain": "www.instagram.com",
        "httpOnly": false,
        "name": "ig_vh",
        "path": "/",
        "secure": false,
        "value": "532"
    },
    {
        "domain": "www.instagram.com",
        "httpOnly": false,
        "name": "ig_pr",
        "path": "/",
        "secure": false,
        "value": "2.5"
    }
]

Maybe this is not final solution, but at least media queries will work (_for some time_ 😅)

myrs on 8 Apr 2018

👍2

@myrs thanks bro! i tryed the same here, and works! :D

luanrox on 9 Apr 2018

@myrs when i try this in browser - it works, and when I try to do this changes in scraper it returns 403 status

zaivst on 9 Apr 2018

@myrs how to edit the cookies we send? I didnt understand the step-2. Thanks for your time

dionvogliqi on 9 Apr 2018

@dionii1 As i said, unfortunately, I'm not really familiar with PHP =S
As far as I understand, you should send cookies in header for this request. Maybe this piece of code from https://github.com/postaddictme/instagram-php-scraper/blob/master/src/InstagramScraper/Instagram.php could be relevant to make necessary changes:

$mid = $cookies['mid'];
$csrfToken = $cookies['csrftoken'];
$headers = ['cookie' => "csrftoken=$csrfToken; mid=$mid;",
    'referer' => Endpoints::BASE_URL . '/',
    'x-csrftoken' => $csrfToken,
];
$response = Request::post(Endpoints::LOGIN_URL, $headers,
    ['username' => $this->sessionUsername, 'password' => $this->sessionPassword]);

myrs on 9 Apr 2018

@zaivst what changes have you made?

myrs on 9 Apr 2018

@myrs I changed ACCOUNT_MEDIAS constant in Endpoints.php and Request::get() function in getMediasByUserId() returns 403 status. But if I try to use string which is returns by Endpoints::getAccountMediasJsonLink($id, $maxId) in browser - it returns correct response.

zaivst on 9 Apr 2018

@myrs this

{
    "domain": "www.instagram.com",
    "httpOnly": false,
    "name": "ig_pr",
    "path": "/",
    "secure": false,
    "value": "2.5"
}

You know what is it? With this i maded work recent medias without credentials

rhcarlosweb on 9 Apr 2018

@zaivst as I understand, this is because cookies are not set.
@zaivst @dionii1 could you check this one? One more time, I'm not familiar with PHP and structure of this project, buy I imagine here https://github.com/postaddictme/instagram-php-scraper/blob/master/src/InstagramScraper/Instagram.php#L209 (line 209) is where cookies are set and this part should be changed to the following logic:
if there is no session, use default cookies (actually, this is what I'm doing in python):

private function generateHeaders($session)
    {
        $headers = []
        if ($session) {
            $cookies = '';
            foreach ($session as $key => $value) {
                $cookies .= "$key=$value; ";
            }
            $headers = [
                'cookie' => $cookies,
                'referer' => Endpoints::BASE_URL . '/',
                'x-csrftoken' => $session['csrftoken'],
            ];
        } else {
            $rur = "PRN";
            $ig_vw = "1038"
            $csrftoken = "ObRXje2ByOUmAnxqPaoFsD0CHvBEK8dQ"
            $mid = "WsqLMgALAAFkkaMz9rbL568BCU5N"
            $ig_vh = "532"
            $ig_pr = "2.5"

            $headers = ['cookie' => "rur=$rur; ig_vw=$ig_vw; csrftoken=$csrftoken; mid=$mid; ig_vh=$ig_vh; ig_pr=$ig_pr;",
                'referer' => Endpoints::BASE_URL . '/',
                'x-csrftoken' => $csrftoken,
            ];
        }
        if ($this->getUserAgent()) {
            $headers['user-agent'] = $this->getUserAgent();
        }
        return $headers;
    }

Also, this line should be changed to:
https://github.com/postaddictme/instagram-php-scraper/blob/master/src/InstagramScraper/Instagram.php#L313
$response = Request::get(Endpoints::getAccountMediasJsonLink($id, $count, $maxId), $this->generateHeaders($this->userSession));
to include count parameter.

Disclaimer: none of this code was run and supposed to work fine 😅

myrs on 9 Apr 2018

@rhcarlosweb wow! Only this cookie is really needed? Sincerely I don't know what this are. Just was sending all cookies I get.
And so, you have a working version?

myrs on 9 Apr 2018

328 Need to test... But this solution works for me 😌

rhcarlosweb on 9 Apr 2018

👍4

@rhcarlosweb wow!! This actually worked for me!! Seams to be some magic, but just using this cookie resolved a whole deal!!
So, there is no need to switch to URL I provided before, but using only this cookies with new URL works fine too.

myrs on 9 Apr 2018

Nice @myrs and thanks for the cookie value, because i don't know how to get this value haha

rhcarlosweb on 9 Apr 2018

@myrs Strange because i have test with a blank value of $this->userSession['ig_pr'] = ""; and works too..
😕 confused haha

rhcarlosweb on 9 Apr 2018

@ryantbrown 👌 So, just waiting for this pull request to be approved

myrs on 9 Apr 2018

Strange because i have test with a blank value of $this->userSession['ig_pr'] = ""; and works too..

M.. maybe Instagram is just waiting this cookie name, no matter the value. Because setting it to some random value, e.g. 42 works fine too!

But yes, when ig_pr not present, returns 403 code.

Nice _user private data protection_ system, anyway 😅

myrs on 9 Apr 2018

😄1

@myrs ya hopefully @rhcarlosweb's PR gets approved quickly. The only thing that is unclear is whether the csrftoken should be set if missing. It seems to work with an empty string but the code you provided suggests it could be set using to ObRXje2ByOUmAnxqPaoFsD0CHvBEK8dQ as well.

Either way I think its good to go.

ryantbrown on 9 Apr 2018

@myrs thanks. it's worked for me.

ghost on 9 Apr 2018

You guys can get the "csrf_token" on login page HTML with REGEX.

https://instagram.com/accounts/login/

I'm getting the csrf_token from this page and using on cookies.

Next on: "{"activity_counts":null,"config":{"csrf_token":"SDYkHgQQsFkO1bCPKDWh35HEaoSOV7rM","viewer":null},""

I can't send the code because i'm using C#

luanrox on 9 Apr 2018

@luanrox i think if added

$cookies = static::parseCookies($response->headers['Set-Cookie']);
$this->userSession['csrftoken'] = $cookies['csrftoken'];

like the others requests getMediasByTag it's gonna work

rhcarlosweb on 9 Apr 2018

Today the error back again =\ someone with the same issue?

rhcarlosweb on 10 Apr 2018

I have tested again with new userSession cookie and $this->userSession['ig_pr'] not work anymore, now needs to use a $this->userSession['sessionid'] and i think is only generate with auth =p

rhcarlosweb on 10 Apr 2018

Shit :(
Stop works here too...

luanrox on 10 Apr 2018

Haven't cracked it yet, but here's what I know so far.

When logged in, cookie 'sessionid' is required.
When not logged in, a new header is required: 'x-instagram-gis: 4b698621d4a2ef5913f90aec25475d04'

I don't know how x-instagram-gis is calculated, but it appears to be a encryption of the parameters. The x-instagram-gis is recalculated for each pagination request, but is the same for the same request. It looks to be some crypto hashing function definitely involving the variables parameter and who knows what else.

I've tried to look at the obfuscated js to see what kind of encryption they are doing, but I haven't found it. Maybe someone can help take a look as well.

Perhaps they are using the same encryption technique that changed query_id to query_hash? Does anyone know how that's encrypted? It is a 32 char output, so i tried to play with md5, but no go.

andrewyoo on 10 Apr 2018

@andrewyoo x-instagram-gis calculated with csrf_token, rhx_gis, window.navigator.userAgent and variables from API call. Here is my refactored hashing function:
function gishash(n,r,t){function e(n,r){var t=(65535&n)+(65535&r);return(n>>16)+(r>>16)+(t>>16)<<16|65535&t}function o(n,r,t,o,u,c){return e((f=e(e(r,n),e(o,c)))<<(a=u)|f>>>32-a,t);var f,a}function u(n,r,t,e,u,c,f){return o(r&t|~r&e,n,r,u,c,f)}function c(n,r,t,e,u,c,f){return o(r&e|t&~e,n,r,u,c,f)}function f(n,r,t,e,u,c,f){return o(r^t^e,n,r,u,c,f)}function a(n,r,t,e,u,c,f){return o(t^(r|~e),n,r,u,c,f)}function i(n,r){var t,o,i,h,g;n[r>>5]|=128<<r%32,n[14+(r+64>>>9<<4)]=r;var v=1732584193,d=-271733879,l=-1732584194,A=271733878;for(t=0;t<n.length;t+=16)o=v,i=d,h=l,g=A,d=a(d=a(d=a(d=a(d=f(d=f(d=f(d=f(d=c(d=c(d=c(d=c(d=u(d=u(d=u(d=u(d,l=u(l,A=u(A,v=u(v,d,l,A,n[t],7,-680876936),d,l,n[t+1],12,-389564586),v,d,n[t+2],17,606105819),A,v,n[t+3],22,-1044525330),l=u(l,A=u(A,v=u(v,d,l,A,n[t+4],7,-176418897),d,l,n[t+5],12,1200080426),v,d,n[t+6],17,-1473231341),A,v,n[t+7],22,-45705983),l=u(l,A=u(A,v=u(v,d,l,A,n[t+8],7,1770035416),d,l,n[t+9],12,-1958414417),v,d,n[t+10],17,-42063),A,v,n[t+11],22,-1990404162),l=u(l,A=u(A,v=u(v,d,l,A,n[t+12],7,1804603682),d,l,n[t+13],12,-40341101),v,d,n[t+14],17,-1502002290),A,v,n[t+15],22,1236535329),l=c(l,A=c(A,v=c(v,d,l,A,n[t+1],5,-165796510),d,l,n[t+6],9,-1069501632),v,d,n[t+11],14,643717713),A,v,n[t],20,-373897302),l=c(l,A=c(A,v=c(v,d,l,A,n[t+5],5,-701558691),d,l,n[t+10],9,38016083),v,d,n[t+15],14,-660478335),A,v,n[t+4],20,-405537848),l=c(l,A=c(A,v=c(v,d,l,A,n[t+9],5,568446438),d,l,n[t+14],9,-1019803690),v,d,n[t+3],14,-187363961),A,v,n[t+8],20,1163531501),l=c(l,A=c(A,v=c(v,d,l,A,n[t+13],5,-1444681467),d,l,n[t+2],9,-51403784),v,d,n[t+7],14,1735328473),A,v,n[t+12],20,-1926607734),l=f(l,A=f(A,v=f(v,d,l,A,n[t+5],4,-378558),d,l,n[t+8],11,-2022574463),v,d,n[t+11],16,1839030562),A,v,n[t+14],23,-35309556),l=f(l,A=f(A,v=f(v,d,l,A,n[t+1],4,-1530992060),d,l,n[t+4],11,1272893353),v,d,n[t+7],16,-155497632),A,v,n[t+10],23,-1094730640),l=f(l,A=f(A,v=f(v,d,l,A,n[t+13],4,681279174),d,l,n[t],11,-358537222),v,d,n[t+3],16,-722521979),A,v,n[t+6],23,76029189),l=f(l,A=f(A,v=f(v,d,l,A,n[t+9],4,-640364487),d,l,n[t+12],11,-421815835),v,d,n[t+15],16,530742520),A,v,n[t+2],23,-995338651),l=a(l,A=a(A,v=a(v,d,l,A,n[t],6,-198630844),d,l,n[t+7],10,1126891415),v,d,n[t+14],15,-1416354905),A,v,n[t+5],21,-57434055),l=a(l,A=a(A,v=a(v,d,l,A,n[t+12],6,1700485571),d,l,n[t+3],10,-1894986606),v,d,n[t+10],15,-1051523),A,v,n[t+1],21,-2054922799),l=a(l,A=a(A,v=a(v,d,l,A,n[t+8],6,1873313359),d,l,n[t+15],10,-30611744),v,d,n[t+6],15,-1560198380),A,v,n[t+13],21,1309151649),l=a(l,A=a(A,v=a(v,d,l,A,n[t+4],6,-145523070),d,l,n[t+11],10,-1120210379),v,d,n[t+2],15,718787259),A,v,n[t+9],21,-343485551),v=e(v,o),d=e(d,i),l=e(l,h),A=e(A,g);return[v,d,l,A]}function h(n){var r,t="",e=32*n.length;for(r=0;r<e;r+=8)t+=String.fromCharCode(n[r>>5]>>>r%32&255);return t}function g(n){var r,t=[];for(t[(n.length>>2)-1]=void 0,r=0;r<t.length;r+=1)t[r]=0;var e=8*n.length;for(r=0;r<e;r+=8)t[r>>5]|=(255&n.charCodeAt(r/8))<<r%32;return t}function v(n){var r,t,e="";for(t=0;t<n.length;t+=1)r=n.charCodeAt(t),e+="0123456789abcdef".charAt(r>>>4&15)+"0123456789abcdef".charAt(15&r);return e}function d(n){return unescape(encodeURIComponent(n))}function l(n){return h(i(g(r=d(n)),8*r.length));var r}return v(l(r+":"+t+":"+window.navigator.userAgent+":"+n))}
Call this function like this: gishash("{\"id\":\"5821462185\",\"first\":40,\"after\":\"\"}", rhx_gis, csrf_token).
rhx_gis and csrf_token can be parsed from any embed page source (CORS available on this links);

I've tried to archive this via javascript but here is the problem: I can't set these custom headers due allow-origin limitation for custom headers on instagram side, but this is not a problem in php I guess.

350d on 10 Apr 2018

👍2

@350d, wow, that actually worked! How did you go about extracting (or building) that hashing function? Just curious for future endeavors. Thanks!

andrewyoo on 10 Apr 2018

@andrewyoo just simple debug in browser

350d on 10 Apr 2018

Seems like mentioned solutions here are no longer working :(
Need to find out the way the cookie param is generated for non-authorized users.

footniko on 10 Apr 2018

I've just realized that x-instagram-gis is just an md5 hash 😀

350d on 10 Apr 2018

🎉1

@350d have you tried to make some requests? Because x-instagram-gis is not enough now for non-authorized users. As i understand need valid cookie data which is generated on every request.

footniko on 10 Apr 2018

@footniko you should generate this header for every request

350d on 10 Apr 2018

Yes, but only this header is not enough...

var data = null;

var xhr = new XMLHttpRequest();
xhr.withCredentials = true;

xhr.addEventListener("readystatechange", function () {
  if (this.readyState === 4) {
    console.log(this.responseText);
  }
});

xhr.open("GET", "https://www.instagram.com/graphql/query/?query_hash=bfe6fc64e0775b47b311fc0398df88a9&variables=%7B%22user_id%22%3A%224502807%22%2C%22include_chaining%22%3Afalse%2C%22include_reel%22%3Afalse%2C%22include_suggested_users%22%3Afalse%2C%22include_logged_out_extras%22%3Atrue%7D");
xhr.setRequestHeader("x-instagram-gis", "2b92887dc6325064cf6294b95aa04586");
xhr.setRequestHeader("cache-control", "no-cache");

xhr.send(data);

Returns:

{
    "message": "forbidden",
    "status": "fail"
}

footniko on 10 Apr 2018

add X-CSRFToken and X-Instagram-AJAX: 1 headers

350d on 10 Apr 2018

@350d so you mean following cookies are enough?
x-instagram-gis : generate randomized md5 hash
X-CSRFToken : csrftoken
X-Instagram-AJAX: 1

knissophiliac on 10 Apr 2018

@knissophiliac these headers are not randomized, they are calculated from current csrf_token and rhx_gis and other vars.

350d on 10 Apr 2018

@350d ok. i read your comment above, but i couldn't find rhx_gis one in my responses.

knissophiliac on 10 Apr 2018

@footniko I used the following headers which are working for anonymous crawling:

```
{
"x-instagram-gis" => gistoken,
"cookie" => "csrftoken=#{csrf_token}",
"user-agent" => 'user agent string'
}
````

gistoken is calulated with @350d's function
csrftoken in the cookie
user-agent string also passed in so that gistoken can be calculated and compared on the server side.

andrewyoo on 10 Apr 2018

❤1

@knissophiliac you can get rhx_gis just from a GET on 'https://www.instagram.com'. Also it's pretty much returned in the html of most pages.

@350d It's still working for me...

andrewyoo on 10 Apr 2018

👍2

@andrewyoo confirmed - it works, the only you need to keep in mind - request and parse tokens (rhx_gis and csrftoken) with same user agent as for other requests. Another problem - rate limits. Looks like its per IP based limits.

350d on 10 Apr 2018

👍3

Guys, I'm trying to auth Instagram, but this requires a confirmation. If i do this with ssh, the script doesn't accept cookies. How can I do it? I cant enter the code without CLI.

RainGrid on 10 Apr 2018

@350d I have a working javascript implementation using the request-promise nodeJS library. I am sending the csrftoken as a cookie and the x-instagram-gis value as a header. Instagram seems to accept it.

jeff-an on 11 Apr 2018

@jeff-an I have server side solution too, but before that was a client side solution only

350d on 11 Apr 2018

So the hash function is just an md5 of ":::", right?
I am still getting 403 errors and I am not sure what I am doing wrong.

szokeptr on 11 Apr 2018

@szokeptr and json variables

350d on 11 Apr 2018

@szokeptr you are missing {"id":\

jeff-an on 11 Apr 2018

@350d @jeff-an My original reply contained <query variables> which is striped by github for some reason. Anyways, I am generating the hash correctly, and still cannot get it to work.
I include the cookies, csrftoken, gis hash, everything and constantly get 403 errors.

szokeptr on 11 Apr 2018

@szokeptr read about user agent here

350d on 11 Apr 2018

Hi, It seems to work but only in chrome,

$.ajax({
type: 'POST',
processData: true,
data: {},
dataType: 'jsonp',
url: "https://www.instagram.com/graphql/query/?query_hash=42323d64886122307be10013ad2dcc44&variables={%22id%22:%227204036595%22,%22first%22:30,%22after%22:%22%22}"
}) .done(function( data ) {
console.log( data);
});

We can use dataType: 'jsonp', to overcome Cross-origin resource sharing (CORS).

AsgharRehman on 11 Apr 2018

@AsgharRehman If it works in chrome, it will work anywhere. Probably it works in Chrome because you're logged in to Instagram in Chrome. If you log out it won't work.

footniko on 11 Apr 2018

@footniko This is working fine, May be we need to set the same cookies as it's generated when someone logged in to instagram?

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://www.instagram.com/graphql/query/?query_hash=42323d64886122307be10013ad2dcc44&variables={\"id\":\"256462779\",\"first\":30,\"after\":\"\"}");
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type:application/x-www-form-urlencoded', 'Cookie: sessionid=IGSC020ee8ab76d1e8b01223ab92304cbcd58075ce0bdbc771e872b77046bd82cf40:KXP3eyA1gJybewPt4CDR6ICsiF1y99Eq:{"_auth_user_id":7204036595,"_auth_user_backend":"accounts.backends.CaseInsensitiveModelBackend","_auth_user_hash":"","_platform":4,"_token_ver":2,"_token":"7204036595:iWyZVjjdX5sVFV5VsozthzDCIzBh0muN:99a84e59d3aa505f433844956d47d95e4bf095a24a6e1e397f2418c6384671d2","last_refreshed":1523431960.5035405159}; '));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 20);
$result = curl_exec($ch);
$last = curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);

print_r(array($result,$last));

curl_close($ch);

AsgharRehman on 11 Apr 2018

$.ajax({
type: 'POST',
processData: true,
data: {},
dataType: 'jsonp',
url: "https://www.instagram.com/graphql/query/?query_hash=42323d64886122307be10013ad2dcc44&variables={%22id%22:%227204036595%22,%22first%22:30,%22after%22:%22%22}"
}) .done(function( data ) {
console.log( data);
});

@AsgharRehman This solution returns this error Uncaught SyntaxError: Unexpected token: how to resolve this issue?

soni2268 on 11 Apr 2018

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://www.instagram.com/graphql/query/?query_hash=42323d64886122307be10013ad2dcc44&variables={\"id\":\"256462779\",\"first\":30,\"after\":\"\"}");
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type:application/x-www-form-urlencoded', 'Cookie: sessionid=IGSC020ee8ab76d1e8b01223ab92304cbcd58075ce0bdbc771e872b77046bd82cf40:KXP3eyA1gJybewPt4CDR6ICsiF1y99Eq:{"_auth_user_id":7204036595,"_auth_user_backend":"accounts.backends.CaseInsensitiveModelBackend","_auth_user_hash":"","_platform":4,"_token_ver":2,"_token":"7204036595:iWyZVjjdX5sVFV5VsozthzDCIzBh0muN:99a84e59d3aa505f433844956d47d95e4bf095a24a6e1e397f2418c6384671d2","last_refreshed":1523431960.5035405159}; '));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 20);
$result = curl_exec($ch);
$last = curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);

print_r(array($result,$last));

curl_close($ch);

@AsgharRehman returns false

soni2268 on 11 Apr 2018

How is the query_hash calculated? is it the same as the "gistoken" ? or is it the csrp_token?

ahait on 12 Apr 2018

@ahait The query_hash specifies which query you want to execute (user media, followers etc.), it changes from time to time, but you should be able to just use the same as the current web version.

szokeptr on 12 Apr 2018

https://www.instagram.com/vasiliizaikovskii/?__a=1 this works!

This doesn't work today.
Got 403 status. :(

kenjones91 on 12 Apr 2018

👍1

@kenjones91 omg, they're closing more up. This will kill my site. 🙁

fattony80 on 12 Apr 2018

😕3

@350d can you please provide x-instagram-gis md5-hash calculation function and example?
That would be very helpful, thank you!

myrs on 12 Apr 2018

Sorry @350d, didn't see szokepter's comment and your answer to it 😅
Anyway, this is the working example of x-instagram-gis before beeing hashed:
'ee6624bcbf4d2264c61623dc1eb00208:Ku1sDF1yPG8qHY0huKHLTazsALW4PYcM:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36:{"id":"290023231","first":12,"after":"AQBc59gud60EvXETIDKdNYu-h9ZbXaBU4LJz55hC07cWpvjmGCT-XEb74-pUDLcdh2ChWyWkj9n7IZ6eBbgCaOodJ300tHiaHpaJBIXTrllVrQ"}'

myrs on 12 Apr 2018

@soni2268 Lets see how long it works with those additional cookies #337

AsgharRehman on 12 Apr 2018

As pointed before, CSRF seems to expire in one year. The same rhx_gis could also be used for all request, so no need to get it from page source.
The solution to make this scrapper work again could be the following (putting my python code, the final step is adapt it to PHP 😅):

from hashlib import md5
import requests

# >> this values to be set dinamicaly
user_id = '290023231'
first = "12"
after = "AQBc59gud60EvXETIDKdNYu-h9ZbXaBU4LJz55hC07cWpvjmGCT-XEb74-pUDLcdh2ChWyWkj9n7IZ6eBbgCaOodJ300tHiaHpaJBIXTrllVrQ"
# <<

# >> use this csrftoken, rhx_gis and user_agent for all requests
csrftoken = "Ku1sDF1yPG8qHY0huKHLTazsALW4PYcM"
rhx_gis = "ee6624bcbf4d2264c61623dc1eb00208"
user_agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
# <<

# >> calculate md5 for each request
query_variables_template = '{"id":"%s","first":%s,"after":"%s"}'
query_variables = query_variables_template % (user_id, first, after)
string_to_hash = rhx_gis + ':' + csrftoken + ':' + user_agent + ':' + query_variables
x_instagram_gis = md5(string_to_hash).hexdigest()
# <<

# >> set headers
csrftoken_cookie = "csrftoken=%s" % csrftoken
headers = {
    "x-instagram-gis": x_instagram_gis, 
    "cookie": csrftoken_cookie,
    "user-agent": user_agent
}
# <<

# >> set cookies
browser_cookies = [
    {
        'name': 'csrftoken',
        'value': csrftoken
    }
]
session = requests.Session()
for cookie in browser_cookies:
    c = {cookie['name']: cookie['value']}
    session.cookies.update(c)
# << set cookies

# get response as JSON
media_url = 'https://www.instagram.com/graphql/query/?query_hash=42323d64886122307be10013ad2dcc44&variables={"id":"%s","first":%s,"after":"%s"}'
response = session.get(media_url % (user_id, first, after), headers=headers, verify=False).json()

myrs on 12 Apr 2018

Everything is working fine now.
:)

kenjones91 on 12 Apr 2018

Hm.. it actually works again with old good https://www.instagram.com/<username>/?__a=1&max_id=<end_cursor> thing. But for how long? 🤔

myrs on 12 Apr 2018

@350d this seems to be broken again. i dont have to look through the debugger right now but can you confirm?

jeff-an on 13 Apr 2018

https://www.instagram.com/<username>/?__a=1 broken again. Returns 403.
The last several days it was dying and resurrecting, so probably we shouldn't rely on this one.

myrs on 13 Apr 2018

Right but the query_hash=42323d64886122307be10013ad2dcc44 endpoint is not working for me either at the moment.

jeff-an on 13 Apr 2018

Looks like this cat and mouse game can't be finished if we continue to discuss these updates and fixes in public...

350d on 13 Apr 2018

👍3 😕2

Looks like the hash of x-instagram-gis no longer includes the user-agent string.

jeff-an on 13 Apr 2018

👍4

@jeff-an you are absolutely right.
Python: md5.new(rhx_gis + ":" + csrf_token + ":" + params).hexdigest()
This works for me

mnatsakanyank on 13 Apr 2018

The medias seem to work now with latest changes and logged in account, however it quickly reaches the limit.
Body: message => rate limited; status => fail;

Does anyone know more details on the limits? What is the limit, is it based on user/ip/both?

gthedev on 13 Apr 2018

@gthedev based on IP

350d on 13 Apr 2018

@350d that's weird, if I use same login details but different IPs, the limit error is still there?

gthedev on 13 Apr 2018

The medias seem to work now with latest changes and logged in account, however it quickly reaches the limit.
Body: message => rate limited; status => fail;

Does anyone know more details on the limits? What is the limit, is it based on user/ip/both?

The maximum request is 200 per hour!
Check detail here:
https://stackoverflow.com/questions/49585077/instagram-api-limit-reduced-to-200-from-5000

kenjones91 on 13 Apr 2018

👍1

@kenjones91 I'm pretty sure the rate limits are set on the combination username and IP. If I use my proxys witha n account that had a raye limit, it works again, and if I use another account on an IP that's limited, it also works. So my guess is it locks an account associated with ip address for a certaik moment of time. Not sure how long though. I've never been locked out for longer than a day so the lock-out period isn't that long.

bomboem on 13 Apr 2018

I just solve the problem with parsing html page of account and then taking json from javascript. Yes it is just 12 medias, but it works :) I "love" instagram more and more )))

zaivst on 15 Apr 2018

🎉1

@zaivst I think I might do that too. Mind sharing your code? Thanks!

Scottzonn on 16 Apr 2018

@Scottzonn somethink like this and still works :) ...

function getMediasFromURL($username, $count = 12)
{
    $medias = array();
    $doc = new DOMDocument();
    $doc->loadHTML(implode("",file('https://www.instagram.com/'.$username.'/')));
    $jsNodes = $doc->getElementsByTagName("script");        
    $jsNodeTmp = "";
    foreach($jsNodes as $node){
        if(strpos($node->nodeValue,"window._sharedData")!==false){
            $jsNodeTmp = $node->nodeValue;
            break;
        }
    }       
    $medias = array();
    if($jsNodeTmp){
        $jsNodeTmp = trim(str_replace("window._sharedData","",$jsNodeTmp)," ;=");
        $json = json_decode($jsNodeTmp);
        $jsonMedia = $json->entry_data->ProfilePage[0]->graphql->user->edge_owner_to_timeline_media->edges;
        foreach($jsonMedia as $jsonMediaItem){
            if(count($medias) < $count)
                $medias[] = $jsonMediaItem;
        }
    }
    return $medias;
}

But in this case scrapper is not neccesary to use... Hope real solution will be found.

zaivst on 16 Apr 2018

👍1

@zaivst will this solution only work for the first 12 medias?

Saiirex on 16 Apr 2018

@Saiirex unfortunatly yes

zaivst on 16 Apr 2018

@zaivst is there currently a way to get more then 12 medias with JS?

Saiirex on 16 Apr 2018

No workarounds proposed by devs?

MUTOgen on 16 Apr 2018

@Saiirex I want this too, but need to use this parser...

zaivst on 16 Apr 2018

@zaivst it would be perfect to integrate this into this project, like optional way to parse something

MUTOgen on 16 Apr 2018

it would be perfect to integrate this into this project, like optional way to parse something

Below simple code to perform HTML parsing and get media data from JS (Instagram.php)

    /**
     * Get medias shared by user (HTML parser)
     *
     * @param     string    $username
     * @return    Media[]
     * @throws    InstagramException
     */
    public function getMediasSharedByAccount($username)
    {
      $response = Request::get(Endpoints::getAccountPageLink($username));
      if (static::HTTP_NOT_FOUND === $response->code) {
          throw new InstagramNotFoundException('Account with given username does not exist.');
      }
      if (static::HTTP_OK !== $response->code) {
          throw new InstagramException('Response code is ' . $response->code . '. Body: ' . static::getErrorBody($response->body) . ' Something went wrong. Please report issue.');
      }
      $regex = '/window\._sharedData.*<\/script>/';
      preg_match($regex, $response->raw_body, $data);
      $data = $data[0];
      $data = str_replace('window._sharedData = ', '', $data);
      $data = str_replace(';</script>', '', $data);
      $data = json_decode($data, true, 512, JSON_BIGINT_AS_STRING);
      $nodes = $data['entry_data']['ProfilePage'][0]['graphql']['user']['edge_owner_to_timeline_media']['edges'];
      if (!isset($nodes) || empty($nodes)) {
        return [];
      }
      $medias = [];
      foreach ($nodes as $mediaArray) {
        $medias[] = Media::create($mediaArray['node']);
      }
      return $medias;
    }

ZipDriver on 16 Apr 2018

👍2

Guys, what is the working version today ? :D

RainGrid on 16 Apr 2018

@RainGrid i guess there is no "out of box" version

MUTOgen on 16 Apr 2018

@MUTOgen I keep retrieving the first comments from media?__a=1 endpoint, but it is not possible to get information from user?__a=1. And I can not work with the credentials because it requires confirmation.

RainGrid on 16 Apr 2018

-_- it seems like all the authentication/csrf/cookie validation was just removed

Edit: nevermind, auth is back up. im glad i didnt write a bunch of code for nothing!

jeff-an on 17 Apr 2018

I did some investigation today.
"https://www.instagram.com/graphql/query/?query_hash=42323d64886122307be10013ad2dcc44&variables={"id":"7269904749","first":12,"after":"AQBD8UO7BSS7272VzYx5K-XJPzigMTgfvjwlYWGPZnlYeE40vGSb573s3az81O7-F4yLfHD1mfCPlJfCurg9gRjQXRwa5EIJCYIOURDcowK2VQ"}"

is working fine if i am adding the X_Instagram-GIS header to the request headers. But how are this headers calculated?

Saiirex on 17 Apr 2018

Above have already been told how X_Instagram-GIS calculated, but the game in "cat and mouse" continues and today encryption looks like this in PHP:
$gis = md5(join(':', array( $page_data['rhx_gis'], json_encode($variables) )));

welcomemax on 17 Apr 2018

👍3 ❤1

looks like we have to wait until they calm a bit down...

Saiirex on 17 Apr 2018

So anyone knows what are the current rate limits for graphql? I am getting
{ "message": "rate limited", "status": "fail" }
after ~20 requests

mnatsakanyank on 17 Apr 2018

@mnatsakanyank - @raiym said ~200per/h but @andrewyoo claimed in #330 it dropped

lerni on 18 Apr 2018

On the web api, which we have been discussing, it’s far lower than 200/hr. I have tested it empirically by loading a ig page and kept infinite scrolling. You’ll hit 429s pretty quick. The counter resets when u login, but it’s the same limit when logged in. I have tried infinite scrolling the mobile api through the mobile app and it appears to be much higher but I don’t know the upper bound.

I believe the stated rate limit of 200/hr is when using the blessed developer graph api which we are not using.

I have tried (a few days ago) 200/hr and even 120/hr and have gotten 429s.

andrewyoo on 18 Apr 2018

guys, i am feeling paranoid. So i was trying to find out rate limit of the graphql endpoint without having logged in user using postman and browser. Eventually, i got to the limit and my ip was blocked, the strange part is that my phone was also blocked, whenever I am trying to scroll down in any instagram profile using mobile chrome incognito mode I am getting 429.. and the facts are

I haven't used instagram web on my phone for ages
I am using 3g so my laptop and phone should have different ips
I have Facebook and Instagram apps installed on my phone

Can it be that they have MAC address of all my devices? Or can someone explain whats happening...

I have destroyed my phone and laptop, and moving to the mountains ))

mnatsakanyank on 18 Apr 2018

😕3

Now i have this error:

Missing token 'user-agent' in CORS header 'Access-Control-Allow-Headers' from CORS preflight channel

My code:

`var variables = '{"id":"' + userId + '","first":12,"after":"' + nextId + '"}';
var nextUrl ='https://www.instagram.com/graphql/query/?query_hash=42323d64886122307be10013ad2dcc44&variables=';

            nextUrl = nextUrl + variables;

            var instagramGis = md5(gis + ":" + variables);

            $.ajax({
                url: nextUrl,
                beforeSend: function(request) {
                    request.setRequestHeader("user-agent", window.navigator.userAgent);
                    request.setRequestHeader("x-instagram-gis", instagramGis);
                    request.setRequestHeader("X-CSRFToken", csrfToken);

                },
                type: 'GET',
                success: function(data){
                    media = data.data.user.edge_owner_to_timeline_media;
                    nodes = media.edges;
                    nextId = media.page_info.end_cursor;

                    var newMediaFromNode = getMediaFromNodes(nodes, false);

                    $.each(newMediaFromNode, function(index, element){
                        finalMedia.push(element);
                    });
                },
                error: function (xhr, status, error) {
                    alert(error);
                },
                async : false
            });`

Saiirex on 18 Apr 2018

Instagram-php-scraper: Can't retrieve user medias

Most helpful comment

All 109 comments

328 Need to test... But this solution works for me 😌

Related issues