Instapy: [Feature Request] Random scrolling stops me from getting blocked!

[Flaunkerton2395]

For quite a while, I would get blocked after liking/commenting after only about 5 photos. However for the last week, I have been running without headless browser and randomly scrolling through whatever page the script is on. This has pretty much let me run uninterrupted for 2+ hours at a time as long as I scroll at random like a human being is using Instagram.

Is there a way we could program random scrolling on a page? It seems as though Instagram categorizes bots as ones that just sit on a page and don't actually interact with it at all.

[Flaunkerton2395]

Also confirmed that I used to have issue #5042 but this has fixed it

[Flaunkerton2395]

5032

[prototype127]

same issue . it was working fine before yesterday and now it gives me action block message on simply liking and commenting

[prototype127]

@timgrossmann @converge any solutions ??

[Flaunkerton2395]

@timgrossmann is this a feature we can implement?

[prototype127]

@Flaunketron2395 lets find the solution together ?

[realsnad]

@timgrossmann is this a feature we can implement?

Ok, I implemented the feature. Let's test together. All you need to do is add the following function to the end of util.py as the last function:

def random_scroll(browser):
    page_height = browser.execute_script("return document.body.scrollHeight")
    update_activity(browser, state=None)

    scroll_quantity = random.randint(3, 5)
    total_scroll_height = 0

    for _ in range(scroll_quantity):
        scroll_height = random.randint(int(page_height / 2), page_height)
        scroll_sleep = random.randint(1, 3)

        browser.execute_script("window.scrollBy(0, " + str(scroll_height) + ")")
        total_scroll_height += scroll_height

        update_activity(browser, state=None)
        sleep_actual(scroll_sleep)

    browser.execute_script("window.scrollBy(0, -" + str(total_scroll_height) + ")")
    update_activity(browser, state=None)

And then call it also in util.py in function web_address_navigator by adding it as the last line outside of if condition as follows:

random_scroll(browser)

For reference, in case you're using 0.6.3 web_address_navigator would look like this after the change:

def web_address_navigator(browser, link):
    """Checks and compares current URL of web page and the URL to be
    navigated and if it is different, it does navigate"""
    current_url = get_current_url(browser)
    total_timeouts = 0
    page_type = None  # file or directory

    # remove slashes at the end to compare efficiently
    if current_url is not None and current_url.endswith("/"):
        current_url = current_url[:-1]

    if link.endswith("/"):
        link = link[:-1]
        page_type = "dir"  # slash at the end is a directory

    new_navigation = current_url != link

    if current_url is None or new_navigation:
        link = link + "/" if page_type == "dir" else link  # directory links
        # navigate faster

        while True:
            try:
                browser.get(link)
                # update server calls
                update_activity(browser, state=None)
                sleep(2)
                break

            except TimeoutException as exc:
                if total_timeouts >= 7:
                    raise TimeoutException(
                        "Retried {} times to GET '{}' webpage "
                        "but failed out of a timeout!\n\t{}".format(
                            total_timeouts,
                            str(link).encode("utf-8"),
                            str(exc).encode("utf-8"),
                        )
                    )
                total_timeouts += 1
                sleep(2)

    random_scroll(browser) # added for random scrolling

This change makes bot scroll randomly after navigating to each page. However, there are lots of places in the code where the currently open page gets reloaded using browser.execute_script("location.reload()") as the second try of the failed action (WebDriverException or NoSuchElementException, for example). The bot won't scroll after such reloads since I added the function call only to web_address_navigator. I think it's not a big deal since such "first-second try" exceptions don't occur that often but that's definitely what we can look into in more detail after we confirm that this solution actually helps.

[bry1026]

This solution seems to be working so far for me. Been running uninterrupted for an hour now.

[realsnad]

This solution seems to be working so far for me. Been running uninterrupted for an hour now.

Hour is nothing :) Wait a couple more and see. Mine is working so far as well.

[Flaunkerton2395]

I am running without a headless browser but I don't see the page actually scrolling. @bry1026 @ym85 do you actually see the page moving as the script is running?

[realsnad]

I am running without a headless browser but I don't see the page actually scrolling. @bry1026 @ym85 do you actually see the page moving as the script is running?

Yes, I see each page scrolling several times down and then back up. I tested the code before I posted :)

[bry1026]

Also running with headless set to false and page doesn’t actually scroll for me.

[realsnad]

Make sure you added random_scroll(browser) to the end of web_address_navigator.

[bry1026]

Just double checked and I did

On Sat, Sep 14, 2019 at 11:31 PM ym85 [e-mail] wrote:

Make sure you added random_scroll(browser) to the end of
web_address_navigator.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/timgrossmann/InstaPy/issues/5068?email_source=notifications&email_token=AEYUJ367LAXOSU5SJBOCC5DQJWUBPA5CNFSM4IWR4XPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6XIM4Y#issuecomment-531531379,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEYUJ326BO7L7IGLMNXMO63QJWUBPANCNFSM4IWR4XPA
.

>

Bryan Marin | Software Engineering Intern | bryan.[email protected]
[e-mail] | 954-394-3495

[Flaunkerton2395]

Do we have to maybe put the call for scrolling in a different place?

[realsnad]

Do we have to maybe put the call for scrolling in a different place?

Drop your util.py here. I’ll check it out.

[Flaunkerton2395]

Here you go!
util.py.zip

[realsnad]

Here you go!
util.py.zip

Seems that everything is fine and should be working as designed. Here is a short video of what I see using headless=False.

And here is my improved version of random_scroll:

def random_scroll(browser):
    scroll_y_orig = 1000
    random_percent = round(random.uniform(0.80, 0.98), 2)
    scroll_y_random = random.randint(
        int(scroll_y_orig * random_percent), scroll_y_orig
    )

    scroll_quantity = random.randint(3, 5)
    total_scroll_y = 0

    for _ in range(scroll_quantity):
        scroll_y = random.randint(
            int(scroll_y_random / 2), scroll_y_random
        )
        scroll_sleep = random.randint(1, 3)
        sleep_actual(scroll_sleep)

        browser.execute_script("window.scrollBy(0, " + str(scroll_y) + ")")
        update_activity(browser, state=None)

        total_scroll_y += scroll_y

    scroll_top_sleep = random.randint(1, 3)
    sleep_actual(scroll_top_sleep)

    # scroll to the top
    browser.execute_script("window.scrollBy(0, -" + str(total_scroll_y) + ")")
    update_activity(browser, state=None)

[prototype127]

def random_scroll(browser):
scroll_y_orig = 1000
random_percent = round(random.uniform(0.80, 0.98), 2)
scroll_y_random = random.randint(
int(scroll_y_orig * random_percent), scroll_y_orig
)

scroll_quantity = random.randint(3, 5)
total_scroll_y = 0

for _ in range(scroll_quantity):
    scroll_y = random.randint(
        int(scroll_y_random / 2), scroll_y_random
    )
    scroll_sleep = random.randint(1, 3)
    sleep_actual(scroll_sleep)

    browser.execute_script("window.scrollBy(0, " + str(scroll_y) + ")")
    update_activity(browser, state=None)

    total_scroll_y += scroll_y

scroll_top_sleep = random.randint(1, 3)
sleep_actual(scroll_top_sleep)

# scroll to the top
browser.execute_script("window.scrollBy(0, -" + str(total_scroll_y) + ")")
update_activity(browser, state=None)

hey can you drop your util.py file ?

[prototype127]

Make sure you added random_scroll(browser) to the end of web_address_navigator.

iam not getting it , where should i add random-scroll(browser) can you help me plz ? as there are lots of web_address_navigator option in util.py

[realsnad]

hey can you drop your util.py file ?

Here you go: util.zip

But I got bad news. After 9.5 hours of work and only 56 successful follows I get 429 error again: "cannot get number of posts for username", "Sorry, couldn't check for number of posts of username".

Maybe we should lower peak_server_calls_hourly and peak_server_calls_daily? Right now I have 500 and 4700, respectively.

[Flaunkerton2395]

@ym85 how is it possible for your scrolling to work but ours does not? Are you calling the scrolling function in your quickstart too?

[prototype127]

@Flaunkerton2395 just copy his util.py ... as it works for me

[prototype127]

@ym85 thanks for replying bro ! Is this 429 error is action blocked ? And one more question as there is one more bug in the script .. . When i add a comment and put @username in it. .. the bot simply writes the comment but do not click on the profile as it shows 2 3 profie but the next action it should take to click on the profile and then post .. sorry for my bad English as iam not a native emglish speaker :page_with_curl:

[prototype127]

Okay so its been 35 mins and i got action blocked ... the scrolling was fine and i was running it slow.. but didn't worked. :(((((((((((( .. .. but what i have noticed when i log out and again log in it works again for sometime. .. so what if we add an option of login logout ... as it works when we log in again after logging out. We can add this feature as it works better than the scrolling option.

[prototype127]

@ym85 @Flaunkerton2395 bro try log in log out method

[Flaunkerton2395]

@prototype127 I copied his exact util.py and it still will not work for me. Are you sure it scrolls on every single page for you?

Also I've noticed logging in and out does seem to 'reset' blocking for me at least for a few hours.

[prototype127]

@Flaunkerton2395 yes iam dang sure ! On every single page it was scrolling and after running for 35 mins i got a action blocked message . Yeah log in log out is a better way . . Do we have to edit qucikstart.py for it ?

[prototype127]

Or if could just add 3 options scrolling + moving lil cursor and log in log out mode for every 12 - 13 mins :P ... it would be fine then maybe

[turtlebking]

hey can you drop your util.py file ?

Here you go: util.zip

But I got bad news. After 9.5 hours of work and only 56 successful follows I get 429 error again: "cannot get number of posts for username", "Sorry, couldn't check for number of posts of username".

Maybe we should lower peak_server_calls_hourly and peak_server_calls_daily? Right now I have 500 and 4700, respectively.

we can add if-statment at the end of "def web_address_navigator(browser, link):"
(30% chance to perform scolling action)

if random.randrange(0, 100) < 30:    
    random_scroll(browser) # added for random scrolling

[realsnad]

hey can you drop your util.py file ?

Here you go: util.zip
But I got bad news. After 9.5 hours of work and only 56 successful follows I get 429 error again: "cannot get number of posts for username", "Sorry, couldn't check for number of posts of username".
Maybe we should lower peak_server_calls_hourly and peak_server_calls_daily? Right now I have 500 and 4700, respectively.

we can add if-statment at the end of "def web_address_navigator(browser, link):"
(30% chance to perform scolling action)

if random.randrange(0, 100) < 30:    
    random_scroll(browser) # added for random scrolling

Thanks, man. But I wouldn't waste time on this since we've already confirmed that this method isn't helping much.

[turtlebking]

Thanks, man. But I wouldn't waste time on this since we've already confirmed that this method isn't > helping much.
I agree. :)

[realsnad]

You guys can check out my idea in #5042.

[breuerfelix]

you guys are doing awesome stuff ! please don't forget to make pull request so everybody is able to profit as much as you do :P

even though this solution does not 100% fix this problem we can still use it because there wont be a 100% fix. It will be many small steps and each step will lead to a little less detection :)

[alexsmedile]

Is it been implemented in 6.4 ?

[realsnad]

Is it been implemented in 6.4 ?

I didn’t PR it since it is not helping with the issue. So, I don’t think it is in 0.6.4.

[alexsmedile]

I can finally run my bot for hours, here's how I did it:

Upgrade to InstaPy 6.4 (idk about dev branch), then add @ym85 's scrolling feature and set peak server calls to less than 200 per hour.

bot.set_quota_supervisor(enabled=True, # limit hourly and daily peak
                sleep_after=["likes", "comments", "follows", "unfollows", "server_calls"],
                sleepyhead=True, stochastic_flow=True, notify_me=True,
                peak_likes_hourly=24,
                peak_likes_daily=560,
                peak_comments_hourly=6,
                peak_comments_daily=120,
                peak_follows_hourly=6,
                peak_follows_daily=120,
                peak_unfollows_hourly=6,
                peak_unfollows_daily=120,
                peak_server_calls_hourly=120,
                peak_server_calls_daily=2600)

Set action delays to slower than slow and use the random function:

bot.set_action_delays(
            enabled=True,
            like=45,
            comment=45,
            follow=56,
            unfollow=56,
            randomize=True,
            random_range_from=100,
            random_range_to=300)

DO NOT make any manual interaction while the bot is running.

Other precautions: Change password and delete InstaPy _cookies_ before starting your bot again. Run InstaPy on your home PC or use a _proxy_ with a domestic IP. Connect no more than 3 devices per user. Do not make the SAME repeated actions again and again (if you just follow/unfollow they will catch you). Make your code _as random as possible_ and make it sleep pretty much as a human would sleep. Post regularly and if you want you can still use your phone to interact manually (if the bot's been off for at least one hour). This is the only way I found to run the bot smoothly without blocks.

[Flaunkerton2395]

@alexsmedile so is the scrolling addition actually working for you? It never did anything for me..

[brunomiguelleal]

@ym85 the random scrolling is mitigating the issue?

[realsnad]

@ym85 the random scrolling is mitigating the issue?

Nope. I think it’s just a coincidence that “it worked” for some people.

[brunomiguelleal]

@ym85 I believe InstaPy actions need to be changed in order to have the same sequence as a normal human user. Unfortunately I'm really busy (between work and taking care of a baby daughter) with no time to change and test InstaPy.
Usually, what I do as a normal user is quite simples:

1. Open Instagram (login, etc.)
   Repeat:
2. Search for a tag
3. like (all or almost all) photos I scroll
4. click follow on the photos I really like (before follow I can go into the account profile to check more pictures - but not always)
5. write comments on some photos

As soon as I have some time, this is what I intend to create using InstaPy functionalities (such as logging, etc)

[prototype127]

(Not a native english speaker ) .. when i get action block msg i immediately log out with my instagram and then run the bot again it works again for sometime then again blocks me .. i again log out and then log in when it shows me the action block msg .. it works again then for sometime and it goes on .. if we could apply a feature of log in log out when instagram shows the action block msg .. iam damn sure it will gonna be worth it. As i have tested other bots ( for liking the pictures) happens same ... this log in log out feature will help us.

[prototype127]

Plz test this feature manually .. if you got action blocked then immediately log out and start the bot again .. it will work !

[prototype127]

@ym85 @Flaunkerton2395 @alexsmedile

[Wakkox]

I can finally run my bot for hours, here's how I did it:

Upgrade to InstaPy 6.4 (idk about dev branch), then add @ym85 's scrolling feature and set peak server calls to less than 200 per hour.

bot.set_quota_supervisor(enabled=True, # limit hourly and daily peak
                sleep_after=["likes", "comments", "follows", "unfollows", "server_calls"],
                sleepyhead=True, stochastic_flow=True, notify_me=True,
                peak_likes_hourly=24,
                peak_likes_daily=560,
                peak_comments_hourly=6,
                peak_comments_daily=120,
                peak_follows_hourly=6,
                peak_follows_daily=120,
                peak_unfollows_hourly=6,
                peak_unfollows_daily=120,
                peak_server_calls_hourly=120,
                peak_server_calls_daily=2600)

Set action delays to slower than slow and use the random function:

bot.set_action_delays(
            enabled=True,
            like=45,
            comment=45,
            follow=56,
            unfollow=56,
            randomize=True,
            random_range_from=100,
            random_range_to=300)

DO NOT make any manual interaction while the bot is running.

Other precautions: Change password and delete InstaPy _cookies_ before starting your bot again. Run InstaPy on your home PC or use a _proxy_ with a domestic IP. Connect no more than 3 devices per user. Do not make the SAME repeated actions again and again (if you just follow/unfollow they will catch you). Make your code _as random as possible_ and make it sleep pretty much as a human would sleep. Post regularly and if you want you can still use your phone to interact manually (if the bot's been off for at least one hour). This is the only way I found to run the bot smoothly without blocks.

I did this but it's super slow, every 20 minutes the bot goes to sleep.

[alexsmedile]

@Wakkox Yes, it's slow. In 1 hour you are going to give no more than 20 likes.

[Vrzhezhevsky]

@Wakkox Yes, it's slow. In 1 hour you are going to give no more than 20 likes.

Hi! Can u upload your quickstart.py as an example?
I'm kinda new to this app. Just so I can adapt it. I'm not sure if there any functions that overlap with standard ones. Also getting " undefined variable 'bot' "
Thx in advance!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this problem still occurs, please open a new issue