Instapy: return self._execute(Command.GET_ELEMENT_TEXT)['value']

[fgisslen]

Traceback (most recent call last):
File "/home/fredrik/InstaPy/5b7fab2fbd3a84001490510f.py", line 25, in
session.interact_user_followers(['xxx','xxx'], amount=30, randomize=True)
File "/home/fredrik/InstaPy/instapy/instapy.py", line 1820, in interact_user_followers
self.logfolder)
File "/home/fredrik/InstaPy/instapy/unfollow_util.py", line 919, in get_given_user_followers
channel, logger, logfolder)
File "/home/fredrik/InstaPy/instapy/unfollow_util.py", line 722, in get_users_through_dialog
quick_username = dialog_username_extractor(quick_button)
File "/home/fredrik/InstaPy/instapy/unfollow_util.py", line 761, in dialog_username_extractor
if person and hasattr(person, 'text') and person.text:
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webelement.py", line 76, in text
return self._execute(Command.GET_ELEMENT_TEXT)['value']
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webelement.py", line 628, in _execute
return self._parent.execute(command, params)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 320, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=68.0.3440.106)
(Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 4.15.0-33-generic x86_64)

New issue @uluQulu :)

[uluQulu]

There is a new event listener attached to the page which prevents to navigate back 😂😂😂

I will have time only at night to look for a solution.
Maybe @CharlesCCC can help you, too 👉🏼

All of the other errors popped up this morning are of this sort.

---

Cheers 😁

[fgisslen]

Any solutions yet?

[CharlesCCC]

Can you please post your full config here as well .(without password of course)

[fgisslen]

@CharlesCCC

import os
import sys
import time
from tempfile import gettempdir
from time import sleep
from selenium.common.exceptions import NoSuchElementException
from instapy import InstaPy
from proxy_extension import create_proxy_extension
proxy = 'xxx'
port = xxx
insta_username = 'xxx'
insta_password = 'xxx'
session = InstaPy(username=insta_username, password=insta_password, headless_browser=True, nogui=True, multi_logs=True, bypass_suspicious_attempt=True, proxy_address=proxy, proxy_port=port)
try:
session.login()
session.set_relationship_bounds(enabled=False, potency_ratio=-1.21, delimit_by_numbers=True)
session.set_user_interact(amount=5, randomize=True, percentage=50, media='Photo')
session.set_do_follow(enabled=True, percentage=20)
session.like_by_tags(['xxx','#xx','xx'], amount=30)
sleep(90)
session.like_by_feed(amount=30, randomize=True, unfollow=False, interact=False)
sleep(90)
session.set_do_like(enabled=True, percentage=100)
session.interact_user_followers(['xxx,'xxx'], amount=30, randomize=True)
session.follow_user_followers(['xxx','xxx'], amount=40, randomize=False, interact=True)
session.follow_likers (['xxx','xxx'], photos_grab_amount=10, follow_likers_per_photo = 3,randomize=True, sleep_delay=600, interact=True)
sleep(90)
session.follow_commenters(['xxx','xxx'], amount=30, daysold=30,max_pic = 100, sleep_delay=600, interact=True)
except Exception as exc:
if isinstance(exc, NoSuchElementException):
file_path = os.path.join(gettempdir(), '{}.html'.format(time.strftime('%Y%m%d-%H%M%S')))
with open(file_path, 'wb') as fp:
fp.write(session.browser.page_source.encode('utf8'))
print('{0}\nIf raising an issue, please also upload the file located at:\n{1}\n{0}'.format(
'*' * 70, file_path))
raise
finally:
session.end()

[CharlesCCC]

@fgisslen I believe you had similar problem with #2745 and please checkout @uluQulu 's fix here https://github.com/timgrossmann/InstaPy/issues/2745#issuecomment-415974637

let us know how it works after made the change.

[RalliPi]

I can confirm that it's not working with all the changes applied (latest master).
The Problem seems to be, that buttons in dialog_username_extractor are stale after some simulated follows (or follows in general). But I couldn't encounter why...
A temporary solution that seems to work for now (for me at least) is to wrap the contents of the for loop inside this method into a try except pass block...

[uluQulu]

Hey there
@fgisslen @CharlesCCC
I have inspected the problem early morning, the problem is found.
IG overwrites its history to navigate back limit of 1. How it makes a problem?
To get the user ID from the profile page of the user after clicking the user link in dialog, it opens FeedPage and that page has no any info about user ID so it reloads the page
location.reload()
in the first catched WebDriverException and now the page has been navigate to itself with the GET method and now the page is ProfilePage and it has the user ID to grab. Fine.
If you noticed it

• dialog page>> navigate to user page >> re-navigate to it (_reload_) >> window.history.go(-1)

That's where problem lies. IG does not allow. These messages from the in-page scripts says it, too.
"_A history supports only one prompt at a time_"
"_ ignores the history prop. To use a custom history, use import { Router } instead of import { BrowserRouter as Router }._"
"_Hash history go(n) causes a full page reload in this browser_"
etc.

You can notice the same behavior in normal Google Chrome tab.
Navigate to a user page and open the dialog, click on a user link and there reload the page and press (Alt + LeftDirectionKey) == it will ignore the action and will stay on that stucked page for hours.

Solution?
I can produce a solution now since the cause is clear and I'll share it with you once finds an optimal solution.

---

Cheers 😁

[uluQulu]

@fgisslen
Here is the fix https://github.com/uluQulu/InstaPy/commit/99774d65deaf1d44660971eb95a9c66bcdd43ef2 [THIS IS INTENDED FOR THE STRUCTURE IN MY PR]

Although it is written for the structure in my PR, the procedure is the same.

Well, I wrote it for you.

To apply the FIX for the _current InstaPy files_, do this

1-) add these 2 _definitions_ to the _end_ of util.py file
find_user_id
new_tab

2-) add these imports at the _top_ of unfollow_util.py file

from .util import find_user_id
from .util import new_tab

3-) in unfollow_util.py, find and replace all of these lines

                dialog_address = "//div[text()='Followers' or text()='Following']/../../following-sibling::div"
                dialog = browser.find_element_by_xpath(dialog_address)
                user_link_in_dialog = dialog.find_element_by_xpath(
                                "//a[contains(@href,'{}')]".format(person))
                click_element(browser, user_link_in_dialog)
                sleep(2)
                userid = browser.execute_script("return window._sharedData.entry_data.ProfilePage[0].graphql.user.id")

with

                user_link = "https://www.instagram.com/{}/".format(person)
                with new_tab(browser):
                    web_adress_navigator(browser, user_link)
                    userid = find_user_id(browser, person, logger)

4-) remove this line from unfollow_util.py

browser.execute_script("window.history.go(-1)")

UPDATED STEPS

5-) import these at the top of util.py file

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException

6-) add this new _definition_ to the end of in util.py file
explicit_wait()

7-) import this at the top of unfollow_util.py file

from .util import explicit_wait

8-) in unfollow_util.py file add this new line

simulated_list.extend(quick_follow)
# declare the dialog box once again after the DOM change
explicit_wait(browser, "VOEL", (dialog_address, "XPath"), logger)   # ADD THIS LINE
dialog = browser.find_element_by_xpath(dialog_address)

9-) add these lines before quick_index = random.randint(0, len(buttons)-1) line

                if channel == "Follow":   # ADD THIS LINE
                    buttons = dialog.find_elements_by_xpath(   # ADD THIS LINE
                        "//button[text()='Follow']")   # ADD THIS LINE
                elif channel == "Unfollow":   # ADD THIS LINE
                    buttons = dialog.find_elements_by_xpath(   # ADD THIS LINE
                        "//button[text() = 'Following']")   # ADD THIS LINE
                quick_index = random.randint(0, len(buttons)-1)

It will get buttons again after DOM changes

_If something works unexpectedly, please let me know 👋🏼_

---

Cheers 😁

[fgisslen]

@uluQulu
Thank you my friend! I will try it now!

[uluQulu]

UPDATE: added new steps

@fgisslen @CharlesCCC @HCWc @cippoco @jat-x and others coming into this thread,

• 5th, 6th, 7th, 8th and 9th steps are new. Apply it if you have not.
• Apply it correctly and please give a feedback 🙂
• Really apply it correctly, indentation is so IMPORTANT
• If you have not applied prior steps YET, then apply them too

I hope it will solve all of the problems raised lately out of timeouts in _slower connections_, _laggish proxy sessions_, and low-memory devices in which page load took more time than expected.

---

Cheers 😁

[sunnygill]

@uluQulu I am still facing the issues after syncing to latest from master and applying all the 7 steps

[sunnygill]

Traceback (most recent call last): File "testfollow.py", line 352, in <module> followUserFollowers() File "stufffollow.py", line 114, in followUserFollowers session.follow_user_followers(settingsstuff.getRandomUsers(40), amount=10, randomize=True, interact=True, sleep_delay=140) File "/local/mnt/workspace/sgill/build218/InstaPy/instapy/instapy.py", line 2052, in follow_user_followers self.logfolder) File "/local/mnt/workspace/sgill/build218/InstaPy/instapy/unfollow_util.py", line 924, in get_given_user_followers channel, logger, logfolder) File "/local/mnt/workspace/sgill/build218/InstaPy/instapy/unfollow_util.py", line 712, in get_users_through_dialog quick_username = dialog_username_extractor(quick_button) File "/local/mnt/workspace/sgill/build218/InstaPy/instapy/unfollow_util.py", line 760, in dialog_username_extractor if person and hasattr(person, 'text') and person.text: File "/usr2/sgill/.local/lib/python3.4/site-packages/selenium/webdriver/remote/webelement.py", line 68, in text return self._execute(Command.GET_ELEMENT_TEXT)['value'] File "/usr2/sgill/.local/lib/python3.4/site-packages/selenium/webdriver/remote/webelement.py", line 461, in _execute return self._parent.execute(command, params) File "/usr2/sgill/.local/lib/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute self.error_handler.check_response(response) File "/usr2/sgill/.local/lib/python3.4/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document (Session info: headless chrome=67.0.3396.99) (Driver info: chromedriver=2.39.562737 (dba483cee6a5f15e2e2d73df16968ab10b38a2bf),platform=Linux 3.13.0-151-generic x86_64)

[uluQulu]

@sunnygill
If you have really applied 7 of them you should get lots of errors. Cos I just downloaded the current state of repo and added those changes myself and saw that I forgot to mention some imports like in 5th step.
Anyways..
Everything worked properly once again.

If any of you having problems after applying all of the steps above and wants to troubleshoot it more, let me know.

---

Cheers 😁

[jat-x]

I just try solutions with 5th, 6th, 7th, 8th and 9th steps/

Traceback (most recent call last):
File "C:\Users\Марина\Documents\GitHub\InstaPy\NEWsetup.py", line 118, in
from instapy import InstaPy
File "C:\Users\Марина\Documents\GitHub\InstaPyinstapy__init__.py", line 2, in
from .instapy import InstaPy
File "C:\Users\Марина\Documents\GitHub\InstaPyinstapyinstapy.py", line 44, in
from .unfollow_util import get_given_user_followers
File "C:\Users\Марина\Documents\GitHub\InstaPyinstapy\unfollow_util.py", line 815
user_link = "https://www.instagram.com/{}/".format(person)
^
IndentationError: unexpected unindent

[uluQulu]

@jat-x
You have done great!
Just take care of indentation near line 815 in unfollow_util.py file.

---

Cheers 😁

[jat-x]

@uluQulu
Traceback (most recent call last):
File "C:\Users\Марина\Documents\GitHub\InstaPy\NEWsetup.py", line 170, in
session.follow_user_followers(list_fuf, amount=amount_fuf, randomize=random_fuf, sleep_delay=120)
File "C:\Users\Марина\Documents\GitHub\InstaPyinstapyinstapy.py", line 2097, in follow_user_followers
followed = self.follow_by_list(person, self.follow_times, sleep_delay, interact)
File "C:\Users\Марина\Documents\GitHub\InstaPyinstapyinstapy.py", line 733, in follow_by_list
self.logfolder)
File "C:\Users\Марина\Documents\GitHub\InstaPyinstapy\unfollow_util.py", line 608, in follow_given_user
follow_restriction("write", acc_to_follow, None, logger)
File "C:\Users\Марина\Documents\GitHub\InstaPyinstapy\unfollow_util.py", line 1114, in follow_restriction
simulated_list.extend(quick_follow)
NameError: name 'simulated_list' is not defined

[uluQulu]

@jat-x
If you find it hard to apply the changes manually, just download my fork- https://github.com/uluQulu/InstaPy and put your quickstart script into it.

You will have to install plyer module from pip else than that everything is the same.

---

Cheers 😁

[cippoco]

@uluQulu
dear, i've update to the last version of selenium, download your last fork, but still this error,

Error: INFO [2018-08-31 16:43:38] [...] Session started!
INFO [2018-08-31 16:45:30] [...] Logged in successfully!
INFO [2018-08-31 16:49:24] [...] Starting to follow user Followers..
INFO [2018-08-31 16:49:24] [...] User '...' [1/88]
INFO [2018-08-31 16:50:56] [...] Simulated follow : 1
INFO [2018-08-31 16:51:07] [...] --> Followed '...''!
INFO [2018-08-31 16:53:03] [...] Timed out with failure while explicitly waiting until visibility of element located!
b'Message: \n'
Traceback (most recent call last):
File "quickstart.py", line 8, in
session.follow_user_followers(['...'], amount=5, sleep_delay=36, randomize=True, interact=False)
File "C:\Users\Administrator\Desktop\InstaPy-masterinstapyinstapy.py", line 2476, in follow_user_followers
self.logfolder)
File "C:\Users\Administrator\Desktop\InstaPy-masterinstapy\unfollow_util.py", line 859, in get_given_user_followers
channel, jumps, logger, logfolder)
File "C:\Users\Administrator\Desktop\InstaPy-masterinstapy\unfollow_util.py", line 667, in get_users_through_dialog
dialog = browser.find_element_by_xpath(dialog_address)
File "C:\Program Files (x86)\Python\Python36-32lib\site-packages\selenium\webdriver\remote\webdriver.py", line 393, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Program Files (x86)\Python\Python36-32lib\site-packages\selenium\webdriver\remote\webdriver.py", line 966, in find_element
'value': value})['value']
File "C:\Program Files (x86)\Python\Python36-32lib\site-packages\selenium\webdriver\remote\webdriver.py", line 320, in execute
self.error_handler.check_response(response)
File "C:\Program Files (x86)\Python\Python36-32lib\site-packages\selenium\webdriver\remoteerrorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[text()='Followers' or text()='Following']/../../following-sibling::div"}
(Session info: chrome=68.0.3440.106)
(Driver info: chromedriver=2.41.578737 (49da6702b16031c40d63e5618de03a32ff6c197e),platform=Windows NT 10.0.14393 x86_64)

[uluQulu]

DUPLICATED comment: https://github.com/timgrossmann/InstaPy/pull/2782#issuecomment-416997222
@cippoco
Unfortunately I cannot see what is going on in your browser window.
Try to help me understand the problem by answering these questions 🙋🏼‍♂️

• Does it open dialog box properly? ["_Followers_" dialog box of the first user in follow_user_followers()]
• Does it scroll down properly?
• Does it do simulated follow? [clicking on a random follow button- user A]
• Does it open a new tab?
• Does it navigate to the profile page of user A? [in the newly opened tab] [and then wait a second to get the user ID]
• Does it close the newly opened tab successfully?
• Does it continue to scroll down properly?

It must crash in one of the steps above. Which one it is? [accurate answer please]

---

Cheers 🤓

[uluQulu]

Also @cippoco
Replace browser.close() with browser.execute_script("window.close()")

[cippoco]

@uluQulu
Nothing, if i change the browser.close() it seems work for a bit longer , but when it close the tab of the user to follow, sometimes it return on the page of the user in follow_user_followers() whit the user popup to scroll and simulate, sometime return on that page whitout the popup so it crash:

1. Does it open dialog box properly? ["Followers" dialog box of the first user in follow_user_followers()]
   Yes
2. Does it scroll down properly?
   Yes
3. Does it do simulated follow? [clicking on a random follow button- user A]
   Yes
4. Does it open a new tab?
   Yes
5. Does it navigate to the profile page of user A? [in the newly opened tab] [and then wait a second to get the user ID]
   Yes
6. Does it close the newly opened tab successfully?
   Yes
7. Does it continue to scroll down properly?
   No, sometimes when it close the tab, it return on the page of the firsth user in the list follow_user_followers(), then it crashes whit the error attached before:

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element:

[uluQulu]

Well, @cippoco, I can't get it how it can sometimes lose the pop up window.

Due to the absence of accurate feedback provided by the affected users, I could not get the idea of "_What is going on in there?_" and it has been days I requested a feedback.

Thanks, @cippoco although that information is uncomplete but you at least tried. 🙂

That's why I decided to remove that idea of working with tabs and you can chill now cos there will be no any issues anymore on that matter because I temporarily will disable getting user ID for follows from dialog box. @CharlesCCC

I think, in future people can achieve it efficiently using other methods.
Good luck!

---

Cheers 😁

[cippoco]

Dear @uluQulu thanks for your interest!