Instapy: Issue with wrong chromedriver format

[ianjuma]

Maybe you should allow people to fetch platform specific chromedrivers (from chrome sites) instead of distributing one. I had an issue with Exec format being wrong ?

Traceback (most recent call last):
  File "./quickstart.py", line 12, in <module>
    InstaPy(username='test', password='test')\
  File "/home/synod/AT/cronos/InstaPy/instapy/instapy.py", line 31, in __init__
    self.browser = webdriver.Chrome('./assets/chromedriver', chrome_options=chrome_options)
  File "/usr/local/lib/python2.7/dist-packages/selenium-2.53.6-py2.7.egg/selenium/webdriver/chrome/webdriver.py", line 62, in __init__
    self.service.start()
  File "/usr/local/lib/python2.7/dist-packages/selenium-2.53.6-py2.7.egg/selenium/webdriver/common/service.py", line 64, in start
    stdout=self.log_file, stderr=self.log_file)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 8] Exec format error

I guess this will be easy with the package being on PyPI

[ianjuma]

looks like the page has changes - also

[6/93]
Invalid Page: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//article/div[2]/div[1]/ul[1]/li[1]"}
  (Session info: chrome=57.0.2987.110)
  (Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.4.0-71-generic x86_64)

might be easy to just distribute this via PyPI?

[eduardocalazansjr]

I guess this will be easy with the package being on PyPI

Got the same issue under my ubuntu droplet in DO.
Chromedriver is specified in instapy.py file, put yours in PATH and use it

looks like the page has changes - also

Can't reproduce under OSx and Ubuntu, but i'll try to find a better approach to fetch the element.

[timgrossmann]

@vlntdds
I think the problem only occurs with videos...

That's why I originally had:

a_elems = browser.find_elements_by_xpath('//a[@role = "button"]')

#handle videos      
link_elem = a_elems[0] if len(a_elems) < 2 else a_elems[len(a_elems) - 1]

[timgrossmann]

@ianjuma I explicitly mentioned that you should get the fitting chromedriver for your system in the readme... Sorry

[razvanilin]

I get an error with the chrome driver too and I tried with multiple versions, all built for linux 32bit (like my server) - Tried 2.29, 2.28, 2.27, 2.10

Traceback (most recent call last):
  File "quickstart.py", line 10, in <module>
    InstaPy(username='test', password='test')\
  File "/var/www/html/instaPy/instapy/instapy.py", line 31, in __init__
    self.browser = webdriver.Chrome('./assets/chromedriver', chrome_options=chrome_options)
  File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/chrome/webdriver.py", line 62, in __init__
    self.service.start()
  File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/common/service.py", line 96, in start
    self.assert_process_still_running()
  File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/common/service.py", line 109, in assert_process_still_running
    % (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service ./assets/chromedriver unexpectedly exited. Status code was: 127

[timgrossmann]

@razvanilin Did you replace the chrome driver in the assets folder?

[razvanilin]

@timgrossmann yes, I did that. Also tried to give it execute access and 755 permissions

[timgrossmann]

Oh, ok are you running it on your sever?

[razvanilin]

Yup, I run it on a DO droplet

[timgrossmann]

Weird, I run it on a Ubuntu DO Droplet, too.
Everything works fine...

What size? Probably 512MB if yes, that could be the problem.

[razvanilin]

It's the 1GB actually. I wonder if it's because I'm running Fedora 19 on it. It's quite old now (can't find the time to upgrade).

[timgrossmann]

Ahh ok... That's weird.

Did you try to run it manually on the server?
Also installed pyvirtualdisplay?

[razvanilin]

I installed all the requirements from the txt file, I modified the quickstart script with my instagram credentials and then ran it with python3.5 quickstart.py

I got the chromedriver from here: https://chromedriver.storage.googleapis.com/index.html?path=2.29/

[eduardocalazansjr]

@razvanilin Did you uncommented the lines 27 and 28 (that deals with the virtual display) from instapy.py?

[razvanilin]

I just did that because you mentioned it and it complained about missing xvfb. I installed it and then I got the same error as above when running my script.

[timgrossmann]

Maybe just try to install the newest version of selenium and clarifai...
I changed the requirements.txt since I actually nearly always use the newest version

[razvanilin]

I'm pretty sure I have everything up-to-date since I installed everything just before trying this library. Maybe I'll just try to rebuild everything again

[timgrossmann]

Do you have all this installed?

sudo apt-get -y install python3-pip python3-dev build-essential libssl-dev libffi-dev xvfb

I just followed how-to-test-or-scrape-javascript-rendered-websites-with-python-selenium

[eduardocalazansjr]

@razvanilin try to do the following changes in instapy.py

chrome_options = Options()
chrome_options.add_argument('--dns-prefetch-disable')
chrome_options.add_argument('--no-sandbox')
service_log_path = "./assets/chromedriver.log"
service_args = ['--verbose']
self.browser = webdriver.Chrome('./assets/chromedriver', chrome_options=chrome_options, service_args=service_args, service_log_path=service_log_path)
self.browser.implicitly_wait(25) 

If the error persists, post the chromedriver.log file (located inside the assets folder)

[mareknebesar]

Have the same issue:
Invalid Page: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//article/div[2]/div[1]/ul[1]/li[1]"} (Session info: chrome=57.0.2987.133) (Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.4 x86_64)
chromedriver.log is empty.

[eduardocalazansjr]

@mareknebesar i will do a pull request ASAP.
until that, take a look at #33, i posted a GIST with the fixed code.

[mareknebesar]

@vlntdds I already did (found it a moment after posting here). Confirm that it fixes the issue, good job ;-)

[mareknebesar]

@vlntdds There might be still a bug because of different structure of the page - I am running it with set_do_comment(True, percentage=10) and in 60 images so far nothing has been commented.

[timgrossmann]

@vlntdds Are there some updates on the actual topic of this issue?
The chrome driver problem?

[razvanilin]

Well I was a bit dumb and didn't think about installing Chrome on my server. Since the chromedriver allows running chrome headlessly, you obviously need chrome installed first. I never install it on my servers and that's why I didn't think about it in the beginning.

Maybe it's worth updating the readme with this info.

I'm now facing some other issues but I believe they are related to the ones @mareknebesar encountered. I will try to fix them later today and see how it goes.

Message: no such element: Unable to locate element: {"method":"xpath","selector":"//a[@href="/<user>/followers/"]"}
  (Session info: chrome=57.0.2987.133)
  (Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.4.0-66-generic x86_64)

[ianjuma]

@timgrossmann chrome driver issue seems straight forward, missed it on the README; current issue seems to be the one @mareknebesar is experiencing; works on and off - commenting too

Seen the related gist; working now. @timgrossmann maybe just not distribute the chrome driver entirely?

[mareknebesar]

Although it is running, I still experience a lot of issues, usually between 10 - 30th the process falls down. Can't really find out what's the issue. Here are some cases from the command line for reference:

Image from: anamacedo
Link: https://www.instagram.com/p/BSiKclDht8I/?tagged=hongkong
Traceback (most recent call last):
  File "quickstart.py", line 17, in <module>
    .like_by_tags(['#hongkong', '#HK'], amount=100) \
  File "/Users/marek/GIT/InstaPy/instapy/instapy.py", line 228, in like_by_tags
    self.ignore_if_contains, self.username)
  File "/Users/marek/GIT/InstaPy/instapy/like_util.py", line 56, in check_link
    print('Description: ' + image_text)
TypeError: must be str, not NoneType

Image from: hyeyeonpark__
Link: https://www.instagram.com/p/BSiRNgQjUan/?tagged=hongkong
Traceback (most recent call last):
  File "quickstart.py", line 17, in <module>
    .like_by_tags(['#hongkong', '#HK'], amount=100) \
  File "/Users/marek/GIT/InstaPy/instapy/instapy.py", line 228, in like_by_tags
    self.ignore_if_contains, self.username)
  File "/Users/marek/GIT/InstaPy/instapy/like_util.py", line 56, in check_link
    print('Description: ' + image_text)
TypeError: must be str, not NoneType

Traceback (most recent call last):
  File "quickstart.py", line 17, in <module>
    .like_by_tags(['#hongkong', '#HK'], amount=100) \
  File "/Users/marek/GIT/InstaPy/instapy/instapy.py", line 228, in like_by_tags
    self.ignore_if_contains, self.username)
  File "/Users/marek/GIT/InstaPy/instapy/like_util.py", line 51, in check_link
    user_name = browser.execute_script("return window._sharedData.entry_data.PostPage[0].media.owner.username")
  File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 484, in execute_script
    'args': converted_args})['value']
  File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 249, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 193, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Cannot read property '0' of undefined
  (Session info: chrome=57.0.2987.133)
  (Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.4 x86_64)



md5-2e6354b38c9e41333881215c97ffab5b



Traceback (most recent call last):
  File "quickstart.py", line 17, in <module>
    .like_by_tags(['#hongkong', '#HK'], amount=100) \
  File "/Users/marek/GIT/InstaPy/instapy/instapy.py", line 228, in like_by_tags
    self.ignore_if_contains, self.username)
  File "/Users/marek/GIT/InstaPy/instapy/like_util.py", line 51, in check_link
    user_name = browser.execute_script("return window._sharedData.entry_data.PostPage[0].media.owner.username")
  File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 484, in execute_script
    'args': converted_args})['value']
  File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 249, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 193, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Cannot read property '0' of undefined
  (Session info: chrome=57.0.2987.133)
  (Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.4 x86_64)

[ianjuma]

I experience this too; I think it pops up when the link is dead.

selenium.common.exceptions.WebDriverException: Message: unknown error: Cannot read property '0' of undefined

If the posted image has been removed.

[eduardocalazansjr]

@mareknebesar Just realized that if the user left the caption empty and after the post date write an comment, it doesn't count as a caption.
Could you try the new like_util.py and send me some feedback about the first error?

I'll drive into the Cannot read property '0' of undefined now.

[eduardocalazansjr]

@ianjuma just fixed that error too, try this

[timgrossmann]

@ianjuma Hmm I thought about not distributing the chromedriver at all... But I don't think this will fix the problem, will it?
People will still try to run it as they downloaded it.

[ianjuma]

@timgrossmann at least for the os.exec wrong format error; I think it will help; then maybe the guide to fetch relevant chrome driver on the README file takes precedence/ or is highlighted.

[timgrossmann]

@ianjuma Well, let's just try it out then 😉

[mareknebesar]

@vlntdds All seems to be working fine now for me with your fix.

[hanishassim]

Hi, I'm having the 127 error code as well, anyone can shed some light on how to solve this? I'm using 1GB droplet, ubuntu 16.04, and chromedriver 2.29.

[razvanilin]

Make sure you have Chrome installed. http://askubuntu.com/questions/510056/how-to-install-google-chrome#answer-510063

[wemakefuture]

Hey, i just setup up a 32 bit version Linux 14.04 (cause its cheap 10$ p.a.)

and had the same issue installing chromium with newest chromedriver worked fine.

sudo apt-get install chromium-browser