Instapy: Each action printed twice in logs

[bongo48]

Expected Behavior

Printing all actions once

Current Behavior

All actions are printed twice: one with a date stamp and one without.
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
INFO [2018-11-07 17:35:37] [insta_nick] Session started!
INFO:insta_nick:Session started!
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooo

.................................................................
INFO [2018-11-07 17:36:40] [insta_nick] Logged in successfully!
INFO:insta_nick:Logged in successfully!
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

---

INFO [2018-11-07 17:36:44] [insta_nick] Starting to get the Followers data..
INFO:insta_nick:Starting to get the Followers data..
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

InstaPy configuration

## InstaPy configuration
import os
import time
from tempfile import gettempdir
from selenium.common.exceptions import NoSuchElementException
from instapy import InstaPy
session = InstaPy(username=insta_username,
password=insta_password,
headless_browser=False,
multi_logs=False)
try:
session.login()
##### clarifai api
session.set_use_clarifai(enabled = True, api_key = 'xxxxxxxxxxxxxxxxxxx')

session.unfollow_users(amount=1500, nonFollowers=True, style="RANDOM", unfollow_after=15*60*60, sleep_delay=600)        
session.set_relationship_bounds(enabled=True,
                 potency_ratio=-1.05,
                  delimit_by_numbers=True,
                   max_followers=30000,
                    max_following=7000,
                     min_followers=35,
                       min_following=77)
session.set_do_follow(enabled=True, percentage=90)
session.set_user_interact(amount=1, randomize=True, percentage=90, media='Photo')
session.set_do_comment(enabled=True, percentage=100)
session.set_comments([''])
session.set_delimit_commenting(enabled=True, max=32, min=2)
session.set_do_like(enabled=True, percentage=90)
session.clarifai_check_img_for (['nsfw', 'people', 'text', 'graphic design', 'pet', 'animal', 'food', 'women', 'low-quality', 'words', 'logo', 'fashion', 'child', 'baby', 'man', 'girl'])
session.clarifai_check_img_for (['landscape', 'travel', 'sunset', 'ocean', 'nature', 'road', 'sky', 'sun', 'beach'], comment=True, comments=[u'Wow! :ok_hand:'])

###### unfollow
session.unfollow_users(amount=1500, nonFollowers=True, style="RANDOM", unfollow_after=15*60*60, sleep_delay=600)    

users_to_follow1 = session.grab_followers(username="xxxxxxxxxx", amount=300, live_match=True, store_locally=True)    
session.follow_by_list(users_to_follow1[:250], times=1, sleep_delay=100, interact=True)

[sionking]

you declared logger twice

[bongo48]

you declared logger twice

Thanks for the quick response. Any idea how can I declare just once? I uninstalled python and installed again, did new copy of instapy with a new quickstart configuration it didn't help...

[converge]

More people had told me about it, sounds like a bug, but I can't reproduce it.

[kiankji]

+1

[pez5001]

I was able to get around this by setting show_logs=False when creating the InstaPy session object.
Also configured my own logging using logging.basicConfig() before importing any instapy code in my runner script, but I'm not 100% sure that had any effect.

[bongo48]

show_logs=Falsedoesn't help in my case. If you explain how to configure logging with logging.basicConfig() then I can try too.

[sionking]

is it looking like that:
INFO [2018-11-10 23:58:49] [xxx] --> b'flav1one'
INFO:instapy.print_log_writer:--> b'flav1one

?

[bongo48]

It looks like that:

INFO [2018-11-07 17:36:40] [insta_nick] Logged in successfully!
INFO:insta_nick:Logged in successfully!

[converge]

tried it in many ways, Im unable to reproduce it.

[pez5001]

Here's the logging config I used:

    logging.basicConfig(
        level=logging.DEBUG,
        format="%(asctime)s %(levelname)7s %(funcName)16s(): %(message)s",
        datefmt="%Y-%m-%d %H:%M:%S",
    )
    for l in ("urllib3", "selenium", "requests", "urllib3.util.retry", "urllib3.connectionpool", "urllib3.util.retry"):
        logging.getLogger(l).setLevel(logging.WARNING)

[converge]

@pez5001 could you propose a PR ?

[bongo48]

I've noticed that when you switch off Clarifai then it prints correctly.

[converge]

@bongo48 sounds like a good start

[converge]

I think I found the issue, and dont know how to fix it yet.

if you remove
from instapy.util import smart_run

from quickstart.py and use the old try/catch, it will run.

Couldn't find the reason why import smart_run is duplicating the logs. Could you take a look ? @uluQulu , maybe you have some idea.

*Im just able to see the issue on a virtualenv.

UPDATE: now even without the smart_run import the duplicate log is popping up. We need help to find what's causing the bug.

[hw711]

I'm having same issue on one of my machines but not on my other machine.

[hw711]

the second copy (the one without the date) would print even when show_logs=False

[uluQulu]

@bongo48, @hw711, @kiankji, @pez5001
Can you write about your setups a bit?
E.g.,

• [x] I run it on my Windows 10 and with python 3.7.0 directly [NO PROBLEMS]
• [x] Also ran it on virtualenv (_v16.1.0_) on the same machine with the same python [NO PROBLEMS]

This most prob is a environment, platform dependant issue and can be fully tracked only after good feedbacks from you having the issue.

Also, a little information about your extra imports into the running files could be great. And if you use threads, schedulers, etc.

We're debugging it with @converge at Discord #development, you can join up to the discussion 💭

---

Cheers 😁

[hw711]

the machine having issue is a google cloud platform vm running windows server 2012 R2 Datacenter with python 3.7.1. Not using any special import.

[bongo48]

Windows 10 - Python 3.7.1 - pip packages versions:

beautifulsoup4 4.6.3
certifi 2018.10.15
chardet 3.0.4
clarifai 2.4.1
Click 7.0
configparser 3.5.0
easyprocess 0.2.3
emoji 0.5.1
future 0.17.1
gitdb2 2.0.5
gitpython 2.1.11
google-cloud 0.34.0
googleapis-common-protos 1.5.5
grep 0.3.2
grpcio 1.16.0
idna 2.7
instagram-py 0.0.1
jsonschema 2.6.0
packaging 18.0
pillow 5.3.0
pip 18.1
pip-review 1.0
pip-tools 3.1.0
plyer 1.3.1
protobuf 3.6.1
pyparsing 2.3.0
pyvirtualdisplay 0.2.1
requests 2.20.1
selenium 3.141.0
setuptools 39.0.1
six 1.11.0
smmap2 2.0.5
urllib3 1.24.1
wheel 0.32.2

[visualheroes]

Hey for me it started when i upgraded every package in the requirement list (manually as i use a virtualenv).
At that time i was running a not-up-to-date instapy and the only change was the upgraded packages (same script).
Running on OSX, Python 3.7, all latest packages, my script do not use smart_run.

[visualheroes]

A quick freeze of my packages:

certifi==2018.10.15
chardet==3.0.4
clarifai==2.4.1
configparser==3.5.0
EasyProcess==0.2.3
emoji==0.5.1
future==0.17.1
gitdb2==2.0.5
GitPython==2.1.11
googleapis-common-protos==1.5.5
grpcio==1.16.0
idna==2.7
jsonschema==2.6.0
Pillow==5.3.0
plyer==1.3.1
protobuf==3.6.1
PyVirtualDisplay==0.2.1
requests==2.20.1
selenium==3.141.0
six==1.11.0
smmap2==2.0.5
urllib3==1.24.1

Hope it helps.

[uluQulu]

Haha 😂
Thanks @visualheroes 👏🏼👏🏼
The problem is found!

Just upgrade* grpcio package from pip to 1.16.1 [from 1.16.0] ⤴

pip install --upgrade grpcio
# or
pip install grpcio==1.16.1

*assumes that 1.16.1 is available for your system;

Details

@bongo48

I've noticed that when you switch off Clarifai then it prints correctly.

It's cos grpcio package is a dependency of the clarifai package and in your system it was being triggered only by _Clarifai_.

Extra

Well, I have also found the original discussion held at GitHub channel (_15 days ago_) of grpc 🌝 package which explains this problem in details by @attila123 and people offer the solution(s) that ends up to be released in its stable version-1.16.1
See it here: https://github.com/grpc/grpc/issues/17135

---

@converge it was really related to basicConfig and __init__ 🐬

Cheer 😁

[visualheroes]

It was grpcio in deed. Everything is back to normal now. ;)

Great work @uluQulu!

[converge]

congratulations @uluQulu !! happy that we have you on the team 😃 😃 😃

I spent a lot of time debugging it, far the hardest debug I ever did.

[bongo48]

Great, thank you masters :)

[gitpatrickhub]

This fix worked for me on Raspberry Pi 3, as well. Thanks for the fix.

[julienteyssier]

Hi, I have the same error message but the action " pip install --upgrade grpcio" did not help me at all :/

Here is my quickstart.py code which is really simple : `""" Quickstart script for InstaPy usage """

imports

from instapy import InstaPy
from instapy.util import smart_run

login credentials

insta_username = '*'
insta_password = ''

get an InstaPy session!

set headless_browser=True to run InstaPy in the background

session = InstaPy(username=insta_username,
password=insta_password,
headless_browser=False)

with smart_run(session):
""" Activity flow """
session.login()
session.set_relationship_bounds(enabled=True,
potency_ratio=None,
delimit_by_numbers=True,
max_followers=1200000,
max_following=3000,
min_followers=10,
min_following=120,)

session.set_user_interact(amount=3,
percentage=100,
randomize=True,
media='Photo')
session.follow_likers (['*'], photos_grab_amount = 6, follow_likers_per_photo = 400, randomize=False, sleep_delay=600, interact=True)`

And I have got this error message in the terminal : File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen **response_kw) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen **response_kw) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen **response_kw) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen _stacktrace=sys.exc_info()[2]) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/util/retry.py", line 398, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=61527): Max retries exceeded with url: /session/37bb6f34d37cd4f03d555a134964356a/execute (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10610c2b0>: Failed to establish a new connection: [Errno 61] Connection refused'))
I am in macOS, if someone have an idea, he would be my god !

[julienteyssier]

I think it's related to 'session.follow_likers' because If I use other scripts without it, it works perfectly

[hw711]

@AlokinVetob, I don't what you are experiencing is the same issue as original issue of this thread. your issue seems (I'm guessing since I didn't bother to look up the the source code) like a try catch block trying to retry when catching first exception then catch another exception.

[Areahints]

My Error Message

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connection.py", line 159, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/util/connection.py", line 80, in create_connection
    raise err
  File "/usr/local/lib/python3.5/dist-packages/urllib3/util/connection.py", line 70, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.5/http/client.py", line 1106, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1151, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1102, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
    self.send(msg)
  File "/usr/lib/python3.5/http/client.py", line 877, in send
    self.connect()
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connection.py", line 181, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connection.py", line 168, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0722416cc0>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "care.py", line 48, in <module>
    session.like_by_locations(["7088402","246584565","288240960"], amount=200) 
  File "/usr/local/lib/python3.5/dist-packages/instapy/instapy.py", line 1309, in like_by_locations
    skip_top_posts)
  File "/usr/local/lib/python3.5/dist-packages/instapy/like_util.py", line 83, in get_links_for_location
    web_address_navigator(browser, location_link)
  File "/usr/local/lib/python3.5/dist-packages/instapy/util.py", line 1029, in web_address_navigator
    current_url = get_current_url(browser)
  File "/usr/local/lib/python3.5/dist-packages/instapy/util.py", line 1562, in get_current_url
    current_url = browser.execute_script("return window.location.href")
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 636, in execute_script
    'args': converted_args})['value']
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 319, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/remote_connection.py", line 374, in execute
    return self._request(command_info[0], url, body=data)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/remote_connection.py", line 397, in _request
    resp = self._conn.request(method, url, body=body, headers=headers)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/request.py", line 72, in request
    **urlopen_kw)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/request.py", line 150, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/poolmanager.py", line 323, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.5/dist-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=57088): Max retries exceeded with url: /session/94d2d12bd2a43b388f721d05d48cb169/execute (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0722416cc0>: Failed to establish a new connection: [Errno 111] Connection refused',))

[rubinho101]

Hi, after two days of working with InstaPy on Mac I keep getting this issue. I tried the solutions mentioned above but it didnt work out for me. Any ideas?

File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/urllib3/util/retry.py", line 399, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=49813): Max retries exceeded with url: /session/1/execute (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x104d545d0>: **Failed to establish a new connection: [Errno 61] Connection refused**',))

[DrunkenHusky]

Just for all of you who still get this error:
Use the solution from @uluQulu
pip install --upgrade grpcio
#or
pip install grpcio==1.16.1

And delete the workspace from instapy. It will be shown when you start the script or if you have configured in the quickstart.py-file you can check it out there:

# set workspace folder at desired location (default is at your home folder)
set_workspace(path=NONE)

You also can just change the workspace to a new location and it will also work like a charme!
# set workspace folder at desired location (default is at your home folder)
set_workspace(path='/root/instaPy/nameofinstagramaccount')