Instapy: [SOLVED] Handle all seleniums errors (crashes, timeouts, connections errors, etc)

[ghost]

InstaPy is ending work when we get errors like:

• WebDriverException: Message: unknown error: session deleted because of page crash,
• TimeoutException: Message: timeout
• NoSuchWindowException: Message: no such window: target window already closed
• ConnectionResetError: [Errno 104] Connection reset by peer
• etc

But we can handling all this errors and continuing our work.

My solution for instapy.py

1. Create a function:

def browser_request(self, func, args=[], kwargs={}):
    attempt = 1
    resp = None
    while not resp and attempt <= 6:
        try:
            resp = func(*args, **kwargs)

        # If error is NoSuchElementException - raise it
        except NoSuchElementException as s:
            raise s

        except (
            WebDriverException,
            TimeoutException,
            NoSuchWindowException,
            ConnectionResetError,
        ) as s:

            if attempt == 6:
                raise s

            self.logger.error('Selenium error %s: %s (attempt %s)' % (type(s).__name__, s, attempt))

            if attempt in range(2, 6):
                sleep_time = 90
            else:
                sleep_time = 30
            sleep(sleep_time)

            # Close old session
            self.browser = args[0]
            browser_quit = self.browser.quit()
            if self.nogui:
                self.display.stop()

            # Start new session
            if self.nogui:
                self.display = Display(visible=0, size=(800, 600))
                self.display.start()
            self.set_selenium_local_session()
            self.login()
            args[0] = self.browser

        attempt += 1

    return resp

1. All requests to selenium do though this function, example:

    def interact_by_users(self,
                          usernames,
                          amount=10,
                          randomize=False,
                          media=None):
        """Likes some amounts of images for each usernames"""
        if self.aborting:
            return self

        total_liked_img = 0
        already_liked = 0
        inap_img = 0
        commented = 0
        followed = 0

        usernames = usernames or []
        # Reset like counter for every username
        liked_img = 0

        for index, username in enumerate(usernames):
            self.logger.info(
                'Username [{}/{}]'.format(index + 1, len(usernames)))
            self.logger.info('--> {}'.format(username.encode('utf-8')))

            try:
                links = self.browser_request(
                    get_links_for_username,
                    [
                        self.browser,
                        username,
                        amount,
                        self.logger,
                        randomize,
                        media,
                    ]
                )
            except NoSuchElementException:
                self.logger.error('Element not found, skipping this username')
                continue

            if links is False:
                continue

            # Reset like counter for every username
            liked_img = 0

            # Will we follow this user?
            following = random.randint(0, 100) <= self.follow_percentage

            for i, link in enumerate(links):
                # Check if target has reached
                if liked_img >= amount:
                    self.logger.info('-------------')
                    self.logger.info("--> Total liked image reached it's "
                                     "amount given: {}".format(liked_img))
                    break

                self.logger.info('Post [{}/{}]'.format(liked_img + 1, amount))
                self.logger.info(link)

                try:
                    inappropriate, user_name, is_video, reason = self.browser_request(
                        check_link,
                        [
                            self.browser,
                            link,
                            self.dont_like,
                            self.ignore_if_contains,
                            self.ignore_users,
                            self.username,
                            self.like_by_followers_upper_limit,
                            self.like_by_followers_lower_limit,
                            self.logger
                        ]
                    )

                    if not inappropriate:

                        if (self.do_follow and
                            username not in self.dont_include and
                            following and
                            self.follow_restrict.get(
                                username, 0) < self.follow_times):

                            followed += self.browser_request(
                                follow_user,
                                [
                                    self.browser,
                                    self.follow_restrict,
                                    self.username,
                                    username,
                                    self.blacklist,
                                    self.logger,
                                    self.logfolder
                                ]
                            )
                        else:
                            self.logger.info('--> Not following')
                            sleep(1)

                        liking = random.randint(0, 100) <= self.like_percentage
                        if self.do_like and liking:
                            liked = self.browser_request(
                                like_image,
                                [
                                    self.browser,
                                    user_name,
                                    self.blacklist,
                                    self.logger,
                                    self.logfolder,
                                ]
                            )
                        else:
                            liked = True

                        if liked:
                            total_liked_img += 1
                            liked_img += 1
                            checked_img = True
                            temp_comments = []
                            commenting = random.randint(
                                0, 100) <= self.comment_percentage

                            if self.use_clarifai and (following or commenting):
                                try:
                                    checked_img, temp_comments = self.browser_request(
                                        check_image,
                                        [
                                            self.browser,
                                            self.clarifai_api_key,
                                            self.clarifai_img_tags,
                                            self.logger,
                                            self.clarifai_full_match
                                        ]
                                    )
                                except Exception as err:
                                    self.logger.error(
                                        'Image check error: {}'.format(err))
                            if (self.do_comment and
                                user_name not in self.dont_include and
                                checked_img and
                                    commenting):

                                if temp_comments:
                                    # use clarifai related comments only!
                                    comments = temp_comments
                                elif is_video:
                                    comments = (self.comments +
                                                self.video_comments)
                                else:
                                    comments = (self.comments +
                                                self.photo_comments)
                                commented += self.browser_request(
                                    comment_image,
                                    [
                                        self.browser,
                                        user_name,
                                        comments,
                                        self.blacklist,
                                        self.logger,
                                        self.logfolder
                                    ]
                                )
                            else:
                                self.logger.info('--> Not commented')
                                sleep(1)
                        else:
                            already_liked += 1

                    else:
                        self.logger.info(
                            '--> Image not liked: {}'.format(reason.encode('utf-8')))
                        inap_img += 1
                except NoSuchElementException as err:
                    self.logger.info('Invalid Page: {}'.format(err))

            if liked_img < amount:
                self.logger.info('-------------')
                self.logger.info("--> Given amount not fullfilled, image pool "
                                 "reached its end\n")

        self.logger.info('Liked: {}'.format(total_liked_img))
        self.logger.info('Already Liked: {}'.format(already_liked))
        self.logger.info('Inappropriate: {}'.format(inap_img))
        self.logger.info('Commented: {}'.format(commented))

        self.liked_img += liked_img
        self.already_liked += already_liked
        self.inap_img += inap_img
        self.commented += commented

        return self

So if we will get an error - script will not raise exception.
It will try to wait for 10 or 90 seconds (usefull when we got connetion error) and start new selenium session (usefull if selenium was crushed).

Maybe someone have better solution?

[sionking]

First, Thank you !
Second it was in my todo list for a while.
3'rd please check loop on valid connection, for some exception it is best solution.

import socket
def is_connected():
    try:
        # connect to the host -- tells us if the host is actually
        # reachable
        socket.create_connection(("www.google.com", 80))
        return True
    except OSError:
        pass
    return False

Best solution I found.

[sionking]

BTW 4 of my accounts stopped working today at the same time, I thought the got the bot !
But the true is that instagram was not live for couple of minutes

[MrBeardedGuy]

Can one of you guys create a PR with these fixes???? 😀

[riccardorighettiphoto]

Hello,
thanks a lot for this fix, I have experienced many timeout errors caused by selenium in the last month. I am testing your fix on a git branch.
Bye.

[tsirolnik]

Is there a PR for this? I get this error constantly

[Tjorriemorrie]

Just remember that if there is an error, it means something is wrong. To ignore it will make the library very unstable. It's much better to change your script/quickstart to have a retry loop. You can also then handle the errors per specific action. There are a lot of errors that should be raised, instead of handled, e.g. when IG changes something the library needs to be changed to cater for it. These errors should be handled at the higher level.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this problem still occurs, please open a new issue