Instapy: Database locked when running multiple scripts in multiple terminals.

[joanroig]

Expected Behavior

When running more than one script in the same system, the database should be accessible by all instances. The problem appears when a script is running follow by list, it seems to keep the database unaccessible for a long time:
session.follow_by_list(subarray, times=1)

Current Behavior

If I try to run multiple scripts at once (following scripts, like follow 50 users from a list), I usually see 'Dap! Error occurred with follow Restriction: Database locked'

Possible Solution (optional)

I've been running the same scripts for months with no problem, so probably there is change that affects the database lock time.

InstaPy configuration

I'm using customized scripts that work as similar as 'follow_unfollow_and_send_telegram_msg.py', so I run 4 o 5 scripts with the same behavious but not at the same time, the first one starts every day at 10:00, the second one at 10:10, etc, and since the last major update the scripts are failing to start due the database lock problem. My setup:

def setup():

    # set workspace if executed on a mac (useful for debugging)
    if platform == "darwin":
        set_workspace(path=get_workspace('mac'))
    else:
        set_workspace(path=None)

    return InstaPy(username=insta_username,
                   password=insta_password,
                   headless_browser=True,
                   multi_logs=True)

[panchoma]

Hi! Same problem for me;
I have 3 sessions opened for the same user at the same time, one is following, other is liking and the last is unfollowing.
Sometimes it works correctly, but other times it turns out the issue that the database has been blocked and the session is automatically closed.
Any ideas what to do in this case?
Previously I installed 3 instances of instapy, one for each session and each one used its own database document. But with the new update, all the sessoions use the same document, since it's no longer hosted within each instance, now the document is in the same folder for all.

Thanks in advance!

@timgrossmann
@uluQulu

[joanroig]

I've left only one instance running for the whole day with no problems, the trouble appears when multiple scripts try to access the database at once.
I have not too much time those days for going into code, and the old version of the tool is no longer working :(

[CharlesCCC]

I have experienced similar problem before. One solution will be replace the file DB SQLite with MySQL or something that is not a file based DB. (which should able to handle insert much more in parallel than SQLite). but that will require much more work, a simple working around for now is to comment out the updateactivity method so it will not insert/update for each call, but it got trade off as well. (like can't use quota superviosr etc.).

I'm wondering how does @sionking handle this issue, consider he also run InstaPy in a large scale.

[panchoma]

I found this solution, that works find in my case, proposed by @uluQulu, https://github.com/timgrossmann/InstaPy/issues/3011#issuecomment-424718897

Having instapy's instances per sessions and adding this two lines to the programs:

from instapy import Settings

Settings.database_location = "./db/instapy_{}.db".format(insta_username)

creates a database document in the folder of each instance.

3011

[joanroig]

Thanks, I think this problem & solution will be useful to be included in advanced configuration or troubleshoot documentation.
I'm also interested on how @sionking handles the issue, for now I'll try the separate database solution, thanks @panchoma.
One more question for @uluQulu, in the past I had problems with some logs like the follow restriction file, I messed it up due a power outage, so the JSON was corrupted. These logs are not really used internally, so the database is the only real file used when checking follow restrictions and so, I'm right?

[joanroig]

Working fine on my low-power raspberry, so I close the issue.

[ishandutta2007]

database_location

@panchoma why haven't you checked in this ?