Botocore: Clarify multithreading documentation

[dfee]

The documentation for boto3 states that:

It is recommended to create a resource instance for each thread / process in a multithreaded or multiprocess application rather than sharing a single instance among the threads / processes.(emphasis mine)

The documentation than goes on to show a code example where a session is created per thread, not a resource.

Reading through previous github issues, I see a note that we should create a separate session per thread. The comment that immediately follows says "resource", however.

So, do we need one session per thread, or are sessions thread safe, but not resources? Is there a 1:1 mapping between the thread safety of a resource and a client?

As a bonus question, how expensive / wasteful is it to create new clients (or sessions, as above) on-demand per executor thread...?

[JordonPhillips]

Thanks for the feedback! I agree that it's a little ambiguous, we'll make sure to get that updated.

[dfee]

In the meantime (before you update the documentation), can you provide some insight to those questions? Thanks :)

[joethompsonduo]

An update is still needed.

[shadycuz]

This ^

[joethompsonduo]

Once again, echoing the sentiment that some sort of response is needed. Can you acknowledge that the maintainers are at least receiving these communications?

[jweinst1]

Question, for firing large amounts of s3 restore_object requests, across multiple threads, what is the safest approach to take?

[tuukkamustonen]

Also wondering how this is. I've been carelessly sharing session and client instances over threads for years without (perceived) problems. Not saying it's the way to go, but I wonder what are the cases where it might break?

Even the docs state:

It is recommended to create ...

I read that as is "you may do just fine without... but we don't guarantee it", which is pretty vague.

[rahulr3]

Can someone comment ? I'm working on a multithreaded application and each thread is creating it's own session/ resource. Are sessions/resource thread safe ? How expensive it is to create session, resource, clients at scale ?

[jsyrjala]

@JordonPhillips It would be nice to have some documentation about the thread safety matters.

[rkiyanchuk]

@kyleknap @jamesls

Could someone please provide the clarification:
is boto threadsafe per resource, client, or session?

It's been more than a year with no response :(

[lhufnagel]

Am I right in assuming that with the switch to urllib3 (which promises thread-safety) in botocore 1.11.0 we should be good?

[NickWoodhams]

I would also like some insight on this. I have been using put_object in celery workers and getting intermittent errors and I cannot figure out if this is due to the number of concurrent workers and limitations from AWS as to the number of clients, or an issue of threading within Boto.

Could someone please provide some insight? I would be very grateful.

[ori-n]

Could someone please share a code example for working with a session per thread? Thanks

[hajapy]

I would echo that this remains an issue. Clearer documentation would be helpful, as would general thread safety in the construction of clients/resources/sessions.

[nebi-frame]

I have no idea why no one from AWS gives a clear explanation for all these issues raised.

[LyleScott]

Instead of

import boto3
s3 = boto3.resource("s3")
s3.Object(bucket_name, filename).put(Body=s.getvalue())

I used

import boto3.session
sess = boto3.session.Session()
c = sess.client("s3")
c.put_object(Bucket=bucket_name, Key=filename, Body=s.getvalue())

and then multithreading worked fine. the boiler plate for that is something like:

# i'm using a partial here ... which is the  slightly more complicated case.
from functools import partial
from io import StringIO
from multiprocessing.pool import ThreadPool

def generate_db_rows(db_client) -> None:
    s = StringIO()
    s.write("some content...")

    # actual line that matters... this is a session per thread.
    sess = boto3.session.Session()

    client = sess.client("s3")
    client.put_object(Bucket='something', Key='thread_unique_filename.csv', Body=s.getvalue())

table_names = ("foo", "bar", "baz")
func = partial(generate_db_rows, db_client)
with ThreadPool(processes=10) as pool:
        pool.map(func, table_names)

[rohan-mo]

@JordonPhillips I've just seen this documentation and it is still vague as was originally pointed out by the issue author.
Could you please help clarify the documentation here?

[Luis-Palacios]

I'm joining the party here, I need to query from different regions depending on certain parameters on each request and just realized initializing a boto3 resource is not exactly cheap so planning on initialization a resource for each possible region on app start as a singleton and then using the corresponding resource on each request

I started thinking about thread-safe, I went to docs and I was not able to understand if this is going to be thread-safe.

I will highly appreciate clarifications too

[jsmodic]

I've fought with this issue a few times due to adverse effects of role based authentication and multiple botocore sessions.

From reading the code alone, it's clear there are thread safety problems. This initialization pattern combined with how it lazy initializes things and then how it creates clients is definitely not thread safe, and that's just something that immediately stands out.

However, if you look at other parts, it's obvious there is careful consideration to thread safety.

So it really looks like it is supposed to be thread safe but isn't in practice.

[rohan-mo]

@jsmodic If you have the time, could you please explain the links and _why_ those code snippets are not thread safe.

[jsmodic]

The shared state. See the error this person gets on SO. That dictionary is not protected in any way

Also at one point they attempted to fix that but seemingly have rolled it back since then.

It's not obvious that there are more thread safety problems, but session.py still has at least that.

[mattsb42-aws]

This is called out in the boto3 docs[1], but it could be more visible. What is not clear in those docs is whether botocore sessions have the same issue or if using a common botocore session across multiple boto3 sessions is sufficient.

[1] https://boto3.amazonaws.com/v1/documentation/api/latest/guide/resources.html#multithreading-multiprocessing

[rohan-mo]

@mattsb42-aws @thomas-barton I don't think this issue should be closed yet.

The change clarifies the documentation, but as pointed out in this comment above, botocore.session.Session isn't thread-safe (maybe, this can also be confirmed by the aws team). And since boto3.session.Session internally initializes a botocore.session.Session, is it actually thread-safe? I mean, does it implement some work around to make the internally used botocore.session.Session thread-safe or is it susceptible to the same issues as botocore.session.Session?

[jsmodic]

boto3 defers directly to the botocore session and doesn't do anything special. Creating a client in threads is not threadsafe. You can test that one pretty easily with some quick code:

from concurrent.futures import ThreadPoolExecutor
import boto3

def test_session(boto3_session):
    boto3_session.client("s3")
    return "Got a client"

with ThreadPoolExecutor(max_workers=20) as executor:
    boto3_session = boto3.Session()
    futures = []
    for _ in range(100):
        futures.append(executor.submit(test_session, boto3_session))
    for future in futures:
        print(future.result())

You'll get it to fail easily in that block I posted above from the botocore session:

Traceback (most recent call last):
  ....
  File "/.../python3.6/site-packages/botocore/session.py", line 923, in get_component
    del self._deferred[name]
KeyError: 'credential_provider'

You'll also see the same problem if you use the default session:

from concurrent.futures import ThreadPoolExecutor
import boto3

def test_session():
    boto3.client("s3")
    return "Got a client"

with ThreadPoolExecutor(max_workers=20) as executor:
    futures = []
    for _ in range(100):
        futures.append(executor.submit(test_session))
    for future in futures:
        print(future.result())

(Same error as above)

And for completeness the botocore version has the same problem (since it's all derived from the botocore session not being threadsafe):

from concurrent.futures import ThreadPoolExecutor
import botocore

def test_session(session):
    session.create_client("s3")
    return "Got a client"

with ThreadPoolExecutor(max_workers=20) as executor:
    futures = []
    boto_session = botocore.session.get_session()
    for _ in range(100):
        futures.append(executor.submit(test_session, boto_session))
    for future in futures:
        print(future.result())

(Same error)

[pedros007]

I also ran into problems with multithreading client creation. My use case was a bunch of head_object requests to see if n different keys exist in a bucket.

Here's something which seems to work: In the main thread, create a client with a config using a number for max_pool_connections. Use the client by all the child threads. Make sure max_pool_connections is the number of threads you launch. Here's a snippet where I call some my_head_object_function(key, client) which returns True if the object exists at the given key using the given client.

    num_threads = 16
    cfg = botocore.config.Config(max_pool_connections=num_threads)
    client = boto3.client("s3", config=cfg)
    futures = {}
    with ThreadPoolExecutor(max_workers = num_threads) as executor:
        for key in keys:
            f = executor.submit(my_head_object_function, key, client)
            futures[f] = key
    # ... process the results of the futures as_completed

[flyingdev]

@jsmodic I had similar issue recently and resolved by creating session per each thread.
This is what boto3 documentation recommends.

    class MyTask(threading.Thread):
        def run(self):
            session = boto3.session.Session()
            s3 = session.resource('s3')

Slight updated.

from concurrent.futures import ThreadPoolExecutor
import boto3

def test_session():
    session = boto3.Session()
    session.client("s3")

    return "Got a client"

with ThreadPoolExecutor(max_workers=20) as executor:
    futures = []
    for _ in range(100):
        futures.append(executor.submit(test_session))
    for future in futures:
        print(future.result())

So create session in test_session.

Thank you.

[jsmodic]

@flyingdev yes you can do that, but you need to be careful when creating sessions depending on your use case. For example, creating a new session will reauthenticate, and if you're using role based auth on EC2, aws can start throttling / failing your authentication from too many IAM calls if you're aggressively creating sessions.

[dash-samuel]

Hi everyone in my case I can successfully create multiple threads that share the same session, and am able to download from an S3 bucket for example without problems.

This is how I do it:

import concurrent.futures
import boto3
import json

# setup client and session
sess = boto3.session.Session()
client = sess.client("s3")

files = ["path-to-file.json", "path-to-file2.json"] 

def download_from_s3(file_path):
    obj = client.get_object(Bucket="<your-bucket>", Key=file_path)
    resp = json.loads(obj["Body"].read())
    return resp

with concurrent.futures.ThreadPoolExecutor() as executor:
     executor.map(download_from_s3, files)

Creating a session for each thread in my case results in a big slow down, whereas with this approach I am seeing an up to 7x improvement in performance compared to synchronous downloads.

[tomaszhlawiczka]

The doc https://boto3.amazonaws.com/v1/documentation/api/latest/guide/resources.html#multithreading-and-multiprocessing states:

It is recommended to create a resource instance for each thread / process in a multithreaded or multiprocess application rather than sharing a single instance among the threads / processes.

Then how can I benefit from reused connections with max_pool_connections → https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html#botocore-config ?

[jmehnle]

@dash-samuel wrote:

Hi everyone in my case I can successfully create multiple threads that share the same session, and am able to download from an S3 bucket for example without problems.

Your threads are sharing the _client_, which is officially thread-safe. This explains why your example works, but this doesn't mean you can generally share a session across threads. Directly operating (such as _creating_ clients) on a shared session from multiple threads is _not_ thread-safe per the boto3 Session docs.

[ryansonshine]

To summarize, there are two cases, one being multithreading and the other being multiprocessing.

Session: Unsafe in all cases due to shared metadata/urllib3.

Resource: Unsafe in all cases due to its direct interaction with a Botocore session.

Client: (Assuming the client is not used to interact with the underlying Botocore Session) Safe in a threaded environment, unsafe in a multiprocess environment. This is due to forking issues with urllib3’s connection pool and leaves botocore unable to guarantee http messages are read in the right order if the pool isn’t created under the same PID. (https://github.com/psf/requests/issues/4323)

I've opened up a PR https://github.com/boto/boto3/pull/2848 and am requesting feedback if this makes it more clear.

[ryansonshine]

@nebi-frame @jsmodic @mattsb42-aws Do you think the PR https://github.com/boto/boto3/pull/2848 adds clarity to this?

[nateprewitt]

We've merged boto/boto3#2848 today, adding detailed information on multi-threading requirements for each of the main Boto3 primitives (Clients, Resources and Sessions). The updated documentation should be in the next release. I'll leave this open until the end of the week for any further feedback and we'll plan to resolve afterwards. Thanks everyone!

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.