Aws-sdk-cpp: Multiple calls to InitAPI() / ShutdownAPI()?

[asylvest]

Is it valid to have an application which does...

Aws::InitAPI();

// Make some AWS SDK calls

Aws::InitAPI();

// Make some more AWS SDK calls

Aws::ShutdownAPI();

// Make yet more AWS SDK calls

Aws::ShutdownAPI();

// OK now I'm all done - no more SDK calls

There are cases where due to how classes are designed, it's difficult / impossible for them to know if a previous step has initialized the SDK (for example, I have a plugin which calls the SDK - the main application which loads my plugin has no idea about the SDK so couldn't possibly initialize it). If this isn't safe, I can use a singleton on my end (I just hate singletons :-P ).

Thanks.

[JonathanHenson]

No, I would recommend against multiple calls. If this is a dll, you could put the Init and Shutdown calls in dllmain inside the instance check that makes sure there is only one instance loaded per process.

If you are not on Windows, you could put it into a global init somewhere.

I hate global initializers and resisted it for a long time. It is like a disease that ends up infecting everyone that touches a given piece of code. However, it was necessary to properly handle openssl and libcurl, so we ended up being infected as well.

[asylvest]

Thanks. Yeah after I posted this I took a look at the SDK implementation and saw it wouldn't work to call it multiple times (the ShutdownAPI() call would delete everything).

Unfortunately, I do think I'm stuck / infected / something bad. As an example, I've got a plugin abstraction layer that takes in a string for a filename and abstracts away how to read various input file types. I've updated one of these plugins to check if the filename starts with 's3://' and, if so, use the SDK's S3 client to read an object from S3. I've got a simple test program that scans a directory of plugins, finds the appropriate one to load based on my input file type, and prints some header information. This test program has been around for a decade and definitely doesn't know about AWS, so it can't possible call InitAPI(). So, I can do this in my plugin. So far so good.

I have another application which is AWS-aware and uses this same plugin. This second application is making all kinds of other SDK calls, so it needs to call InitAPI() at the top of main. But when the plugin's loaded, I can't have the plugin call ShutdownAPI() at the end.

I had created an RAII class awhile back that wraps the InitAPI() / ShutdownAPI() calls. I ended up adding a private static std::atomic to this that does reference counting... on first construction it calls InitAPI() and upon final destruction it calls ShutdownAPI(). I don't love it but I didn't see another way to handle my two scenarios above (other than updating all programs that might ever use this plugin to initialize the SDK but again that's problematic / invasive since most are not AWS-aware).

[OlafvdSpek]

What's the problem with not calling ShutdownAPI()?

[JonathanHenson]

depends on use case, possibly just a dirty valgrind report. Also possibly a crash.

Sent from my iPhone

On Mar 3, 2017, at 1:51 AM, Olaf van der Spek [e-mail] wrote:

What's the problem with not calling ShutdownAPI()?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[OlafvdSpek]

Crashing would be bad, when / why would it crash?

[JonathanHenson]

if something else is using libcurl or openssl after the sdk is finished--especially if they call cleanup or init functions.

If you are using a custom memory manager that is independent of the sdk usage (a video game engine for example).

If you are using logging, the logging thread will not be shutdown upon exit.

Sent from my iPhone

On Mar 3, 2017, at 8:27 AM, Olaf van der Spek [e-mail] wrote:

Crashing would be bad, when / why would it crash?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[OlafvdSpek]

If both the main app and something else use the SDK (and call InitAPI and ShutdownAPI) you run into troubles anyway right?

Can't init and shutdown use reference counters?

[asylvest]

This is what I ended up doing. Essentially (in my use case I don't need to ever set any SDKOptions so that part's simple)...

class Initialize
{
public:
    Initialize();
    ~Initialize();
    Initialize(const Initialize& ) = delete;
    Initialize& operator=(const Initialize& ) = delete;

private:
    const Aws::SDKOptions mOptions;
    static std::atomic<size_t> mCount;
};

std::atomic<size_t> Initialize::mCount(0);

Initialize::Initialize()
{
    const size_t origCount = mCount++;

    if (origCount == 0)
    {
        Aws::InitAPI(mOptions);
    }
}

Initialize::~Initialize()
{
    const size_t newCount = --mCount;

    if (newCount == 0)
    {
        Aws::ShutdownAPI(mOptions);
    }
}

It would be great if the SDK provided something along these lines, though as you can see this wrapper class is only a few lines long so was easy enough to write.

[Bu11etmagnet]

The InitAPI/ShutdownAPI pair is very un-C++-like. The library should provide a RAII mechanism instead.

[JonathanHenson]

static/global anything is un-c++ like to begin with. Unfortunately we have to integrate with legacy c libraries. I'll look into this and see what we can come up with.

Sent from my iPhone

On Mar 6, 2017, at 3:52 AM, Ráduly, Csaba [e-mail] wrote:

The InitAPI/ShutdownAPI pair is very un-C++-like. The library should provide a RAII mechanism instead.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[crusader-mike]

also, if my app happens to use 3rd party library that uses AWS SDK (and calls it's own Init/Shutdown) -- this will be a nice source of bugs.

Proper solution is to put Init/Shutdown on a counter in AWS SDK (i.e. allow multiple init/shutdown calls).

[dburken]

Have same problem with the ossim code. Linux good, windows bad... ShutdownAPI crashing.

[JonathanHenson]

https://github.com/aws/aws-sdk-cpp/pull/478

[rvernica]

Are there any API changes addressing this issue? Is there any official guidance on working around this? If not, it might be worth re-opening this issue.