Aws-data-wrangler: raise Exception if wr.s3.delete_objects failed

[bryanyang0528]

Is your idea related to a problem? Please describe.
I used wr.s3.delete_objects to delete something on s3. Even if the action failed(due to permission issue), the function still return nothing to me. So I do not know if it is success or not.

Describe the solution you'd like
wr.s3.delete_objects will call client_s3.delete_objects from boto3 to delete something. I would like to parse the response of client_s3.delete_objects to determine if this request is success or not.

reference: S3.Client.delete_objects

[igorborgest]

@bryanyang0528 awesome contribution, thanks!

This enhancement will be released on version 1.2.0.

[bryanyang0528]

@igorborgest The test cases of this project need a real AWS environment to test. But this way is more expensive and slower than the mocking way. Could I add test cases using moto to test some basic s3 operators? I could add this test process to CI setting too. Is this idea good for you?

[igorborgest]

@bryanyang0528 yeah it sounds pretty good to me.

Today we already have a minimal set of tests with moto under testing/test_awswrangler/test_moto.py.

You can run it with:

pytest testing/test_awswrangler/test_moto.py

But for sure there is a lot of room for improvement and also to add it to GitHub Actions. Good ideia!

I don’t see this project running tests only against moto for all, but for sure could be a good alternative for those that want to test it w/o provisioning real AWS resources and for a faster/minimal test on GitHub actions.

[igorborgest]

Thanks a lot @bryanyang0528 !

Released on version 1.2.0