Botocore: bucket names can't have '/' in them?

[teffland]

Trying to get object from Bucket='some-syndication/some-directory' it returns that the '/'doesn't match the parameter validation.

ParamValidationError: Parameter validation failed:
Invalid bucket name "some-syndication/some-directory": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$"

However if I generate a presigned url with the same bucket name, I get the URL and can follow that link to the file. Is this a bug or purposeful?

[rayluo]

Bucket name can not contain /. This is clearly conveyed in that error message.

When you presign a url like 'some-syndication/some-directory', the tool simply treats the parts behind / as "directory".

Are you sure your bucket's name is 'some-syndication/some-directory'? I don't think it can even exist in the first place. Note that you can put your "directory" parts into the Key of your object.

You may find this S3 concepts documentation helpful.

[teffland]

Oh ok, thanks.

[chennav]

Then how do we access a shared bucket through boto3 S3 API??
All I have is the URL s3://exports.localytics/abc.
I have verified via command line tool that I have read access to this bucket and able to download data, my account is authorized to access this S3 bucket.

But using boto3, am not able to i.e.
s3.Bucket('/exports.localytics/abc')
gives the error
Invalid bucket name "/exports.localytics/abc": Bucket name must match the regex "^[a-zA-Z0-9.-_]{1,255}$"

[JordonPhillips]

/abc is a Key, not a Bucket. If your bucket's name is bucket-name you access it with s3.Bucket('bucket-name'). If you want to get the object bucket-name/abc then you use s3.Bucket('bucket-name').Object('abc')

[chennav]

JordonPhillips, I understand that 'abc' is a key, but note that I only have permissions to access the key 'abc'. So, if I try to access separately as you suggested I get the following error,
s3.Bucket('exports.localytics').Object('abc').get()
Traceback (most recent call last):
File "", line 1, in
File "/Library/Python/2.7/site-packages/boto3/resources/factory.py", line 394, in do_action
response = action(self, _args, _kwargs)
File "/Library/Python/2.7/site-packages/boto3/resources/action.py", line 77, in call
response = getattr(parent.meta.client, operation_name)(*params)
File "/Library/Python/2.7/site-packages/botocore/client.py", line 310, in _api_call
else:
File "/Library/Python/2.7/site-packages/botocore/client.py", line 395, in _make_api_call
def __init__(self, events, region_name, endpoint_url, service_model):
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
which is likely because I am trying to access the bucket first and then the key

[JordonPhillips]

You can also use s3.Object('bucket-name', 'key'), could you give that a shot?

[chennav]

yep, that was the first attempt s3.Object('exports.localytics','abc').get()
Traceback (most recent call last):
< same as above comment >
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

[blehman]

How do we access a bucket within a bucket for writing?

import boto3
s3 = session.resource('s3')
# list all buckets (SUCCESS)
for bucket in s3.buckets.all():
    print(bucket.name)
# try to access a bucket within a bucket (ERROR)
mybucket = s3.Bucket('topLevelDir/projects2015/myBucketToWrite')

...Bucket name must match the regex "^[a-zA-Z0-9.-_]{1,255}$"

Trying another method.

from boto3 import client
conn = client('s3')
for key in conn.list_objects(Bucket='topLevelDir')['Contents']:
    print(key['Key'])

ClientError: An error occurred (PermanentRedirect) when calling the ListObjects operation: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.

[kp-cn]

do we have a solution to Chennav's problem of accessing a specified key, if we only have access to that key?

i am facing the same problem and not been able to resolve

my client gave me credentials to a specific key/folder on s3 bucket, while I am able to list content of the folder using tools like cyberduck, I cannot access the key/folder using boto3 (results in 'AccessDenied')

any help would be highly appreciated.

[alap]

Literally the same problem here. I have a pre-signed S3 url
s3://some-location-here/first/second/third/fourth

I'm trying

mybucket = s3.Bucket('some-location-here/first/second/third/fourth')

And I have a glorious Regex error.

[rayluo]

Just a side note for everyone who chimed in after this issue was closed: you'd better create a separated new issue for your case, and optionally mentioned this issue in your new issue. The reason for this suggestion is that, when there are 55 open issues (as of today), your comments in a closed thread won't receive much attention.

[f0ster]

@alap no bucket names can't have the slashes in them.. you have to put slashes in the key to get the pseudo-directory behaviour.. s3 does not have directories

[yinkalawtrades]

I achieved it with the Prefix Key

Here is how i did it

let params = {
    Bucket: bucket,  //put your bucket here without slash e.g somebucket
    Prefix: prefix     //put the subcategory here e.g food/
  };
  s3.listObjects(params, function(err, data) {
    if (err) { // an error occurred
      callback({error: err});
      return;
    }

   //console.log(data)
});

This will list all directories (subbuckets) and files that start with that prefix. You may have to filter by yourself.

P.S when i gave the prefix a forward slash before it (i.e /food/) i got an error. The solution is not to give the prefix a forward slash before it (i.e food/)