Smart_open: uploading failed while dealing with empty file

[amolsaudar123]

Hi,
Currently I am using smart_open library.But I am facing an issue related with empty file .while it is working well with any file but when I deal with any empty file it throws me exceptions
Like
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
can you please Help me out how can I Deal with empty file in smart_open

[amolsaudar123]

Here is the snippet ,where in one file I have some data in test2 ,but in test1 we dont have any data ,so in test1 to upload it to s3 it gives me botocore.exception

[mpenkov]

From the screenshots you are posting, the operation is failing because of an "Access Denied" error.

I can confirm that writing empty files to S3 works with smart_open 3.8.2 and the below code:

(smart_open) misha@cabron:~/git/smart_open$ ipython
Python 3.6.5 (default, Apr  1 2018, 05:46:30)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.0.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from smart_open import open

In [2]: with open('s3://bucket/test_smart_open/empty', 'w') as fout:
   ...:     pass

In [3]: with open('s3://bucket/test_smart_open/empty', 'r') as fin:
   ...:     print(repr(fin.read()))
''

What version of smart_open are you using?

[amolsaudar123]

How do know smart_open version?
can you please help me out

[mpenkov]

Run pip show smart-open

Alternatively, from a python interpreter:

import pkg_resources
print(pkg_resources.get_distribution("smart_open").version)

[MMMohit]

Hi I am facing issue related with empty file,during uploading from hdfs to s3 .whenever I upload any non empty file its uploaded successfully on s3 ,but once I deal with any empty file it gives me exception like
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

and also can you just help me out with smart_open version? How can I know about it using cli
an also I have used code as its snippet is
for line in smart_open(hdfs_path, encoding='utf8'):
#with open(my_file) as friendsfile:
if not line:
logging.info('Writing lines into "*/test.csv"....')
fout.write(line)
#print('friendsfile is empty !!')
else:
#print(" file is not empty!!!")
continue

[mpenkov]

I've tried to reproduce the problem on my end, unsuccessfully (see my first comment). I'd like to help you, but your recent comment provides absolutely no new information to me.

In order for me to be able to help you, I need to know more information:

1. The smart_open version (see my reply above for methods of getting it)
2. A full stack trace (copy paste). You've provided me some screenshots, but they aren't helpful (one of them is a Java stack trace completely unrelated to smart_open).
3. A reproducible example of the problem code. Ideally, this should be something I can simply run on my side to reproduce the problem. Make this sample as small as possible: remove all irrelevant parts. In your case, reading from HDFS is likely unrelated. For an example, see my first comment.

As a side question, does the following work for you?

from smart_open import open
with open('s3://bucket/test_smart_open/empty', 'w') as fout:
    pass  # write empty file
with open('s3://bucket/test_smart_open/empty', 'r') as fin:
    print(repr(fin.read()))  # read empty file back

(obviously, replace bucket with your actual S3 bucket name).

[MMMohit]

smart_open version i s
1.8.0
here is the code snip

[MMMohit]

from smart_open import open
with open('s3://bucket/test_smart_open/empty', 'w') as fout:
pass # write empty file
with open('s3://bucket/test_smart_open/empty', 'r') as fin:
print(repr(fin.read())) # read empty file back
with the given code I tried to read an empty file but its says that
ImportError: cannot import name open

[mpenkov]

Try installing the newest version of smart_open (1.8.2).

Alternatively, try from smart_open import smart_open as open instead.

[MMMohit]

Yes it works for me thanks but after running this code I got output as a

u' '
what is meaning of this output ?

and here is the code
from smart_open import smart_open as open
hdfs_path='hdfs://user/mohitm/sagar/test1.csv'
with open(hdfs_path, 'r') as fin:
print(repr(fin.read()))

[mpenkov]

From this discussion, it is obvious to me that there is no bug in this case, so I'm closing the ticket.

If you have further questions, then the more appropriate place to ask them is stackoverflow.com.