Open-event-server: Image resizing is not working

[iamareebjamal]

https://github.com/fossasia/open-event-server/blob/9bb9f68305c261713ee486f3840ebcf55dc70d23/app/api/helpers/files.py#L80

urlib is throwing 403 Forbidden error for almost all images.

Please use requests and verify image resizing works. Also, create a management command which finds all events which have no resized images and resizes them

[iamareebjamal]

@codedsun Can you please take this up?

[codedsun]

Taking this up

[codedsun]

@iamareebjamal I learnt that this task creates resized image and stores in static folder. How do i test that for 403 error via which api upload image? Please guide me how should I proceed?

As when I am hitting through the frontend and selecting the image like event background, event logo it works fine.

And for management command do I need to add the task in the celery?

[iamareebjamal]

Send this as original image URL in Postman - https://storage.eventyay.com/eventyay.com/events/758/background/Vk0zSTdyZX/1517e947-29df-4a3c-9580-373db3ba9ba9.png

Also, working fine means the image is resized and all image URLs are correctly set. Is that happening?

For management command, no

[codedsun]

@iamareebjamal

They are null

[codedsun]

@iamareebjamal When the event is created the files.py function is run. What is expected that large_image_url, thumbnail_image_url - should be created with a valid url and saved in the db? Am I right?

[iamareebjamal]

Yes

[codedsun]

@iamareebjamal Okay I will debug it tomorrow.

[iamareebjamal]

Status? PATCH endpoint is not as important. That is atleast working with its limitation. This is completely broken

[codedsun]

@iamareebjamal Sorry I missed that due to health issues. I am resuming and working on it now.

[codedsun]

@iamareebjamal I am finding a way to check that celery worker is running or not on my machine also how do I turn on logs for celery.. I see that function is called, but I m debugging more about it.

[iamareebjamal]

celery won't run if you've not run it. It's simple

[iamareebjamal]

Did you follow the instructions and run celery? https://github.com/fossasia/open-event-server/blob/development/docs/installation/basic.md

[codedsun]

Yeah, while running celery worker -A app.celery

I am having error

[2019-12-05 17:39:30,349: WARNING/ForkPoolWorker-3] HEAD http://localhost:9200/event [status:N/A request:0.004s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 159, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 80, in create_connection
    raise err
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 70, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 238, in perform_request
    method, url, body, retries=Retry(False), headers=request_headers, **kw
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 344, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 181, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 168, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x10db7e750>: Failed to establish a new connection: [Errno 61] Connection refused

[iamareebjamal]

ENABLE_ELASTICSEARCH=False in your .env

[codedsun]

@iamareebjamal For the locally saved images on the server it's working fine for me.i.e. From frontend I tried to upload the images, they are getting resized and stored in the events table.

Now as per the requirement, that creating an event by using a postman request with imageurl of not a locally saved file, it's throwing 403 error. I am debugging it now.

[iamareebjamal]

No need to debug. The cause and solution both are written in the issue itself

[codedsun]

@iamareebjamal Lol, what should I do then?

[iamareebjamal]

Follow what's written in the issue

[codedsun]

Also, create a management command which finds all events which have no resized images and resizes them?

How do we write a management command, please guide me?
Also, What should be done for 403 error?

What i understood is that, the image can't be accessed by urllib but that's right as from fronted while uploading images we are uploading that to db and returning a locally saved file URL.

[iamareebjamal]

urlib is throwing 403 Forbidden error for almost all images.

Please use requests and verify image resizing works.

I have literally pasted what I wrote in the issue. Now tell me what you don't understand

[codedsun]

No need to debug. The cause and solution both are written in the issue itself

What's the solution here? I need to fix 403 error of urllib

urlib is throwing 403 Forbidden error for almost all images.

It's throwing for the files which are not saved locally on the server, do I need to fix this for the images which are from the source other than local machine or test for which are locally saved on the server?

Also when I am uploading the images from the frontend , it's working fine. They are not working for images not saved on local machine

Sorry for asking such question but help me where I am not able to understand. Thanks

[iamareebjamal]

Please use requests and verify image resizing works.

[codedsun]

That hyperlink made a ton of difference for a beginner like me. I thought request as a hitting api.

Please use requests and verify image resizing works.

For this you meant to replace using urllib and use requests instead to open the file and then perform the resizing. Am I correct here?

[iamareebjamal]

Yes

[codedsun]

@iamareebjamal If that hyperlink would have been present, it wouldn't have taken a day. Thanks :)

[iamareebjamal]

AttributeError("module 'requests.exceptions' has no attribute 'URLError'")

You didn't test the exceptions @codedsun Please fix ASAP

[codedsun]

@iamareebjamal I did tested, failed to replaced this at one line, it was not replaced to Invalid URL

[codedsun]

@iamareebjamal https://github.com/fossasia/open-event-server/pull/6648

[iamareebjamal]

Task resize.event.images[947ebb58-7870-49b5-8488-fc82b2000077] raised unexpected: ObjectNotFound({'parameter': 'event_id'}, 'Event: 1014 not found')

Resize task is using safe_query which means it removes deleted events when querying. Thus, it throws an error when you try to resize images of deleted events. It's an internal module and hence shouldn't use safe_query. See if there is any way to trigger resize event or speaker through API. If there's not, then change safe_query to normal query

[codedsun]

@iamareebjamal Will check and reply.

[codedsun]

@iamareebjamal

For Events image resizing and speaker image reisizing
The task is only triggered when we patch an event/speaker image or the first time event/speaker is created using POST request.

[codedsun]

Also, guide me for this sir. Thanks