Kombu: Kombu Incorrect parsing of SQS hostname when credentials include a /

[FrenchBen]

Looking at the docs on how to use the AWS SQS queue, it is recommended to use a BROKER_URL of format: sqs://AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY@, as seen in your docs:
http://docs.celeryproject.org/en/latest/getting-started/brokers/sqs.html

Using the example in your docs:

broker_url = 'sqs://ABCDEFGHIJKLMNOPQRST:ZYXK7NiynGlTogH8Nj+P9nlE73sq3@'

but instead using "Alice's" credentials from:
https://aws.amazon.com/blogs/security/how-to-rotate-access-keys-for-iam-users/

$ aws iam create-access-key --user-name Alice
{
    "AccessKey": {
        "UserName": "Alice",
        "Status": "Active",
        "CreateDate": "2013-09-06T17:09:10.384Z",
        "SecretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY",
        "AccessKeyId": “AKIAIOSFODNN7EXAMPLE"
    }
}

Your broker_url is now:

broker_url = 'sqs://AKIAIOSFODNN7EXAMPLE:wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY@'

Full code for the example is now:

from celery import Celery

broker_url = 'sqs://AKIAIOSFODNN7EXAMPLE:wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY@'

app = Celery('tasks', broker=broker_url)

@app.task
def add(x, y):
    return x + y

Run it with: celery -A tasks worker --loglevel=info

What you then end up with, is a parsing issue, such as:

  File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 181, in __init__
    url_params = parse_url(hostname)
  File "/usr/local/lib/python3.6/site-packages/kombu/utils/url.py", line 34, in parse_url
    scheme, host, port, user, password, path, query = _parse_url(url)
  File "/usr/local/lib/python3.6/site-packages/kombu/utils/url.py", line 52, in url_to_parts
    parts.port,
  File "/usr/local/lib/python3.6/urllib/parse.py", line 159, in port
    port = int(port, 10)
ValueError: invalid literal for int() with base 10: 'wJalrXUtnFEMI'`

[georgepsarakis]

@FrenchBen you need to URL-encode the password so that the URL components can be parsed correctly.

[FrenchBen]

@georgepsarakis It'd be good to update the documentation to indicate so - Having:
sqs://aws_access_key_id:aws_secret_access_key@ as the URL format in the doc is quite misleading.

[auvipy]

feel free to send a pr to update the docs

[pawelad]

For the record, this works:

from urllib.parse import quote_plus

from celery import Celery

aws_access_key = quote_plus('AKIAIOSFODNN7EXAMPLE')
aws_secret_key = quote_plus('wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY')

broker_url = f'sqs://{aws_access_key}:{aws_secret_key}@'

app = Celery('tasks', broker=broker_url)

@app.task
def add(x, y):
    return x + y

I'll create a PR for the docs ~tomorrow~ at some point ; )

[auvipy]

looking forward to the PR!

[pawelad]

Thanks for the reminder! : -)

I opened https://github.com/celery/celery/pull/5192 with the suggested docs changes.