Kombu: Postgresql transport password revealed in worker startup log.

[rexroof]

it looks like the results: block is sanitized, but not the postgresql url.

 -------------- celery@767a90b92f91 v4.3.0 (rhubarb)
---- **** -----
--- * ***  * -- Linux-4.9.125-linuxkit-x86_64-with-debian-9.9 2019-07-08 17:44:10
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         yms_server.background_worker:0x7fab050cf748
- ** ---------- .> transport:   sqla+postgresql://newt:Iengoachangedd4oe2ch@yms-pg/yms
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 1 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery

[auvipy]

post in celery-users mailing list

[rexroof]

this also happens with celery beat:

yms-celery_1 | celery beat v4.3.0 (rhubarb) is starting. yms-celery_1 | __ - ... __ - _ yms-celery_1 | LocalTime -> 2019-08-14 14:32:24 yms-celery_1 | Configuration -> yms-celery_1 | . broker -> sqla+postgresql://PSSWDREMOVEDHERE@host/db yms-celery_1 | . loader -> celery.loaders.app.AppLoader yms-celery_1 | . scheduler -> celery.beat.PersistentScheduler yms-celery_1 | . db -> /tmp/celerybeat-schedule yms-celery_1 | . logfile -> [stderr]@%WARNING yms-celery_1 | . maxinterval -> 5.00 minutes (300s)

I'd like to re-open this issue. someone from celery-users said they would open an issue a month ago.

[rexroof]

@xirdneh did you open an issue and I missed it?

[thedrow]

Wait, why are you using postgresql as a broker?
That's not very supported or maintained.
You're welcome to submit a PR that fixes this issue.
It probably belongs in the Kombu repository.
@auvipy What do you think? Should we move the issue?

[auvipy]

Wait, why are you using postgresql as a broker?
That's not very supported or maintained.
You're welcome to submit a PR that fixes this issue.
It probably belongs in the Kombu repository.
@auvipy What do you think? Should we move the issue?

sqla is re-added to be used as development broker by the request of some apache downstream projects. It's not for production use at all. and yep it seems to be a kombu issue :dagger:

[thedrow]

If they want it, they should be the ones to maintain it.
Not us.

[rexroof]

thanks for looking at this.

damn, I wish there was some warning that this wasn't officially supported.
I didn't make this choice, I'm just the ops engineer trying to get
passwords out of our logs.

I did try and do my due diligence to submit a PR to fix this myself. It
appeared that the problem might be that celery wasn't
calling maybe_sanitize_url() in the right place. without digging into some
testing my python breadcrumb tracking was failing me.

On Mon, Aug 19, 2019 at 7:32 AM Asif Saif Uddin [e-mail]
wrote:

Wait, why are you using postgresql as a broker?
That's not very supported or maintained.
You're welcome to submit a PR that fixes this issue.
It probably belongs in the Kombu repository.
@auvipy https://github.com/auvipy What do you think? Should we move the
issue?

sqla is re-added to be used as development broker by the request of some
apache downstream projects. It's not for production use at all. and yep it
seems to be a kombu issue 🗡

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/celery/kombu/issues/1085?email_source=notifications&email_token=AAUKXQ4A5JA4IHAXWYO3PILQFKACTA5CNFSM4IM6YUYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4SS2MQ#issuecomment-522530098,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAUKXQZ46P4PNJMGO2H7KKDQFKACTANCNFSM4IM6YUYA
.

[thedrow]

It's not on the list so it's not supported.

[matteius]

Probably not that hard to add basic password filtering out of the logging startup messages regardless of backends.

[matteius]

Nice, thanks @thedrow that commit looks good!