Crates.io: Trouble setting up dev environment

[ehuss]

It's been a while since I've used crates.io locally, and I'm running into some difficulties that don't seem to be documented in CONTRIBUTING.md (or at least I'm missing it).

Email verification

It seems kinda tricky to verify one's email address. I had to load a file in the /tmp directory and decode it with python to find the URL to open. It would be nice if there was a command to run to force a verification (or a SQL command to run, or something).

Background updates

In order to use a crate that has been published, it looks like I need to run cargo run --bin background-worker? Is that correct? I don't see it documented, so I'm not sure. Or is there maybe a setting to do the index updates synchronously? I'm also curious what exactly the background worker does.

Index not updated

The tmp/index-co repo doesn't seem to get updated automatically by the background worker. I need to git pull it after each publish. Would it make sense to change the documentation to use tmp/index-bare instead? I think that works with Cargo, so I'm not sure if there's a need for index-co at all (maybe it is used in tests?). (Maybe I'm doing things wrong with the background worker?)

[smarnach]

Email verification

The easiest way to verify your email address is update emails set verified=true; (which will verify all emails in the local db), and we should document that.

There seems to be another bug though – the email gets written to the /tmp directory in a rather uselss format. I believe this is a bug in some other crate, but we should look into it anyway.

Background updates

Yes, there are several operations that require running the background worker locally, and yes, it's missing from the documentation.

Index not updated

If I remember correctly, the local index did get updated for me by the background worker, but it's probably two or three months ago since I last did something locally that depended on it, so we should look into this as well.

[carols10cents]

There seems to be another bug though – the email gets written to the /tmp directory in a rather uselss format. I believe this is a bug in some other crate, but we should look into it anyway.

Yeah I was looking into this recently-- the lettre crate is responsible for this; it started writing bytes instead of ASCII text in this commit. I've filed an issue with lettre.

[plaets]

I recently wanted to run crates.io locally, and I can confirm that these issues persist.

Is there any reason why the background worker isn't automatically run from, for example, docker-entrypoint.sh? I know it's documented, but I forgot to run it manually and it took me a while to figure out why the readmes weren't rendered.

Verifying emails also isn't exactly straightforward, is adding a new command to crates-admin (crates-admin verify-email (email) or something similar) a good idea?
If not, as mentioned earlier, documenting the steps to change the verification status in the database probably is the next best solution.

[Turbo87]

FWIW I have a branch here where I've experimented with adding the background worker to the docker-compose setup, but I ran into some troubles with it. I've pushed it to https://github.com/Turbo87/crates.io/commits/docker-worker, so feel free to experiment and finish it :)

[plaets]

Thanks, I'm working on it right now. I will need feedback on some of the changes related to docker-compose, I assume I should just open a PR here, right?