Dpl: Unable to figure out how to encode the deploy key from the docs

[domenic]

Hi @svenfuchs, me again; thanks so much for the help with the new deploy key feature. Here's another issue I ran into.

It appears the intent of the feature is to put the deploy key directly in the .travis.yml. Presumably encrypted using travis encrypt. The readme says

  --deploy_key KEY               A base64-encoded, private deploy key with write access to the repository (type:
                                 string, note: RSA keys are too long to fit into a Travis CI secure variable, but
                                 ECDSA-521 fits, see:
                                 https://developer.github.com/v3/guides/managing-deploy-keys/#deploy-keys)

and https://github.com/travis-ci/dpl/pull/695 suggested

    travis encrypt DEPLOY_KEY="$(cat deploy_key|base64)"
    dpl --provider=pages --deploy-key="$DEPLOY_KEY" --local-dir=build

although that wasn't ported over. But when I follow these instructions, I end up with the following in my logs:

Setting environment variables from .travis.yml

$ export DEPLOY_KEY=[secure]

We were unable to parse one of your secure environment variables.

Please make sure to escape special characters such as ' ' (white space) and $ (dollar symbol) with \ (backslash) .

For example, thi$isanexample would be typed as thi\$isanexample. See https://docs.travis-ci.com/user/encryption-keys.

(see live at https://travis-ci.org/w3c/payment-method-manifest/builds/575963835) and then of course deployment fails to authenticate.

Any ideas?

[domenic]

Oh, additionally, I tried ECDSA-521 and it was too long; I moved to ECDSA-384 instead.

[svenfuchs]

@jyasskin @webknjaz you've been involved with the original PR that added this. could you have a look here? any thoughts?

[jyasskin]

Hm, I wonder if travis encrypt DEPLOY_KEY="$(cat deploy_key|base64)" is getting confused because base64's output ends with a newline? Try

travis encrypt DEPLOY_KEY="$(cat deploy_key|base64 -w0)"

or

travis encrypt DEPLOY_KEY=$(cat deploy_key|base64)

?

[svenfuchs]

with these kinds of issues i am now wondering if we should turn it into --deploy_key PATH and expect that a file with the deploy key exists at that location. we'd encourage people to use travis encrypt-file.

what do you all think?

[webknjaz]

@svenfuchs sounds good. Also, since the Travis CI web UI allows setting longer env vars, would it be possible to support pasting the key as is there?

[svenfuchs]

this PR changes --deploy_key to expect a path to a file with the deploy key https://github.com/travis-ci/dpl/pull/1087. i think this provides a somewhat better UX.

Example: https://travis-ci.org/svenfuchs/test/builds/576260089 has run https://github.com/svenfuchs/test/blob/gh-pages/.travis.yml

@domenic do you want to give this a shot? try using travis encrypt-file to encrypt your deploy key.

you'd want to use the following branch:

deploy:
  provider: pages
  edge:
    branch: sf-deploy_key

@webknjaz I like that idea, but at the moment it does not seem possible to set multiline values to env vars via repo settings, or am i missing something?

[jyasskin]

I did this a long time ago, but I think I avoided encrypt-file because it leads to an unencrypted file in the build tree during the build, at the same time that the deploy step is uploading part of the build tree to a public HTTP server, and I didn't want any chance that a user error could upload the unencrypted secret key.

If that worry doesn't make sense anymore, or there's something else defending against it, I have no objections to the change.

[svenfuchs]

@jyasskin fair point. it already moves the decrypted key out of the working dir to ~/.dpl where it has to sit in order for several commands to succeed (among them testing the key with ssh -i [...] -T [email protected]. if any of these commands (including moving the file) fail then the deploy process gets aborted.

in addition to that i have now added a git pre-commit hook in https://github.com/travis-ci/dpl/commit/6be070d4cb6b99e6169adda556af640017d7fdb0. it gets installed right before we commit.

this build has rejected a fake left over key https://travis-ci.org/svenfuchs/test/builds/576447948#L188 from https://github.com/svenfuchs/test/blob/dpl_test/.travis.yml#L8

[webknjaz]

it does not seem possible to set multiline values to env vars via repo settings

I guess it should be possible to use \n there as it's being rendered into bash anyway.

it leads to an unencrypted file in the build tree during the build

Totally agree. I also dislike a global env var option much because it's available to processes during all steps which setting a secret just for the deployment provider via .travis.yml scopes it just to the deploy step...

[svenfuchs]

@webknjaz i think having users to escape newlines is just as terrible UX though :)

btw the deploy key file doesn't have to be encrypted in a before_script, it can happen in before_deploy.

i'm going to merge the PR before too many people start using what's in master.

thanks everyone!

[domenic]

I can confirm that after #1087 things work great! See https://github.com/w3c/payment-method-manifest/blob/263e3842845041a38dedb5e15024738f85a28d50/.travis.yml#L11-L19 for the setup that works.

Let me know if you're thinking of any potential tweaks, but otherwise I'd be happy to start rolling this out to all the repos where I was using the built-in GitHub pages deploy provider.

[svenfuchs]

@domenic thanks for the confirmation! 🎉

i don't see much room for improvement, except maybe:

• skip_cleanup is true by default starting in v2, so you can remove that option
• the command line parser we now use under the hood does not care about dashes - vs underscores _ any more (except negation via no- needs a dash to work on all supported Ruby versions, sadly), so you can unify your style. i'd recommend underscores as that's what the readme will document, too, but it's really up to you.
• i personally like to order keys according to the general flow, so i'd move before_deploy before deploy (heh), and move notifications to the end (and maybe branches below language, so the scripts sit closer together), but you probably have a reason ordering them the way you have them

but then again, this is all style.

it's great to know this works well for your usecase!

[webknjaz]

having users to escape newlines is just as terrible UX though

True. But Travis CI already requires this for other special chars anyway :)

[webknjaz]

btw the deploy key file doesn't have to be encrypted in a before_script

The problem is that the decryption key is in the env var available in test steps.

[svenfuchs]

The problem is that the decryption key is in the env var available in test steps.

But, I'm confused ... if that's the concern ... how is it better to have the unencrypted key in the env var?

[webknjaz]

@svenfuchs it's not better. I'm saying that having secure: in the deploy: step is more secure because it only hits Dpl.