Staticman: Self-hosted API: InvalidAsn1Error on npm start

[robinmetral]

I'm trying to test a self-hosted version of Staticman.

I followed the instructions on the README:

1. created a config.development.json based on the sample
2. added my GitHub Personal Access Token
3. added my SSH

but when I run npm start:

InvalidAsn1Error: Expected 0x2: got 0x0

Is there anything I missed?

_(PS. there are 5 high severity issues in the dependencies, mostly about mailgun and github)_

[VincentTam]

Its seems an RSA key error. While setting up @staticmanlab, my own Staticman API instance (so as to test Staticman's native GitLab support), I found this article very useful. Hope this helps your setup.

[robinmetral]

Thanks @VincentTam! I was looking for such an article but couldn't find one :slightly_smiling_face:
This should be in the docs!

[robinmetral]

Did anyone ever self-host Staticman with AWS Lambda?

[gabeluci]

@robinmetral So did you get past this error?

Since the public API is in trouble, I figured I'd host my own, but I'm facing this exact error message too. I was following the blog post at flyinggrizzly.io too, but both it and the official docs are slim on details on how to actually populate that rsaPrivateKey parameter (besides "put your private key there").

The official docs say it should look something like this:

rsaPrivateKey: "-----BEGIN RSA PRIVATE KEY-----\nkey\n-----END RSA PRIVATE KEY-----"

The problem I'm running into is: where does that "key" come from?

I'm running this on Linux (Ubuntu Trusty). I generated a key according to the docs, and I have my key in ~/.ssh/id_rsa (private) and ~/.ssh/id_rsa.pub (public). The key is registered with GitHub now and I know it works.

The config.test.json kind of makes it look like it wants the public key (the key ends in "==", just like my .pub).

My ~/.ssh/id_rsa.pub looks like this:

ssh-rsa key [email protected]

Where "key" is my actual public key and my actual email address is at the end. If I take out just the "key" part, and put it in the config file like this:

"rsaPrivateKey": "-----BEGIN RSA PRIVATE KEY-----\nkey\n-----END RSA PRIVATE KEY-----"

Then I get the exact error described here: InvalidAsn1Error: Expected 0x2: got 0x0

My private key (~/.ssh/id_rsa) looks something like this:

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,4EC16432518D6CB6933684F4E3AD31E3

key
-----END RSA PRIVATE KEY-----

I tried to just replace every line break with \n and use it like that, but that gives me this error message: Error: Key format must be specified. That error message seems to be used for anything that's completely invalid.

If I omit the two header lines (Proc-Type and DEK-Info), then I get a different error message: InvalidAsn1Error: Expected 0x2: got 0xb0

So I'm kind of at a standstill now. I don't know how to proceed.

[gabeluci]

Turns out I just needed a rubber duck to talk to. I just figured it out.

The "Key format must be specified" error got me thinking about maybe it's just in the wrong format. Turns out, that was it. The node-rsa module can only import from certain formats, and whatever I had it in wasn't the right one. So I had to convert it to PEM format. I did that with this:

openssl rsa -outform PEM -in ~/.ssh/id_rsa -out id_rsa.pem

I gave it my passphrase when it asked, and then there was a new file called id_rsa.pem. I used the contents of that in the rsaPrivateKey property (after replacing line breaks with \n). And now npm start works!

[VincentTam]

@gabeluci In fact, a shorter OpenSSL command can be found in the discussion of #219.

[robinmetral]

Thanks @gabeluci, solves this issue for me too!

Well, actually I have another error now: I get Staticman API running on port 80 but directly after something about an unhandled error event.

Just to make sure I haven't done any error:

• Did you also replace by \n the line breaks between the key's body and the first and last dashed lines -----BEGIN...
• Did you have to add the .pem key to your Github account?

Oh and P.S., @VincentTam the command in the discussion you mentioned is actually to create a .pem key, @gabeluci's is to convert an existing key.

[VincentTam]

So creating a new key is quicker than converting an existing key in terms of the length of the command.

[robinmetral]

Yes apparently :slightly_smiling_face:

Also for some reason a newly generated key is much shorter (in my case 27 lines instead of 51 when converting an existing key).

But my error persists.

[VincentTam]

Here's another self-hosting guide on a remote host: https://www.datascienceblog.net/post/other/staticman_comments/

The setup maybe helpful.

You may also consult a recently created Staticman instance on Framagit and its source code a successful setup. (public key included)

[robinmetral]

Thanks a ot @VincentTam!

[gabeluci]

Just to make sure I haven't done any error:

• Did you also replace by \n the line breaks between the key's body and the first and last dashed lines -----BEGIN...

I just removed all the line breaks. I didn't replace them with 'n', I just removed them.

The first and last lines with the dashes and "BEGIN.."/"END..." need to stay.

• Did you have to add the .pem key to your Github account?

No. I had added the key already so I didn't add it again. The new .pem file is the same key, just stored in a different format, so you don't need to add it to GitHub again.

[VincentTam]

For @robinmetral's first question, I _keep_ the newlines in the form of JSON friendly string described in the linked PR as well as the linked blog article by Flying Grizzy.

The roles of the PEM key in Staticman are to safely commit confidential information into the Git repo, and to provide authentication in case of authenticated comments.

A Staticman API instance's authority to act on behalf of the associated GitHub and/or GitLab dummy account comes from solely from the GitHub and/or GitLab token stored in the server-side API config file.

[robinmetral]

Thanks for your replies and help @gabeluci @VincentTam!

Actually my mistake was to try to use port 80 like in the config.sample.json. It might be Linux, but I had to change it to a four-digit port number.

@eduardoboucas would you like a PR to improve the docs/README on how to host one's own instance of Staticman? It would help people not to keep making the same mistakes :wink:

[eduardoboucas]

@eduardoboucas would you like a PR to improve the docs/README on how to host one's own instance of Staticman? It would help people not to keep making the same mistakes

That would be super useful, thanks! Perhaps https://staticman.net/docs/api would be the best place for that information? Repo is here: https://github.com/eduardoboucas/staticman.net

[gabeluci]

@robinmetral Port 80 shouldn't be a problem as long as something else (like Apache) isn't already using it. I used port 8080 and used Apache as a proxy.

I'm working on my own write up to describe how I set everything up. Some issues I faced weren't the fault of Staticman itself.

[gabeluci]

Here's my post of my experience setting up my own Staticman instance: https://www.gabescode.com/staticman/2019/01/03/create-staticman-instance.html

[robinmetral]

@gabeluci can I reuse some of your article in the docs? It's great!

[gabeluci]

Yes, absolutely! Feel free.

[VincentTam]

@robinmetral The instructions for hosting Staticman can vary greatly depending on the host. IMHO, that would be _too_ specific to be included in the official documentation. Changing the public RSA key from OpenSSH to PEM form can already clarify 50% of the mystery, and avoid future repetition of similar mistakes.

I suggest you to be cautious when reading @gabeluci's article. It's fine to run the following command if the RSA keypair belongs to a dummy (_not personal_) account.

openssl rsa -outform PEM -in ~/.ssh/id_rsa -out id_rsa.pem

However, in @gabeluci's linked article, it seems that it's a _personal_ SSH key.

My ~/.ssh/id_rsa.pub looks like this:

ssh-rsa key [email protected]

From a cryptographical point of view, each key pair represents an _uncompromisable_ identity. Using a _personal private key_ for Staticman API is analogous to passing your ID card to a robot. That's why a dummy GitHub and/or GitLab account has to created for Staticman. Instead of conversion of the existing SSH key pair, a _new_ key pair should be created.

[gabeluci]

I see where you're coming from, but the only difference between a "personal" key and a "bot's" key is which GitHub account it's associated to. If you associate the key to your bot account, then it's your bot account's key and not your personal key anymore. Although I do understand that the key stored at ~/.ssh/id_rsa is used by default.

When I created mine, I just followed GitHub's own instructions in creating this key, which tells you accept the default location of ~/.ssh/id_rsa. I created it solely for this purpose, so, in my case, it was never intended to be my personal key.

But it is probably worth noting, in case someone does already have a key generated for their personal account. I will update my article to note that. The command can always be made generic by not including a specific path:

openssl rsa -outform PEM -in /path/to/rsa/key -out id_rsa.pem

[VincentTam]

@gabeluci In fact, the RSA key used by Staticman has nothing to do with SSH, GitHub and GitLab. One should _never_ create an SSH key for the dummy GitHub/GitLab account (a.k.a. Staticman bot), one should use a _personal access token_ instead. The former grants _full_ rights, whereas the later grants only the _necessary_ ones.

When I created mine, I just followed GitHub's own instructions in creating this key, which tells you accept the default location of ~/.ssh/id_rsa. I created it solely for this purpose, so, in my case, it was never intended to be my personal key.

So you are admitting that ~/.ssh/id_rsa represents your _personal_ GitHub account?

[gabeluci]

So you are admitting that ~/.ssh/id_rsa represents your personal GitHub account?

No, it doesn't - at least not in my case.

In fact, the RSA key used by Staticman has nothing to do with SSH, GitHub and GitLab.

So are you saying that Staticman only uses the RSA key for encryption and nothing more? I guess that would make sense since the Requirements ask for a GitHub access token too.

I associated the RSA key to my bot account only because the Requirements section of the Staticman docs refers to it as an "SSH key" and links to the Connecting to GitHub with SSH page, so, to me, that implied that I needed to follow every step in that documentation, including associating it to the GitHub account that Staticman is using.

@eduardoboucas I could use some clarification there: Does Staticman only use the RSA key for encryption and nothing else?

[VincentTam]

There's _no_ need for clarifications from the project owner. A picture of @staticmanlab is worth a thousand words.

You can see that @staticmanlab is _working_ in light of its activities shown in its user profile.

The official README is _incorrect_, that's why I'm saying that " changing the public RSA key from OpenSSH to PEM form can already clarify 50% of the mystery".

P.S. I can finally understand why you insist on that complicated command instead of the _simpler_ one suggested in https://github.com/eduardoboucas/staticman/pull/219#issuecomment-419266449.

[eduardoboucas]

@eduardoboucas I could use some clarification there: Does Staticman only use the RSA key for encryption and nothing else?

@gabeluci that's correct.

[gabeluci]

Thanks @eduardoboucas.

P.S. I can finally understand why you insist on that complicated command instead of the _simpler_ one suggested in #219 (comment).

@VincentTam I think you have misunderstood my intentions here. I am not insisting on anything. I am just trying to understand what's going on.

Now that I understand that the RSA key has nothing to do with GitHub, then yes, it makes more sense to ignore GitHub's documentation on the matter and just create it in PEM format in the first place. I'll update my article.

[VincentTam]

@gabeluci Sorry for such misunderstanding.