RFC 8555: https://tools.ietf.org/html/rfc8555
Let's Encrypt is planning to roll out the next version of ACME which they call API v2 endpoint:
https://letsencrypt.org/2017/06/14/acme-v2-api.html
It isn't clear yet what v2 will actually look like because the latest ACME draft v07 has not been standardized yet:
We refer to "V2" as whatever will be standardized by the IETF as an RFC.
The ACME specification hasn't become an RFC yet - draft-07 is the current document
being worked on. It's likely very close to what will be the RFC and the "V2" that we implement.
(from https://community.letsencrypt.org/t/request-whats-new-in-acme-v2-post/37042/6)
Here's the diff between the currently implemented protocol and the latest draft version:
https://tools.ietf.org/rfcdiff?url1=https://tools.ietf.org/id/draft-ietf-acme-acme-01.txt&url2=https://tools.ietf.org/id/draft-ietf-acme-acme-07.txt
The issue is to track that work. Just want to get a head start.
@bradfitz please, assign this to me.
A set of testing-focused packages implementing the upcoming Let's Encrypt endpoint: https://github.com/letsencrypt/pebble.
Could be useful.
Sorry for the noise, trying to edit the issue on mobile.
@x1ddos @mdlayher a new draft is available https://datatracker.ietf.org/doc/draft-ietf-acme-acme/
I'm start looking on it and seems there are infos needed to start work on it, do you already implement some part of specs? I'm considering to start work on it...
@mavimo I haven't read it yet. Do you know if that's the closest to what v2 will be based on or will there be more iterations?
@x1ddos tech spec seems stable (endpoint naming, response payload, ...) but spec are not closed. I suppose should possible starting a WIP implementation (using pebble) to test client implementation.
https://github.com/ietf-wg-acme/acme/
https://tools.ietf.org/wg/acme/agenda
https://datatracker.ietf.org/meeting/100/materials/slides-100-acme-acme-acme/
From Let's Encrypt,
"We are planning to have a public test API endpoint up by January 4, and we’ve set a date for the full launch: Tuesday, February 27."
https://letsencrypt.org/2017/12/07/looking-forward-to-2018.html
Would've been nice if they actually referred to some documentation, unless it's going to be exactly the latest IETF draft but I have some doubts.
@x1ddos any observed divergences from draft-09 would be welcome as issues on the Boulder repo.
@cpu yeah I saw the announcement not long ago. This is great! Will make sure to file issues if I notice anything.
From #23342:
I think there might actually be other ACME-compatible servers but even if there aren't any, I know for a fact there exist non-public implementations. I'd like to keep v1 working.
I'm now through the bulk of the implementation of v2, and there is no reasonable way to keep support for both the previous version and draft-09 in the same package due to substantial divergence in the ordering flow, object fields, and requests. A few helpers could be re-used, but that's about it. How intent are you on keeping support for "v1"? (which is really a Let's Encrypt specific version that doesn't match any previous draft).
@bradfitz Do you have any thoughts on if there's a good approach to pulling in pebble for integration tests? Worst case I can make a separate integration testing package and throw it on GitHub, but I'm wondering if there's a good way to get it running with every CL. I haven't come up with a good way to do it without it getting vendored in the x/crypto
repo, which seems like a no-go.
I'm now through the bulk of the implementation of v2
@titanous where is it. Can I take a look?
@titanous also, it's a shame you're implementing something without at least consulting here. why duplicate work?
also, it's a shame you're implementing something without at least consulting here. why duplicate work?
I did consult, in the inadvertently duplicated #23342, but GitHub appears to have never sent me your reply. Anyway, this was just a weekend project, it's really not a problem for me if we have to change it or even throw the whole thing away. I'll push a CL in a moment.
Change https://golang.org/cl/86635 mentions this issue: acme: Implement IETF draft-09
Aside from too many unnecessary renamings, especially of exported types, golang.org/cl/86635 is a good start. But, I'd like to keep v1 working as is, without breaking. Besides, autocert.Manager would need to be changed too, in order to work with v2.
So, how about an acme.ClientV2
or acme/v2
package, or change the type of acme.Client
to an interface and have multiple implementations, v1 and v2. It is also easy to automatically infer which version a URL points to, for instance by looking at the Directory response during discovery.
But, I'd like to keep v1 working as is, without breaking.
Given that v1 is never going to change (it’s a Let’s Encrypt-specific implementation and they are not going to develop the API after deprecation), wouldn’t it make more sense just to snapshot the current v1 implementation into a GitHub repo somewhere? Alternatively we can put v2 into a different package, but do you want to have autocert also support v1?
Besides, autocert.Manager would need to be changed too, in order to work with v2.
Yes, I didn’t want to do that until we’d finalized the v2 client API.
I really don’t think that an interface would properly encapsulate both versions under the same methods and types.
draft-09 is very close to being the final RFC, so I think it’s worth strongly considering making a clean API break to match the new spec (hence the various renames) instead of saddling ourselves with vestiges of what will be a forgotten draft API design (v1).
do you want to have autocert also support v1
Yes, because I'd like autocert to always work regardless, as in "go get -u .../autocert" and it works at any point in time.
The v2 implementation isn't stable yet anyway. In my experience, being close to final doesn't make it final and may never will. So, how about we create acme/v2
package and place everything v2-related in there, until it's polished or at least until works well with Let's Encrypt and then see how we can integrate it into the acme
and acme/autocert
packages.
Yes, because I'd like autocert to always work regardless, as in "go get -u .../autocert" and it works at any point in time.
To be clear, I was proposing that the cutover to v2 happen after LE turns on their production endpoint, so autocert will always just work.
So, how about we create acme/v2 package and place everything v2-related in there
Sounds good, I'll ping the CL when I have added the remaining tests.
or may even put it in acme/internal/v2. this is what we did with acme
initially and it worked quite well.
Am 08.01.2018 15:54 schrieb "Jonathan Rudenberg" notifications@github.com:
Yes, because I'd like autocert to always work regardless, as in "go get -u
.../autocert" and it works at any point in time.To be clear, I was proposing that the cutover to v2 happen after LE turns
on their production endpoint, so autocert will always just work.So, how about we create acme/v2 package and place everything v2-related in
thereSounds good, I'll ping the CL when I have added the remaining tests.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/21081#issuecomment-355987978, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AABjPfKzBeMlA5vS2pBYm86L4NogwA6Xks5tIiw8gaJpZM4Ocd5Y
.
I don't want to bikeshed on names, but last I heard, the IETF has refused calling this v2 as it's the first version they're ratifying.
I'm concerned that if the IETF ever releases v2 and Go already has a v2, it'll lead to more confusion.
Perhaps naming it ietfv1-acme is the right thing? @x1ddos @titanous
Let's Encrypt has launched their v2 service: https://community.letsencrypt.org/t/acme-v2-production-environment-wildcards/55578
/cc @FiloSottile
The implementation in https://golang.org/cl/86635 is basically ready to go, there are a few minor data structure changes that need to be made to line up with the version that shipped.
It may make sense to update autocert as well in the CL, and just ship this as a complete replacement for v1.
What is missing on CL 86635 ? Letsencrypt ACMEv2 and wildcard services are live(since March 13 ).
What is missing on CL 86635 ?
I guess fixes and improvements that have happened in the mean time:
https://go.googlesource.com/crypto/+log/master/acme
@x1ddos The CL has been updated with all relevant changes. I believe it's ready to go. We should decide if we're going to merge in as a separate package, or update autocert for the API modifications and replace the existing package.
I believe updating autocert for the API modification is risk-free as Letsencrypt runs v2 in production now and as far as I know there was no other CA running acme v1. However the safest bet would be to place acme into a v2 package. We could also add a "Version" field to acme/autocert.Manager if we want to use a single autocert package.
I too would recommend against replacing existing package. Not now anyway.
Let's Encrypt's v2 endpoint has been running for what, 2 month now? That's very little.
@titanous thanks, I'll review latest changes. Just been a bit busy with I/O and other things.
Ideally though, I'd love to have autocert package not changing its exported types at all and work seamlessly with different providers. I guess it's similar to how golang.org/x/oauth2 package does it - can be used with different oauth2 providers.
I believe updating autocert for the API modification is risk-free as Letsencrypt runs v2 in production now and as far as I know there was no other CA running acme v1.
I believe the opposite is true. There are no other CAs that I am aware of with an ACME v2 API in production. There are other CAs running ACME implementations that target Certbot compatibility and are implementing ACME v1. As one concrete example, BuyPass has an ACME server implementing V1 directory endpoints ("new-reg", "new-cert", etc): https://api.buypass.com/acme/directory
I believe the opposite is true.
Then a separate acme v2 package is the only option with seamless integration with autocert.
How can I checkout / fork https://go-review.googlesource.com/c/crypto/+/86635 ? It seems that merging into x/crypto/acme will take a while.
How can I checkout https://go-review.googlesource.com/c/crypto/+/86635
Top-right corner "More" => Download patch:
any chance to get this merged before the Go1.11 release(rc) ?
@winteraz The crypto repo is not versioned with Go releases, so it can be merged independently of the Go release cycle. I believe we are still waiting for review bandwidth from @x1ddos.
Thanks for all of the hard work on this @titanous, @x1ddos, and others! It looks like this is still waiting final review, but that it's gone through several rounds of review up to this point. Has anyone been using this in production / run into any issues? I'm considering downloading and moving over to this in the meantime.
I can confirm cert-manager has been running with a vendored copy of this
library for a while now, and we've not had issues! (Since approx march)
On Tue, 2 Oct 2018 at 19:36, cristiangraz notifications@github.com wrote:
Thanks for all of the hard work on this @titanous
https://github.com/titanous, @x1ddos https://github.com/x1ddos, and
others! It looks like this is still waiting final review, but that it's
gone through several rounds of review up to this point. Has anyone been
using this in production / run into any issues? I'm considering downloading
and moving over to this in the meantime.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/21081#issuecomment-426382932, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAMbP9PLvZL9Nf18awKTjH7AsO79lXN5ks5ug7IigaJpZM4Ocd5Y
.
Have you seen https://community.letsencrypt.org/t/acme-breaking-change-most-gets-become-posts/71025 announcement:
Right now our recommendation for client developers is: If you are working on ACMEv2 implementation, it is probably better to wait for this upcoming spec change, unless you urgently need wildcard support. Please keep an eye on the ACME mailing list.
@x1ddos I had not seen that, thank you for sharing.
It looks like the breaking change has been implemented and is available in the production API. For some reason the post linked above wasn't updated, but I found the official announcement on the community site in a separate thread:
https://community.letsencrypt.org/t/acme-v2-scheduled-deprecation-of-unauthenticated-resource-gets/74380
Not suggesting anyone jump into this, but did want to update everyone on this thread in case anyone else was following that last announcement waiting for an update.
@x1ddos Is there anything we can do to help move this along? I'm unclear if the issue of supporting v1 is a blocker (seems there is no definitive decision?), or if this is just pending final review from you or someone else.
It seems like this change is nearly complete, and once a decision has been made above mostly needs a rebase and the POST-as-GET
update which Let's Encrypt set live in October.
@cristiangraz there are clients which will be using the so-called "v1 Let's Encrypt" or rather loosely based around draft-02.
@bradfitz and I talked during the last GopherCon and we both liked the idea of keeping the lower level x/crypto/acme.Client as is, just add a couple methods completely new like NewOrder introduced around latest draft versions. It would prefer using newer endpoints, for instance "new-nonce" resource instead of hitting "HEAD /anything".
Essentially, you'd point the client to a directory URL and it'll figure out whether it's the old draft-02 or the latest draft-18.
It seems like this change is nearly complete, and once a decision has been made above mostly needs a rebase
If you're talking about golang.org/cl/86635, I don't think it's that simple. Current x/crypto/acme had quite a few changes and improvements since then. I'd rather just add few methods to the existing client to make it work with draft-18 and then adjust autocert.
@x1ddos Can you help me understand the requirement of maintaining compatibility with draft-02? I saw @bradfitz mentioned that as being important in the other duplicate thread on this topic, but what I'm struggling to understand is why autocert
couldn't be updated to use the new v2 API internally. Anyone who updates would just retrieve certs using the new flow and endpoints. The autocert
APIs surface area looks broad enough (e.g. GetCertificate()
).
I'm planning on moving to v2 for thousands of domains to get access to wildcard certs, and the account(s) from v1 are transferrable to v2 (assuming no pending authorizations created from the v1 API are used). I feel like I may be missing something obvious here.
add a couple methods completely new like NewOrder introduced around latest draft versions. It would prefer using newer endpoints, for instance "new-nonce" resource instead of hitting "HEAD /anything".
That makes sense. But I'm unclear on this point:
Essentially, you'd point the client to a directory URL and it'll figure out whether it's the old draft-02 or the latest draft-18.
If a user initializes client with a v1 directory URL, what prevents them from calling client.NewOrder()
? Or a client with a v2 directory URL bypassing NewOrder()
and trying to create authorizations directly. I've only skimmed the spec at this point, maybe @titanous has more thoughts having done so much of this work already, but it seems like that could be more error prone and/or difficult to accomplish without introducing more complexity (which circles back to my first question on maintaining draft-02 compatability)
requirement of maintaining compatibility with draft-02
There are many ACME servers in the wild these days and they support different versions. Fun fact, EJBCA's ACME server implements draft-12, for instance. So, not draft-02, not draft-18 but somewhere in the middle. :) There are other examples.
why
autocert
couldn't be updated to use the new v2 API internally
Autocert uses acme.Client behind the scenes. This issue is in fact tracking the support of what will be the final draft in the acme.Client. Then autocert can add support to use this new API.
Anyone who updates would just retrieve certs using the new flow and endpoints
Why rush after shiny new things which haven't been completely finalized yet, while the current stable API works just fine?
I'm planning on moving to v2 for thousands of domains to get access to wildcard certs
If you're taking about autocert, I don't think it'll help you anyway. It never requests wildcard certs and besides dns-01
challenge based validation is typically required. At least by Let's Encrypt CA if things haven't changed. Autocert doesn't support dns-01
challenges. See #23198 for more details.
If a user initializes client with a v1 directory URL, what prevents them from calling
client.NewOrder()
?
Nothing. In that case the caller will most likely receive an error.
If you're using the lower level acme.Client, you'll probably already know what you're doing after having read the ACME spec. Autocert is different. It'll know whether NewOrder is a valid endpoint and make the calls accordingly.
Or a client with a v2 directory URL bypassing
NewOrder()
and trying to create authorizations directly
That is actually supported by the latest spec. See Pre-Authorization in draft-18 section 7.4.1. The spec doesn't enforce the support though. Depends on the CA.
Hope this clarifies some things.
@x1ddos Thanks for the reply, this is very helpful 👍
If you're taking about autocert, I don't think it'll help you anyway.
Nope, we have our own implementation built on top of this library's API.
I'm not familiar enough with the v2 spec at this point to provide an implementation, but I do manage a large-scale system in need of wildcard certs. I'm going to take this current CL, get it integrated into our production environment as soon as I have some bandwidth in order to get familiar with the spec and the various implementation issues. If no one has jumped on this at that point, I'll revisit this thread to propose a structure that meets the requirements above and see if we can build off of all of the great work @titanous has already done here to get this finalized.
Just to set expectations for anyone following this thread that it may be a few months.
FWIW I'm already working on adding a minimal set of new constructs to support draft-18 in the acme package without breaking existing clients, as opposed to https://golang.org/cl/86635 which either proposes a completely new version or overwrites the current one.
I saw this and I think it needs attention: https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430
In November of 2019 we will stop allowing new account registrations through our ACMEv1 API endpoint. Existing accounts will continue to function normally.
In June of 2020 we will stop allowing new domains to validate via ACMEv1.
In June of 2021 we will entirely disable ACMEv1 as a viable way to get a Let’s Encrypt certificate.
BTW, it seems that the title of this issue needs to be updated, since the ACME protocol has become an IETF standard (RFC 8555).
Hello.
What status of issue now?
Is https://go-review.googlesource.com/c/crypto/+/86635/ work (may use patched version instead of original package)?
Will it merge before September?
Context of question check if I need switch to other library for can create new accounts.
Still working on it: see https://github.com/golang/go/issues/21081#issuecomment-464208674.
https://golang.org/cl/86635 is too large and very hard to review imho.
Any progress?
Or does it exist any other GO package supporting RFC 8555, which is lite and usable (for a program which makes cert request through DNS-challenges)?
Or does it exist any other GO package supporting RFC 8555, which is lite and usable (for a program which makes cert request through DNS-challenges)?
@ClaesBas github.com/eggsampler/acme
and github.com/go-acme/lego
both implement RFC 8555 and DNS-01 challenges.
Change https://golang.org/cl/182937 mentions this issue: acme: add support for RFC8555 compliant discovery
So the plan is to extend the acme package functionality without disrupting existing usage. https://golang.org/cl/182937 is the start. I'm hoping to complete this by end of July and then move on to autocert in August.
@x1ddos I support trying not to break existing users, but if it means every field foo
will get a fooRFC
duplicate, I'm not sure it's worth it. In a few years I expect only the RFC mode will be in use, and I wouldn't want every usage to have to write RFC
over and over.
Can we get a full proposal for the whole API before we start implementing pieces of it, so we can take an overview look, consider alternatives, and make a decision? Feel free to make a new issue for the proposal and to tag me.
I wouldn't want every usage to have to write RFC over and over
Nobody is going to write RFC. It's internal representation. I think you misunderstood what golang.org/cl/182937 is trying to do.
In a few years I expect only the RFC mode
I expect for the non-RFC compliant bits to go away from the acme package much sooner than few years.
Can we get a full proposal for the whole API before we start implementing pieces of it
There's no "whole API". There's just a couple new methods relative to the newOrder
, as opposed to pre-authorization. Nothing is changing in the package API.
Feel free to make a new issue for the proposal and to tag me.
I'm not going to create yet another issue. This one is enough and contains all context. If you need more details just read from the top.
@x1ddos I did in fact mistake the CL for an exported change, thanks for pointing that out.
Still, I understand from the rest of this issue that the CL is part of a chain that will change exported behavior. For that, I need you to file a proposal.
@FiloSottile it doesn't make any sense to submit a formal "Changes to Go" proposal in this context imho. @bradfitz and I wrote and maintained the packages from scratch. We didn't make any proposals. I've already discussed this with @bradfitz long ago and more recently with others like @jdkasten.
Feel free to start the proposal process yourself and take this over if you disagree.
@x1ddos I understand you are one of the major authors of the package, and I appreciate your continued maintenance of it (which reminds me, feel free to send a CL to add yourself to the owners in https://go.googlesource.com/build/+/master/devapp/owners/table.go), but this is now relied upon by a wide community, and it's not a Google package but part of the Go project.
The point of the proposal process is to surface the kind of discussion you might have had with @bradfitz and @jdkasten, and to get the input of the rest of the community, of which Google is only a part. It's also a very valuable process to iron out details.
RFC support is without a doubt a change complex and nuanced enough to deserve a proposal. Please follow the process, let me know if I can help, and thank you for contributing your time.
Ok. We've talked offline and clarified a few things. I'll submit a proposal and we can hopefully sort this out.
Sent the proposal in https://github.com/golang/go/issues/33229.
Change https://golang.org/cl/191602 mentions this issue: acme: support RFC8555 account management
Change https://golang.org/cl/191601 mentions this issue: acme: add KID variant to jwsEncodeJSON
Change https://golang.org/cl/191603 mentions this issue: acme: fetch fresh nonces from newNonce resource
Change https://golang.org/cl/192779 mentions this issue: acme: implement new order-based issuance methods
Change https://golang.org/cl/194379 mentions this issue: acme: update existing methods for RFC8555
Change https://golang.org/cl/194681 mentions this issue: acme/internal: add a prober program
We're getting close to the end of month.
If I don't hear objections in the next day or two, I'm going to start submitting +2'ed changes to make it easier for me to create a final CL to support RFC version in autocert which will naturally use all of those #rfc8555 CLs. None of the CLs break existing users.
All acme pkg CLs:
https://dev.golang.org/reviews?q=acme
https://go-review.googlesource.com/q/hashtag:%22rfc8555%22
The acme.Client is done at this point. One notably missing bit is External Account Binding but you wouldn't care at all if using only Let's Encrypt. I'll be working on it once I'm done with RFC support in acme/autocert.
In pre-RFC world, typically draft-02, you'd do something like this:
The new order-based flow looks something like this:
In RFC, there's still a pre-authorization flow very similar to the above draft-02 but it's optional. For instance, Let's Encrypt does not currently implement it. For the CAs which do support it:
Note that acme.LetsEncryptURL still points to their old v1 aka draft-02 implementation. You'll need to set client.DirectoryURL
to https://acme-v02.api.letsencrypt.org/directory
to point it to their RFC implementation when creating a new client but watch out for your existing resources: keys and authorizations obtained and registered on v1 will not work with v2, I believe. You'll need to start fresh.
We'll want to update acme.LetsEncryptURL
to their RFC implementation at some point.
Docs at https://godoc.org/golang.org/x/crypto/acme are up to date.
If it's of any use, I have a prober program which works with order-based flow as well as both RFC and draft-02 pre-authorization flows in https://golang.org/cl/194681.
Next is acme/autocert. This should be just a single CL. Will send it soon.
Change https://golang.org/cl/199520 mentions this issue: acme/autocert: support ACME RFC 8555
It would be really nice to update that URL constant and finish off this work. I just got hit with the brownout that Let's Encrypt is putting in place and noticed a straight-forward update of crypto/acme didn't fix my problem.
(Boy, am I glad they did that brownout)
@jmhodges did you identify a workaround? We're hit with the same problem
Oh, the solution is straightforward.
When creating your acme.Client
, set its DirectoryURL
field to "https://acme-v02.api.letsencrypt.org/directory"
(for production use) or "https://acme-staging-v02.api.letsencrypt.org/directory"
(for live testing).
I think you can't reuse accounts from the v01 API in the v02 one (because of the account's URI being a permanent identifier that includes the API URLs). Because of that, you'll need to make new accounts for the new API.
You can reuse accounts! No changes there needed.
(This, of course, means that folks have to write code to make acme.Client
, and can't rely on autocert.Manager
to do the right thing.)
https://golang.org/cl/199520 (mentioned above) is in review. Feel free to help test that out.
Change https://golang.org/cl/200638 mentions this issue: acme: update Directory URL to Let's Encrypt v2
Hello, we have customers using an application in production with autocert and letsencrypt, is there a stable workaround that we can use today to enable v2 support for them (I.e. is manually creating the acme.Client with the directory URL referenced above sufficient?), or are more changes needed before we can support this?
@cohix you can apply https://golang.org/cl/199520 manually or wait until it's merged.
@x1ddos is is reasonable to expect it will be merged in time for the v1 EOL next month? I'd prefer not trying to deploy a non-merged patch to my production application :P
@cohix it is now merged. see commit message for details: https://github.com/golang/crypto/commit/0e8c3a9068a2fb325459490d2ce7baa4ffe2d00f
@x1ddos great news. Tested it just now and all is well. Does it address https://community.letsencrypt.org/t/acme-v2-scheduled-deprecation-of-unauthenticated-resource-gets/74380 as well?
👏 hat tip to you all, thank you for averting a small crisis :)
Change https://golang.org/cl/203919 mentions this issue: acme/autocert: always pass AuthzURLs from AuthorizeOrder to deactivatePendingAuthz
Most helpful comment
The implementation in https://golang.org/cl/86635 is basically ready to go, there are a few minor data structure changes that need to be made to line up with the version that shipped.
It may make sense to update autocert as well in the CL, and just ship this as a complete replacement for v1.