* Initial implementation of ZeroSSL API issuer
Still needs CA support for CommonName-less certs
* Accommodate ZeroSSL CSR requirements; fix DNS prop check
* Fix README example
* Fix comment
* Changed solver DNS propagation check to only check authoritative nameservers directly if there are no explicitly given resolvers.
* Changed solver DNS propagation check to only succeed of any one of the checked nameservers has the required TXT entry
Previously, simultaneously solving for example.com and *.example.com
using the DNS challenge (i.e. both SANs on the same cert, which CertMagic
doesn't do; but this is a general-purpose library) would cause a deadlock
because both those names use the same-name TXT record. This caused
problems with previous dependencies like lego and legacy DNS solvers,
but the new acmez library and libdns packages support this.
I've removed the locking around same-name records, which resolves
the deadlock, and improves efficiency as well, since we can now solve
challenges for example.com and *.example.com at the same time.
This is necessary to eliminate confusing naming conventions, since now
we have Manager types, having an issuer called ACMEManager was
confusing.
CertificateManager is a redundant name as this package is called
CertMagic, so that a Manager manages certificates should be obvious.
It's also more succinct. Plus, it's consistent with Issuer which is not
named CertificateIssuer.
Calling checkAuthoritativeNss is wrong as it's not inter-changeable with
checkDNSPropagation.
Though IIUC it's not necessary to follow CNAME when in Wait(), with or
without OverrideDomain, let's wait until the override domain gets some
usage to change this. The reason that following CNAME is not necessary
is that CNAME cannot co-exist with other DNS records, if we succeed in
setting a TXT record on that domain, it cannot have a CNAME record.
@IndeedNotJames
* Add context propagation to the Storage interface
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
* Bump to Go 1.17
* Minor cleanup
* filestorage: Honor context cancellation in List()
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
* Add OverrideDomain option to DNS01Solver
This is to delegate the challenge to a different domain. With this
change, the solver no longer follows CNAME chain when checking for
propagation as well.
* Update solvers.go
* Only check the authoritative NS when OverrideDomain is set
and keep the old code path otherwise.
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* Fix TLS-ALPN-01 challenge for IP Identifiers
See #133
* Add tests for challengeKey function
* Add more tests
* Fix PR comments
* Remove deletion of TLS-ALPN-01 challenge certificate
Turns out this is needed when solving the HTTP challenge in Caddy, in certain situations.
This does not provide access to challenge info in distributed challenge storage (that would require a Config, and isn't exported anyway since it is handled internally).
This allows any challenges initiated within the process to be solved by whatever HTTP or TLS server is running, even if they do not know about the challenges themselves.
This is useful when a process has multiple servers running, but only one can solve the challenges (which is often the case, since a socket belongs to one listener at a time) and they do not know about each other or share configs. The trick is to wrap the solvers with a thin wrapper that stores all the challenge info in memory while the challenge is active.
A nice side-effect is I've simplified/unified the code that gets the challenge info when actually solving the challenges.
* Implement multiple issuer support
This change refactors Config.Issuer to be Config.Issuers, an array of
issuers. Each Issuer will be tried in turn until one succeeds. During
retries, each attempt will try each configured Issuer. When loading
certs from storage, CertMagic will look in each Issuer's storage
location for a qualifying asset. If multiple Issuers have one in storage
then the most-recently-issued cert will be selected.
This is a breaking change in that Config now accepts a slice of Issuers
rather than a single Issuer. The Revoker field is removed, as supporting
it is optional anyway. If the Issuer is also a Revoker, it can be used
implicitly to revoke certificates.
Also added a const for ZeroSSL's ACME endpoint.
* Load matching wildcard on-demand from storage
With this change, a config using on-demand TLS can load a certificate
for "sub.example.com" from storage using a matching wildcard cert
(i.e. "*.example.com") if no better matching certificate is available.
* Fix distributed solving with tls-alpn challenges
The type assertion in handshake.go was problematic since there's no
guarantee that an ACME issuer would be a concrete ACMEManager type.
Refactored the code to accept IssuerKey values generally, rather than
specific ACMEManager values only.
This fixes solving tls-alpn challenges in distributed settings.
More cleanup can be done, another time.
Wildcard domain names collide with the same subdomain for the ACME TXT
record as the non-wildcard parent domain (for example, example.com and
*.example.com both use _acme-challenge.example.com), so we need to solve
those challenges mutually exclusively.
One potential problem with this current implementation is that we don't
wait for the DNS record to un-propagate after it is deleted; I've found
that re-running it works fine, after waiting just a few seconds. I am
not sure how to generalize this logic in all cases though. It is likely
provider-dependent. (I was testing with Cloudflare.)
Should fix https://github.com/caddyserver/caddy/issues/3474
Before when we used lego as our ACME library, DNS solvers abounded in
the lego repository and they could be used directly. Our new acmez lib
is very lightweight, and "bring-your-own-solvers", let alone your own
DNS provider implementations.
DNS providers are implemented in libdns: https://github.com/libdns
This commit adds an implementation of acmez.Solver that solves the DNS
challenge using libdns providers.
Unlike the other solvers, this one is exported because it is not a
challenge type that is enabled by default, and there is more config
surface.
We borrowed some DNS utility functions and tests from the lego repo.
But this is a very lightweight implementation that has a much, much
simpler API and smaller footprint.
After twenty trials, was unable to reproduce nondeterministic stalling
with successful validations.
Was able to consistently reproduce hanging after failed validations,
which this fixes.
Breaking changes; thank goodness we're not 1.0 yet 😅 - read on!
This change completely separates ACME-specific code from the rest of the
certificate management process, allowing pluggable sources for certs
that aren't ACME.
Notably, most of Config was spliced into ACMEManager. Similarly, there's
now Default and DefaultACME.
Storage structure had to be reconfigured. Certificates are no longer in
the acme/ subfolder since they can be obtained by ways other than ACME!
Certificates moved to a new certificates/ subfolder. The subfolders in
that folder use the path of the ACME endpoint instead of just the host,
so that also changed. Be aware that unless you move your certs over,
CertMagic will not find them and will attempt to get new ones. That is
usually fine for most users, but for extremely large deployments, you
will want to move them over first.
Old certs path:
acme/acme-staging-v02.api.letsencrypt.org/...
New certs path:
certificates/acme-staging-v02.api.letsencrypt.org-directory/...
That's all for significant storage changes!
But this refactor also vastly improves performance, especially at scale,
and makes CertMagic way more resilient to errors. Retries are done on
the staging endpoint by default, so they won't count against your rate
limit. If your hardware can handle it, I'm now pretty confident that you
can give CertMagic a million domain names and it will gracefully manage
them, as fast as it can within internal and external rate limits, even
in the presence of errors. Errors will of course slow some things down,
but you should be good to go if you're monitoring logs and can fix any
misconfigurations or other external errors!
Several other mostly-minor enhancements fix bugs, especially at scale.
For example, duplicated renewal tasks (that continuously fail) will not
pile up on each other: only one will operate, under exponential backoff.
Closes#50 and fixes#55
This solves several issues related to solving for multiple names
concurrently. The basic idea is that now we always use our own solvers,
which is actually much simpler. We just wrap them in a distributedSolver
which writes the keyAuth material to storage. Our solvers then proceed
to solve the challenges: either by allowing whatever is currently
listening on the challenge port to solve it, or by starting their own
servers. Our solvers keep track of how many challenges each solver is
answering, and the "last one out turns off the lights" so to speak.
Also, where we used to try dialing a port then listening if it was
available, now we just try listening, and if it fails, we make sure it
is in use by dialing it. I've added locking around this as well to
ensure that races for the socket, along with the counters, do not happen.
Overall, this is a much improved solver implementation that can handle
more use cases at a larger scale than before.
Also, a minor data race was revealed in user.go, which only happens in
some rare edge cases as far as I can tell, but I marked them with a TODO
so we can get around to fixing them later.
This allows for user-loaded certificates to be associated with arbitrary
values such as user-provided IDs or categories. This can be useful if
multiple certificates satisfy a ClientHello but if a specific one still
needs to be chosen. See for example:
https://github.com/mholt/caddy/issues/2588
This is a breaking API change since we need to expose a tags parameter
to the caching functions, but we're not 1.0 yet so we will try this
API change and see how it goes.
* Significant refactor
This refactoring expands the capabilities of the library for advanced
use cases, as well as improving the overall architecture, including
possible memory leak fixes if used over a long period with many certs
loaded into memory. This refactor enables using different configs
depending on the certificate.
The public API has changed slightly, however, and arguably it is
slightly less convenient/elegant. I have never quite found the perfect
design for this package, and this certainly isn't it, but I think it's
better than what we had before.
There is still work to be done, but this is a good step forward. I've
decoupled Storage from Cache, and made it easier and more correct for
Configs (and Storage values) to be short-lived. Cache is the only value
that should be long-lived.
Note that CertMagic no longer automatically takes care of storage (i.e.
it used to delete old OCSP staples, but now it doesn't). The functions
to do this are still there and even exported, and now we expect the
application to call the cleanup functions when it wants to.
* Fix little oopsies
* Create Manager abstraction so obtain/renew isn't limited to ACME
* use go-acme/lego
* Use master branch of go-lego/acme since v2.3.0 still has a dependency on xenolf/lego
* Use golangci-lint since gometalinter is depricated
* different way of installing golangci-lint for appveyor
* Removing golangci-lint from Appveyor because of https://github.com/client9/shlib/issues/13
Also adjust clients so that they use the configured HTTPPort or
HTTPSPort for solving challenges, if different from the default
challenge port (not as preferred as the Alt*Port values, of course)
