* Initial implementation of ZeroSSL API issuer
Still needs CA support for CommonName-less certs
* Accommodate ZeroSSL CSR requirements; fix DNS prop check
* Fix README example
* Fix comment
These are useful for advanced applications (like Caddy) which would
like to remove certificates from the
cache in a controlled way, and operate the
cache with new settings while running.
Eliminates a bajillion nil checks and footguns
(except in tests, which bypass exported APIs, but that is expected)
Most recent #207
Logging can still be disabled via zap.NewNop(), if necessary.
(But disabling logging in CertMagic is a really bad idea.)
* Add context propagation to the Storage interface
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
* Bump to Go 1.17
* Minor cleanup
* filestorage: Honor context cancellation in List()
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
This work made possible by Tailscale: https://tailscale.com - thank you to the Tailscale team!
* Implement custom GetCertificate callback
Useful if another entity is managing certificates and can
provide its own dynamically during handshakes.
* Refactor CustomGetCertificate into OnDemandConfig
* Set certs to managed=true
This is only sorta true, but it allows handshake-time maintenance of the
certificates that are cached from CustomGetCertificate.
Our background maintenance routine skips certs that are OnDemand so it
should be fine.
* Change CustomGetCertificate into interface value
Instead of a function
* Case-insensitive subject name comparison
Hostnames are case-insensitive
Also add context to GetCertificate
* Export a couple of outrageously useful functions
* Allow multiple custom certificate getters
Also minor refactoring and enhancements
* Fix tests
* Rename Getter -> Manager; refactor
And don't cache externally managed certs
* Minor updates to comments
The order of storing the certificates was previously changed so that the
private key would be stored first. For anyone who is creating storage
hooks which push the certificate & key to a 3rd party service (like a
CDN), the certificates are generally refused if uploaded before the
private key.
Loading can trigger the same issue if (for any reason), the certificate
& key have been deleted on the 3rd party service.
I'll admit that this stretches what the storage system should be doing,
but with this trivial change it is really easy to create a reliable
link to a CDN.
I got following panic while Caddy was running:
2021/10/26 08:06:34 panic: certificate worker: runtime error: invalid memory address or nil pointer dereference
goroutine 43 [running]:
github.com/caddyserver/certmagic.(*jobManager).worker.func1()
github.com/caddyserver/certmagic@v0.14.5/async.go:58 +0x65
panic({0x145d400, 0x23d6c50})
runtime/panic.go:1038 +0x215
github.com/caddyserver/certmagic.decodePrivateKey({0xc000738c00, 0x0, 0x0})
github.com/caddyserver/certmagic@v0.14.5/crypto.go:75 +0x2a
github.com/caddyserver/certmagic.(*Config).reusePrivateKey(0xc0003b77c0, {0xc0003b1640, 0x32})
github.com/caddyserver/certmagic@v0.14.5/config.go:602 +0x2b9
github.com/caddyserver/certmagic.(*Config).obtainCert.func2({0x190d3b8, 0xc000655920})
github.com/caddyserver/certmagic@v0.14.5/config.go:487 +0x1d6
github.com/caddyserver/certmagic.doWithRetry({0x190d310, 0xc0000b0440}, 0xc00003bd40, 0xc0007afba8)
github.com/caddyserver/certmagic@v0.14.5/async.go:106 +0x1cc
github.com/caddyserver/certmagic.(*Config).obtainCert(0xc0003b77c0, {0x190d310, 0xc0000b0440}, {0xc0003b1640, 0x32}, 0x0)
github.com/caddyserver/certmagic@v0.14.5/config.go:572 +0x58e
github.com/caddyserver/certmagic.(*Config).ObtainCertAsync(...)
github.com/caddyserver/certmagic@v0.14.5/config.go:427
github.com/caddyserver/certmagic.(*Config).manageOne.func1()
github.com/caddyserver/certmagic@v0.14.5/config.go:332 +0x6f
github.com/caddyserver/certmagic.(*jobManager).worker(0x23e0c60)
github.com/caddyserver/certmagic@v0.14.5/async.go:73 +0x112
created by github.com/caddyserver/certmagic.(*jobManager).Submit
github.com/caddyserver/certmagic@v0.14.5/async.go:50 +0x288
According to Go documentation: https://pkg.go.dev/encoding/pem#Decode
p can be nil (first parameter returned) and so it should be checked
before continuing as per this example:
https://pkg.go.dev/encoding/pem#example-Decode
I also added a test to verify that the fix works. Running the test
without the fix causes a panic.
Test: go test -count=1 './...'
* Begin refactor of ObtainCert and RenewCert to allow force renews
* Don't reuse private key in case of revocation due to key compromise
* Improve logging in renew
* Run OCSP check at start of cache maintenance
Otherwise we wait until first tick (currently 1 hour) which might be too long
* Fix obtain; move some things around
Obtain now tries to reuse private key if exists, but if it doesn't exist, that shouldn't be an error (so we clear the error in that case).
Moved the removal of compromised private keys to have logging make more sense.
* Implement multiple issuer support
This change refactors Config.Issuer to be Config.Issuers, an array of
issuers. Each Issuer will be tried in turn until one succeeds. During
retries, each attempt will try each configured Issuer. When loading
certs from storage, CertMagic will look in each Issuer's storage
location for a qualifying asset. If multiple Issuers have one in storage
then the most-recently-issued cert will be selected.
This is a breaking change in that Config now accepts a slice of Issuers
rather than a single Issuer. The Revoker field is removed, as supporting
it is optional anyway. If the Issuer is also a Revoker, it can be used
implicitly to revoke certificates.
Also added a const for ZeroSSL's ACME endpoint.
* Load matching wildcard on-demand from storage
With this change, a config using on-demand TLS can load a certificate
for "sub.example.com" from storage using a matching wildcard cert
(i.e. "*.example.com") if no better matching certificate is available.
* Fix distributed solving with tls-alpn challenges
The type assertion in handshake.go was problematic since there's no
guarantee that an ACME issuer would be a concrete ACMEManager type.
Refactored the code to accept IssuerKey values generally, rather than
specific ACMEManager values only.
This fixes solving tls-alpn challenges in distributed settings.
More cleanup can be done, another time.
Breaking changes; thank goodness we're not 1.0 yet 😅 - read on!
This change completely separates ACME-specific code from the rest of the
certificate management process, allowing pluggable sources for certs
that aren't ACME.
Notably, most of Config was spliced into ACMEManager. Similarly, there's
now Default and DefaultACME.
Storage structure had to be reconfigured. Certificates are no longer in
the acme/ subfolder since they can be obtained by ways other than ACME!
Certificates moved to a new certificates/ subfolder. The subfolders in
that folder use the path of the ACME endpoint instead of just the host,
so that also changed. Be aware that unless you move your certs over,
CertMagic will not find them and will attempt to get new ones. That is
usually fine for most users, but for extremely large deployments, you
will want to move them over first.
Old certs path:
acme/acme-staging-v02.api.letsencrypt.org/...
New certs path:
certificates/acme-staging-v02.api.letsencrypt.org-directory/...
That's all for significant storage changes!
But this refactor also vastly improves performance, especially at scale,
and makes CertMagic way more resilient to errors. Retries are done on
the staging endpoint by default, so they won't count against your rate
limit. If your hardware can handle it, I'm now pretty confident that you
can give CertMagic a million domain names and it will gracefully manage
them, as fast as it can within internal and external rate limits, even
in the presence of errors. Errors will of course slow some things down,
but you should be good to go if you're monitoring logs and can fix any
misconfigurations or other external errors!
Several other mostly-minor enhancements fix bugs, especially at scale.
For example, duplicated renewal tasks (that continuously fail) will not
pile up on each other: only one will operate, under exponential backoff.
Closes#50 and fixes#55
* Significant refactor
This refactoring expands the capabilities of the library for advanced
use cases, as well as improving the overall architecture, including
possible memory leak fixes if used over a long period with many certs
loaded into memory. This refactor enables using different configs
depending on the certificate.
The public API has changed slightly, however, and arguably it is
slightly less convenient/elegant. I have never quite found the perfect
design for this package, and this certainly isn't it, but I think it's
better than what we had before.
There is still work to be done, but this is a good step forward. I've
decoupled Storage from Cache, and made it easier and more correct for
Configs (and Storage values) to be short-lived. Cache is the only value
that should be long-lived.
Note that CertMagic no longer automatically takes care of storage (i.e.
it used to delete old OCSP staples, but now it doesn't). The functions
to do this are still there and even exported, and now we expect the
application to call the cleanup functions when it wants to.
* Fix little oopsies
* Create Manager abstraction so obtain/renew isn't limited to ACME
* use go-acme/lego
* Use master branch of go-lego/acme since v2.3.0 still has a dependency on xenolf/lego
* Use golangci-lint since gometalinter is depricated
* different way of installing golangci-lint for appveyor
* Removing golangci-lint from Appveyor because of https://github.com/client9/shlib/issues/13
Also adjust clients so that they use the configured HTTPPort or
HTTPSPort for solving challenges, if different from the default
challenge port (not as preferred as the Alt*Port values, of course)
