TLS certs for internal OTS hardware
Wed 26th June 19
Quite a while ago now, a client came to me to test a hardware box that plugged into home networks and provided a service. The device was managed, as this type of device usually is, through a web interface over HTTP. The test was not long before the Chrome update which started to highlight when you were accessing or submitting content over HTTP and so I discussed the upcoming changes with the client and asked if they had any plans for when it came in.
They said they had considered using self-signed certificates but they end up looking just as bad and forced you to explain why a user should ignore a security warning to accept the certificate. As the hardware in question was designed to add extra security to the network, this would not have been a good look.
I got thinking about this and ended up having a discussion with legendary TLS expert Scott Helme. From this we came up with the following workflow which will allow an internal box to have a valid TLS certificate.
The Actors
The cast for our performance:
- OTS - The off the shelf box which needs the certificate
- HQ - The manufacturers
- LE - Let's Encrypt
The Setup
For this to work, HQ needs to have a domain which can be similar to their main one but is purely for the OTS boxes, for example ots.space could have ots-cert.space.
The Workflow
- OTS arrives in its new home and is powered on for the first time. It generates a long random token and sends it, along with its IP address, back to HQ over HTTPS.
- HQ generates a unique name for the device, this could be a short (8'ish) character string or something dictionary based, but will need to be able to be used by humans (e.g. nifty-babbage).
- HQ creates a DNS entry for the device name on the devices domain using the internal IP submitted by OTS (e.g. nifty-babbage.ots-cert.space A 192.168.0.7).
- HQ sends the name back to OTS.
- OTS generates a private key and CSR based on the name it has been given and sends it to HQ.
- HQ uses the CSR and LE's DNS based authentication to generate a certificate for the subdomain.
- HQ creates a package containing the certificate and an authentication token.
- While steps 6 and 7 are going on, OTS is polling HQ over HTTPS to check whether its package is ready, it uses the random token generated in step 1 as its identifier. Once the package is ready, it downloads it, installs the certificate and sets up a redirect on its web server so when it is accessed over HTTP it will redirect the user to the new device name on HTTPS (e.g. http://192.168.0.7 issues a 301 redirect to https://nifty-babbage.ots-cert.space). The DNS will resolve to the internal IP and the user will not get any warnings about invalid certificates as the certificate was issued by LE, a trusted CA.
- A daily job is setup on OTS to check how long until the certificate expires and when it is within a certain number of days (7 seems a good choice) of expiry, it calls home to HQ, using its authentication token, and requests a new certificate. The same process should also be ran every time the box is restarted.
You can see a demo of this in action here; Server is running at HQ and client is running on the OTS.
A few things to note
All the certificates generated by LE are submitted to certificate transparency logs, this means that all the device names can be enumerated by reading the logs. This could be seen as information disclosure but as there is no way to tie the device name to an individual user, unless you track them buying the box in the shop and know when they turned it on, this should not be an issue. Having internal IP addresses in public DNS entries is unlikely to cause a problem for the same reason.
Vanity device names could be offered to users but this could increase the risk posed by the certificate transparency and DNS entries.
LE certificates are valid for 90 days, if the device is left off for 91 days, on boot, the certificate will have expired. This is not a problem as the on boot call home will request a new certificate and it will likely be installed by the time the user visits the web page.
On a network where OTS gets its IP through DHCP and the IP changes frequently, a call can be made out to HQ to update the DNS entry on each change. A short TTL will limit the problems caused by this. If the device is a router and can intercept DNS requests, it would be able to return its own IP without having to make the call out to the external DNS server.
To prevent HQ from a distributed denial of service (DDOS) by all the different OTS boxes calling home at once, the daily job should be set to a random time.
If the manufacturing process allows it, adding a client certificate to the devices as they are produced would allow that to act in place of the random token and authentication token. A problem with doing this is that the certificate would have to have a very long lifespan to allow for the box sitting on the shelf. This could be mitigated by issuing a new client certificate along with its server certificate but this adds complexity without adding any significant security improvements.
As the private key is generated on the OTS, it is never sent across the network. HQ sees, and could potentially cache, the certificate but that is useless without the private key.
Proof of Concept
Rather than just talk about it, I've created a fully working proof of concept for you to play with. It needs a little setting up, but not much, and should be fairly simple for most people to get running.
