Cloud-init Is Not the Only Way

We recently got into it with another engineer who was certain the only way to get keys and configuration into a machine at first boot is a cloud-init script. On a cloud VM, he is right. But we were not building a cloud VM. We were building a fleet of bootable USB sticks that boot bare-metal laptops, with no cloud datasource anywhere in the picture. Deciding that the one standard tool is the only tool is exactly how you talk yourself out of the better answer. Here is the one we built, and how.

The real job

The job was never just inventory. We had a large lot of reclaimed business laptops to move, and the goal was to automate the entire lifecycle end to end: from the moment a technician first touches a unit, through hardware capture, secure wipe and reimage, listing generation, posting to eBay and our storefront, and tracking the sale through to close. Done by hand that is hundreds of hours of work. The target was to collapse it to roughly a week, with people touching hardware and everything in between running itself. The bootable inventory stick is the front end of that pipeline, the tech-touch step that feeds everything downstream, and it is where the interesting engineering lived.

The front end: a bootable inventory USB

Booting each laptop into its own operating system to read it is slow, and it is a security gamble, because you are trusting the state of hardware you have not looked at yet. So the front end is a bootable USB. Boot from it and a lightweight Linux live image reads the hardware hands-off, processor, memory, disk and its health, battery, model, and serial, without ever starting the machine’s own operating system. It lands the result on our storage and hands off to the next stage. It is read-only to the unit you are inventorying.

Why Ventoy, and the trade-off that made it interesting

We could have shipped a plain, self-contained bootable image, written straight to each USB, and been done. Persistence would have been trivial or unnecessary. We chose Ventoy on purpose.

Ventoy turns a USB drive into a multi-boot drive you manage by copying files. Drop an ISO on the drive and it shows up in a boot menu, no reflashing. That makes the drive endlessly versatile: put as many ISOs on it as you want, inventory, wipe, rescue, an OS installer, and pick at boot, or ship a single-purpose stick that carries only the inventory image. One drive can be a whole workbench or a purpose-built tool you hand a technician. We theme the boot menu with our own branding, so a tech boots into a company picker rather than stock Ventoy.

The clearest payoff is what happens after inventory: we wipe and reimage each laptop by booting a Windows 11 installer from the same drive. Windows 11 needs two builds for a mixed fleet, a standard one and a NoTPM one for machines that cannot pass the TPM and Secure Boot check, so with plain single-purpose sticks you carry the inventory stick plus a TPM Windows stick plus a NoTPM Windows stick. A bench that needed 30 inventory sticks suddenly needs 60 to 90, tripling a line item that was already expensive. On Ventoy all of those ISOs live on one drive behind the boot menu: inventory the unit, then wipe it with the right Windows build, from the same stick. That is where the versatility pays for itself, and it turns a pile of single-use sticks into reusable tooling.

That choice is exactly what created the hardest engineering in the project. Ventoy boots by loop-mounting the ISO and presenting that as the boot device, which breaks two things a normal live image takes for granted: finding its own autorun, and keeping persistence. The work below is not inherent to inventorying laptops. It is the price we chose to pay for Ventoy’s versatility, and it was worth it.

Keys and variables, without cloud-init

Here is the part that started the argument. There is no cloud datasource on a bootable USB, so there is no cloud-init. Instead, a first-boot routine baked into the image does the job a cloud-init script would: it fetches the public keys it needs from a source we control, over HTTPS, and it generates its own secrets on the device. Nothing sensitive is shipped inside the image at all.

That is not a workaround for a missing cloud-init. It is arguably the better pattern, and the clearest way to see why is to reverse the direction. cloud-init pushes secrets down into an instance from a datasource. We push a public key up from the device and keep every secret on the device. Reverse the arrow and the whole “how do we protect the injected secret” problem shrinks, because there is no injected secret to protect.

Safe to lose

USB sticks that you clone a dozen times and hand to technicians get lost. So we built the whole thing assuming one would, and made that a non-event. Being precise, because it is easy to overclaim: a stick that has been in service is not empty. Its persistence holds that one stick’s own identity, including its remote-access key. What makes a lost stick safe is not that it forgets everything. It is that what it remembers is useless to whoever finds it.

  • The image we clone from ships clean. No credentials, no token, no identity.
  • Every stick generates its own identity on first boot, unique per stick, so one lost stick tells you nothing about the others.
  • The remote path is off by default and only works if that exact stick was authorized on our side, and anything authorized is revoked centrally in seconds.
  • It is read-only to the machine you are inventorying and copies nothing off it, so no laptop’s data ever ends up on the stick.

A lost stick costs us a stick, not the network.

And it does not care about the network

First boot needs a brief moment of connectivity to fetch its keys and enroll. After that, capturing a unit does not depend on the network at all. If our storage is unreachable when a unit is inventoried, the tool writes the same files locally and holds them for automatic retransmit once it is back online. A dead or flaky network at the bench, which is common at a field site running on cellular or satellite, never costs you a capture.

How we did it

For the engineers, here is the build, layer by layer, with the identifying details left out on purpose.

Boot and persistence

Ventoy loop-mounts the ISO and presents that as the boot device. A live Linux OS looks for its own autorun and config on the device it thinks it booted from, which under Ventoy is the read-only ISO, not the Ventoy data partition. That is the well-known Ventoy autorun behavior (issue #2304), and it is why a config file dropped next to the ISO is never found. The fix was to bake the autorun into the ISO itself with xorriso, replaying the El Torito boot image so the ISO stays bootable. Baked in, it runs no matter how the stick boots, and it cannot be tampered with through the stick’s storage.

Persistence under Ventoy is not the live OS’s usual overlay partition. You create a persistence backing file, a casper-rw ext image, on the drive and wire it to a specific ISO through a persistence entry in ventoy.json. Persistence is therefore per-image and file-based. Without that file wired, every first-boot change, the generated identity, hostname, keypair, and enrollment, lives in a RAM-only overlay and evaporates at the next power cycle. Wire the per-image file and that identity survives reboots. That single file is exactly what turns “generates a unique identity” into “keeps its own unique identity,” and getting it right was most of the work.

Because persistence is one file per image, rotating or wiping a stick’s identity is surgical. To make a clean golden clone you clear or drop that one casper-rw file; you do not reflash the drive. And adding the image to an existing Ventoy drive is non-destructive: it is a file copy plus one entry merged into ventoy.json, after a snapshot, and it never reformats the operator’s drive.

First-boot self-configuration

On first boot a run-once routine, a systemd oneshot gated by a stamp file so it fires exactly once per clone, does what a cloud-init user-data script would do elsewhere. It generates a fresh WireGuard keypair on the device, fetches the public authorized-keys it needs over HTTPS from a source we control, enrolls a lightweight management agent, derives a stable per-stick hostname, and writes a self-verification report so you can confirm every hook fired. No private key is ever baked into the image; the only material baked in is public.

For the hostname, each stick names itself on first boot. Today that comes from a per-stick identifier written at clone time. The cleaner approach, and the one we are moving toward, is to derive it from a short hash of the WireGuard public key the stick just generated, so every clone self-names without collisions and the machine’s name is tied to the identity it actually presents.

The enrollment and peer-registration trust boundary

This is the part worth reading closely, because it is where the cloud-init argument partly survives, and it is the first thing a good reviewer asks about.

The keypair is minted on the stick and the private half never leaves it. What travels is the public key, upward. During enrollment the stick reports its identifier, its public key, and a one-time token over HTTPS. A small authenticated handler validates the token and identity, then either adds the peer to our edge router, which is also the WireGuard hub, over the router’s API, or drops it into a pending queue for a person to approve. That handler is deterministic. There is no model or inference anywhere in this path; it is a few lines of glue and one API call per stick. The stick never holds router credentials and cannot add itself.

Because the router is the hub, the same device that admits a peer also enforces its allowed-IPs and firewall scope. Admission and containment happen in one place under our control, and pulling the peer kills it at that same box.

Where does the one-time token come from? Not the golden image, which ships tokenless. At clone time, on a controlled bench, provisioning pulls a unique one-time token from the server into that one stick’s persistence, and the server burns it on first enroll. So the trust anchor is a trusted operator on our bench at clone time, never the field, and never the cloned artifact. You cannot make bootstrap trust out of nothing; the design just keeps the anchor off the stick that gets lost. cloud-init anchors that trust in the cloud platform’s injection. We anchor it at the bench, and the control plane only ever handles public keys and a low-value, single-use token, never a real secret. That makes it cheaper and lower-risk to run than a datasource that has to hold the actual secrets, not more expensive.

Remote assist in the field

We added the management agent and a pre-built WiFi profile together for a specific reason: a stick in the field comes online on its own and we can remote in to help a technician who is stuck, without walking anyone through network setup. The pre-built SSID is what gets the unit onto a network with zero configuration; the agent is what lets us reach it.

The WiFi profile is deliberately an untrusted network, not a sensitive one. It can be a one-time SSID, a reusable network that exists only for this process, or a guest WiFi we simply know the password to. Its only job is to get the unit online so it can phone home. Everything that matters after that, the enrollment, the key pull, the peer registration, is authenticated over HTTPS on top, so the tool treats whatever network it lands on as hostile by default. That is the point: the security never depended on the network being trusted. A found stick’s WiFi password is worth about as much as a coffee shop’s. It gets you onto an isolated network that reaches nothing, which is why the baked-in profile is not a credential worth protecting.

Hardware capture, hands-off and read-only

The collector mounts the target’s partitions read-only and never writes to the machine. It reads model, serial, and service tag from the firmware tables, disk model and SMART health from the drives, battery and TPM state, and the primary MAC. It reads the installed OS off the unit’s own partitions read-only, and recovers the firmware Windows OEM key from the ACPI MSDM table. The unit is never booted into its own OS and, in resale mode, never altered.

Resilient landing

The collector writes a per-unit JSON, a human summary, and a live log, and lands them on our storage. The land step is idempotent by serial and queue-backed: if storage is unreachable, the files are held locally and retransmitted when connectivity returns, so a bad network never loses a capture. A big on-screen serial and a clear landed-or-not indicator tell the technician instantly whether to move on or re-run. Right after a capture lands, the screen pops a QR code the technician scans with a phone to shoot the listing photos and submit them straight to the server, where they attach to that unit’s record by serial.

One codebase, two modes

The same collector runs in two modes, selected by a baked mode flag, so we maintain one tool instead of two. Resale mode is capture-only and never alters the unit. Recovery mode does more: it can pull user data off the drive, including BitLocker-protected volumes, before the disk is wiped.

The rest of the pipeline

Downstream, the captured data is matched and de-duplicated by serial, listing sheets are generated per unit, and units flow out to eBay and our storefront with the sale tracked back to close. The listing photos the technician submitted at the bench converge into the same per-unit record. People touch hardware; the software does the rest. That is how a job that would have taken hundreds of hours came down to about a week.

What we are testing next

The recovery side is where we are pushing next. We have alpha builds in testing that take on the steps a technician still does by hand: pulling user data off a drive and handling encrypted volumes before a wipe, through each machine’s own authorized recovery path, so a full recover-and-reset runs closer to hands-off. Anything that touches recovery and user data has to earn trust before it ships, so those builds stay in alpha on our own bench until they do. We will write that one up when it is ready.

The takeaway

cloud-init is a good tool. It is not the only tool. The moment you treat the standard pattern as the only pattern, you stop looking for the one that fits your actual constraints. On a bootable USB fleet with no cloud datasource, a first-boot routine baked into the ISO, one that pulls its public keys from a source we control and mints its own secrets on the device, does the same job, brings keys and variables in at boot, without cloud-init, and with better properties if a stick is ever lost.

Threat-model your own tooling the way you would a server. Bake the security in from the first line. Be willing to build the thing the textbook did not hand you, and be honest about the trade-offs you make. Thinking a little outside the box here produced a tool that is safer to lose, cheaper to reproduce, works offline, and never needed a cloud datasource at all.

This is the same thinking we bring to the systems and networks we run for businesses across Central Texas. If you want a second set of eyes on how your own tools and access are built, ask us. No pressure, no sales pitch. bytemenetworks.com/contact-us

Posted in
Scroll to Top