Emerald Onion

Fundraiser

Emerald Onion needs your help! We are a U.S. tax-deductible 501(c)(3), and we are fundraising for new server hardware that supports a modern security feature called AMD SEV-SNP, which we discuss in Phase 2 of this post below.

Please visit emeraldonion.org/donate to give before 31 December 2025. Email donations@emeraldonion.org with any questions!

Phase 1: Building our Diskless Relay Architecture

Throughout 2025, we've been working on a project to strip out unnecessary components from our relay architecture and move towards a philosophy of simplicity. We brought our minimalist relays online in June 2025.

Our new relays use a custom JeOS image based on Alpine Linux. A single relay packs down into a bootable Unified Kernel Image weighing in at just 66 MB. This includes everything the relay needs to function; a kernel, the Tor process, and very little else:

We build an image for each relay, and deploy it using Terraform to our Proxmox-based hypervisors in Hurricane Electric's Fremont 2 datacenter in California. The relays boot entirely into RAM, with no persistent storage attached to the VMs.

To update the relays, we build a new set of images and push them out to the Proxmox layer. This gets us very close to the ideal container-like workflow, but with full VMs.

The small footprint of these relay images allows us to minimise the remote attack surface of our relays, and reduce the risk of a threat actor compromising relay keys.

As the main function of Tor relays is relaying traffic from Point A to Point B, they lend themselves quite well to this diskless image-based approach. We can get the relays quite close to being “stateless”, but since Tor relays maintain a persistent cryptographic identity, we do need to preserve some state.

A Tor relay can be broken down into only a few files that need to persist across image deployments. These are the only files we need to preserve in order for a relay to maintain its long-term identity:

File	Purpose	Persistence Mechanism
/etc/tor/torrc	Relay config	Statically generated and injected into image at build time.
/var/lib/tor/keys/ed25519_master_id_public_key	Long-term identity public key	Generated by Tor's OfflineMasterKey process and injected into image at build time. The private key for this never leaves the generation server.
/var/lib/tor/keys/ed25519_signing_cert	Short-term identity public key	Generated by Tor's OfflineMasterKey process and injected into image at build time.
/var/lib/tor/keys/ed25519_signing_secret_key	Short-term identity private key	Generated by Tor's OfflineMasterKey process and injected into image at build time.
/var/lib/tor/keys/secret_id_key	Long-term identity private key (legacy)	Generated by Tor's OfflineMasterKey process and injected into image at build time.
/var/lib/tor/keys/secret_onion_key	Ephemeral circuit handshake key (legacy)	Collected from running relays via SSH before new images are built. Must be persisted in new images otherwise all Tor circuit traffic stops on redeployment for 2-3 hours while a new ephemeral key propagates through the network.
/var/lib/tor/keys/secret_onion_key_ntor	Ephemeral circuit handshake key	Collected from running relays via SSH before new images are built.
/var/lib/tor/state	Self-measured bandwidth metrics for the relay	Collected from running relays via SSH before new images are built. Not strictly required but used by Tor Metrics (cosmetic only).

The general steps our build process follows are:

1. Our local Ansible script builds a base image for each relay including the above state files and a minimal packageset, using alpine-make-rootfs.
2. The images are packed into a Unified Kernel Image using the ukify tool.
3. Images are scanned using Syft to produce a Software Bill of Materials (SBOM) in both Syft and CycloneDX formats, which allows us to keep a history of the contents of all our deployed images.
4. The images are copied to our Proxmox servers using Terraform, and the existing relays are replaced in-place with the new version of the image.

So far, building relays and persisting just the above state has been working very well for our threat model:

1. If relays are seized and powered off, there is no data to retrieve, which ensures protection for the users' traffic flowing through our exits.
2. Including only the bare minimum number of packages we need reduces the chance of a vulnerability impacting our nodes.
3. Building and pushing out container-style images helps us ensure the relay configs are always exactly as they should be, cutting out config drift or consistency issues.

We're currently working on rebuilding our relay identity management system so that relay keys are fetched from an off-site central control plane after boot, and held only in RAM. This will ensure that in the case of relay seizure there is nothing at all held on-disk that might compromise either our users or our relay infrastructure.

As part of our commitment to the Tor relay operator community, today we have open-sourced the relay orchestration framework we use to manage our live exit relays. Please consider the codebase a continual work in progress. We also commit to doing all future development in the open here.

You can find the code at: https://code.disobey.net/EmeraldOnion/emerald-relays

Physical Threats

We're quite proud of this year's progress on our minimal relay deployment. We think the work we've done so far does a good job at addressing a number of the threats faced by Tor exit operators.

But we must accept the reality of being a Tor exit operator in the United States in 2025. More than ever before, it is critically important that we consider the risk posed by our servers being outside of our physical control.

Emerald Onion owns and colocates all our own server equipment, rather than leasing VPS space or equipment owned by third-parties. But this equipment is still located within a datacenter outside of our physical control.

So we need to build a system that validates that the images built in our trusted infrastructure are the same ones which are relaying Tor users' traffic in production, and that they have not been modified by a compromised physical host.

Tor Exit Operators: A High-Level Threat Model

When we plot our current threat model, the impact of this year's work becomes clear. A lot of the threats that a Tor exit operator typically faces are now mitigated in our setup:

Threat Category	Threat	Mitigation	EO Status
Confidentiality	Malicious ISP	EO do not entrust our relay traffic to a single ISP. We own our own IPv4 and IPv6 blocks as AS396507, and peer with other networks directly	Mitigated
Confidentiality	Malicious hosting provider	EO own all of our own hardware, and install it ourselves into our colocated datacenter presence	Mitigated
Integrity	Software vulnerabilities	Minimal JeOS deployment with SBOM that can be continually evaluated so we know when patching is required	Mitigated
Availability	OS-level tech debt (Major version upgrades etc)	Continual image rebuild in CI	Mitigated
Availability	Configuration drift	Immutable image-based deployment	Mitigated
Integrity	Software supply chain attacks	JeOS deployment includes only the packages required for functionality	Partially Mitigated
Confidentiality	Insider threat	Public image build infrastructure, with builds verifiable by all EO volunteers (SEV-SNP)	WIP (Phase 2)
Confidentiality	Relay impersonation following physical seizure	Ensure that relay keys are fetched into RAM and never exist on disk (protected by SEV-SNP)	WIP (Phase 2)
Integrity	Untrusted hardware (out of our physical control)	Image validation via SEV-SNP	WIP (Phase 2)
Integrity	Untrusted hypervisor OS (out of our physical control)	Image validation via SEV-SNP	WIP (Phase 2)
Integrity	Advanced physical attacks against SEV-SNP (RAM interposing, TEE.Fail etc)	Classic physical controls (rack cameras, locks, chassis intrusion events)	WIP (Phase 2)
Confidentiality	Global Passive Adversary / Netflow capture	Much greater global and geopolitical distribution of exit capacity across the whole network. See our recent post for more of our thoughts on this	Future

It's clear from the threat model above that there are a number of potential threats faced by Tor exit operators such as Emerald Onion that could be mitigated through the use of SEV-SNP, which we'll explore in the rest of this post.

Phase 2: Enhancing Trust in our Relays

Since we're using an immutable image-based deployment, we're able to make use of some of the technologies from the world of Confidential Computing (CoCo). In particular, we're looking to make full use of SEV-SNP, which is AMD's implementation of a Trusted Execution Environment (TEE).

SEV-SNP (Secure Encrypted Virtualization – Secure Nested Paging) is a technology that allows isolating guest VMs from an untrusted host OS. It's already used for CoCo in environments such as AWS and Google Cloud, but we're looking to continue the approach of owning and colocating all our own hardware by investing in new servers that support this feature. SEV-SNP is available on AMD EPYC from 7003 (Milan) onwards. Our current relays run on EPYC 7002 (Rome) chips.

Typically, SEV-SNP is built to protect workloads against an untrusted cloud provider, as seen in this presentation by AMD's Jeremy Powell in 2022:

In our case though, we are considering ourselves the untrusted cloud provider as, although we're providing our own hardware, the fact it's outside of our physical control means we can't (and shouldn't) trust it.

SEV-SNP tackles this by creating an environment that requires we only trust the integrity of the underlying CPU, which makes use of an immutable bootrom baked into the silicon at the factory:

With SEV-SNP, we no longer need to trust the hypervisor, host OS, or the rest of the hardware. We will of course continue to uphold our usual high standards for the security of these components, but re-architecting our Tor exits around Confidential Compute concepts will cut out the majority of the remaining threats we identified in the threat model above.

It's important to acknowledge that attacks against Trusted Execution Environments do exist, such as TEE.fail. But these attacks require sophisticated attackers with physical access, who need to make themselves very visible in the process. We are aware that the solution being proposed here isn't perfect, and weaknesses will always exist. But we believe that this is a worthwhile step to take, as it substantially raises the bar for an attacker. We are also looking into how we can tie more traditional physical security controls such as rack cameras and chassis intrusion logs into this approach.

When we implement our SEV-SNP relays, we will make public our entire build chain — ensuring that relays are built in the open with publicly-available build artifacts. These artifacts can be used by EO to ensure continual attestation of the running relays against the expected values produced by the build system. Longer-term, we aim to investigate the possibility of building an architecture that allows public validation of our relays by anyone, though that is a more challenging piece of work. We welcome collaboration and information sharing with others who have worked in the Confidential Computing space.

Fundraiser Details

Please remember to visit emeraldonion.org/donate to donate before 31 December 2025 so Emerald Onion can purchase at least three used HPE ProLiant DL325 Gen10 “V2” servers with AMD Epyc 7xx3 CPUs in the new year. That's our goal! Your donation will allow Emerald Onion to perform the necessary security research and deployment of Phase 2 (above) in our Fremont, California datacenter.

We are also committing to publishing our detailed findings and all our deployment tooling, to help Tor relay operators and other privacy infrastructure operators learn from our work. Thank you so much for considering to donate to help make the Tor network a better, safer place.