Emulating Oxide: How Faux Can You Go?

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording. How do you test configurations you haven't built yet? Steve, Andrew, and Rain join Bryan and Adam to talk about the transformative power of the Oxide emulation environment, VOXEL, and how it has accelerated development in many domains. In addition to Bryan Cantrill and Adam Leventhal, speakers included Steve Karam, Andrew Stone, and Rain Paharia. Previously, on Oxide and Friends: - OxF s03e06 - Rack-scale Networking (P4 and DDM with Ry Goodfellow) - OxF s01e26 - The Pragmatism of Hubris (with Cliff Biffle) - OxF s03e09 - Get You a State Machine for Great Good (Wicket with Andrew Stone) - OxF s01e25 - Tales from the Bringup Lab - OxF s01e18 - Dijkstra's Tweetstorm - OxF s03e31 - Hiring Processes with Gergely Orosz Some of the topics we hit on, in the order that we hit them: - **voxel** -- Virtual OXide Emulation Lab: emulated rack deployments on one Helios host (omicron on Falcon/propolis VMs, SoftNPU switches, FRR routers); optional real SP/RoT firmware via sp-emu. https://github.com/oxidecomputer/voxel - **Falcon** -- tool for standing up virtual network/rack topologies of propolis VMs on illumos for testing. https://github.com/oxidecomputer/falcon - **Tofino** -- Intel/Barefoot programmable switch ASIC (Tofino 2) in Sidecar; P4-programmable. Discontinued by Intel. - **SoftNPU** -- software P4 target emulating the switch dataplane, so the stack runs without Tofino hardware. https://github.com/oxidecomputer/softnpu - **P4** -- DSL for describing packet-processing pipelines; how the Tofino/SoftNPU dataplane is programmed. https://p4.org - **Gimlet** -- Oxide's compute sled. - **Sidecar** -- Oxide's rack switch board, carrying the Tofino 2. - **raclette** -- reduced-size lab rack (a few sleds + switch) used for dev/test rather than a full 32-sled rack. *(my read; unverified)* - **Hubris** -- Rust microkernel/RTOS for the SP and RoT: statically-defined tasks, memory protection, synchronous IPC. https://github.com/oxidecomputer/hubris - **Humility** -- debugger for Hubris systems; dumps tasks, stacks, ringbufs over SWD. https://github.com/oxidecomputer/humility - **a4x2** -- pre-voxel testbed topology: 4 emulated Gimlets, 2 SoftNPU switches, omicron on one Helios host. - **trust quorum** -- rack secret split into shares across sleds; a quorum must be present to reconstitute the storage-encryption key, so a stolen sled is useless alone. https://docs.oxide.computer/guides/system/initial-rack-setup - **illumos** -- open-source Solaris-derived OS; the host OS (Helios) is an illumos distro. https://illumos.org - **zones** -- illumos OS-level virtualization; each control-plane service runs in its own zone. - **Scrimlet** -- Gimlet attached to a Sidecar ("switch Gimlet"); hosts the switch zone, dpd, MGS. - **DDM** -- Delay-Driven Multipath, the underlay routing protocol, implemented in Maghemite (ddmd, ddmadm). https://github.com/oxidecomputer/maghemite - **Dendrite** -- dataplane controller; dpd programs Tofino or SoftNPU and exposes an API to the control plane. https://github.com/oxidecomputer/dendrite - **swadm** -- CLI against dpd for port config, links, addresses, routes. - **A0** -- sled power state where the host is fully powered (vs. A2, standby with the host off); driven by the SP sequencer task. - **iCE40** -- Lattice FPGA on Gimlet/Sidecar for power sequencing and board glue logic. https://github.com/oxidecomputer/quartz - PRs needed! If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!