Migrating 400G to 800G: Field Notes From Real Rollouts
Every 400G-to-800G migration we have sat through broke in week one, and it was never the optics. It was the assumptions that came with them.
The slide deck for a 400G-to-800G migration is always the same and always correct: cages support it, modules are available, budgets are approved. Then week one happens. Across the rollouts we have either run or been called into, the same handful of issues keep showing up, none of them exotic, all of them discoverable in advance. This is the version of the story with the names changed and the invoice totals removed.
Week one: the cage that wasn't
The most reliable first-week surprise is the phrase "800G-capable" on a switch datasheet. It can mean an OSFP cage, a QSFP-DD800 cage, or a QSFP-DD cage that accepts 800G modules only from a vendor-qualified list, with per-port power derating once the chassis is fully populated. A procurement that stopped reading at "800G" discovers in week one that half the ordered modules are the wrong form factor, or that the right ones are capped at a lower power class than their label suggests.
The check that prevents this takes one email: ask the switch vendor for the per-port maximum module power at 45 °C inlet with all ports populated, and the qualified module list that goes with it. If the answer hedges on "all ports populated", treat the hedged number as real - it usually reflects a measured airflow limit, not an oversight. The 800G selection guide turns this into an ordering checklist.
Week two: MPO polarity, again
If the migration includes new DR8 trunks, expect the polarity conversation to come back from the dead. Plants that have run duplex LC for a decade have institutional knowledge about duplex: clean it, plug it, it works. DR8 lives on 8- and 16-fiber MPO trunks where the fiber mapping - Type A, Type B, cassette gender - decides whether a path exists at all.
The failure signature is distinctive and worth memorizing: the link refuses to come up, DDM shows either no RX power or RX power on the wrong lanes, and every component tests fine in isolation. We watched a crew install a fresh batch of trunks from a new supplier; the cassettes were a different polarity type than the previous batch, nothing was labeled differently, and four hours vanished. The MPO polarity explainer exists precisely because this keeps happening. The practical mitigation is boring and effective: one trunk SKU, one cassette type, labeled at both ends, with the polarity convention printed on the install plan - not in an email thread.
The breakout wrinkle
Somewhere around week two, the team discovers that a DR8 module and a 2x400G-DR4 breakout present themselves differently to the switch: one port or two. The optics look interchangeable in the BOM. The automation disagrees. Confirm what the network management system expects before the cabling crew starts, not after.
Week three: power and spin-up
Fully populated, an 800G chassis carries around a kilowatt of optics on top of the ASIC. Three things have bitten real deployments at this stage:
| Issue | Symptom | Prevention |
|---|---|---|
| PSU headroom miscalculated | Alarms at full population; forced module derating | Budget optics power per rack, not per chassis |
| Inrush at simultaneous module enable | Power events during mass re-enable after maintenance | Stagger port enables where the platform allows |
| Afternoon thermal peak | Temperature flags, occasional link drops | Re-run telemetry sweep at peak, not at commissioning |
None of these show up on a lab bench at half population and a 25 °C room. The full-population, real-ambient test is the migration's actual acceptance test; everything before it is rehearsal.
What DDM taught us
The single highest-leverage habit from these rollouts costs nothing: take a per-port DDM snapshot at commissioning, and again after two weeks of production. Compare, don't inspect. A TX bias that moved from 41 to 47 mA at the same case temperature is a finding; the same two numbers on different ports mean nothing.
The comparison habit is what makes the later diagnostics tractable. When something eventually degrades - and at this port count, something will - the baseline tells you whether it moved, and the not-linking diagnostic order tells you what to check next. Migrations without baselines diagnose everything by swap-and-pray, which is slower and teaches nothing.
One vendor per rail
Keep one module vendor per rail. Mixed vendors inside a rail produce two sets of alarm thresholds and two bias behaviors, and every anomaly investigation starts with "which vendor is this port again?". The price difference that motivated the mix is never larger than the engineering time it costs.
Spares are part numbers, not speeds
A spare that is "the same speed but a newer SKU" has a different firmware load and a different register behavior. It will work on the bench and surprise you in production. Spares should be the same part number as the production modules they back up, and they should spend their shelf life in the same qualification record.
Five things we would do differently
- Pin the cage and power class in the purchase order, not in a follow-up email. The words "800G-capable" have cost more than any single module.
- Standardize one MPO trunk SKU and label polarity on the plan. New supplier + unlabeled convention = one lost afternoon minimum.
- Test at full population and real ambient before sign-off. The gap between commissioning conditions and production conditions is where all three weeks' problems hide.
- Baseline DDM on day one and diff it at two weeks. It is the cheapest telemetry program in networking and pays for itself at the first degraded lane.
- Phase by rail, not by rack. Completing one rail end-to-end surfaces integration issues (breakout support, monitoring, spares) while the blast radius is one rail instead of the fabric.
The quiet conclusion
None of these problems is about 800G being fragile. The optics themselves have been the most reliable part of every rollout we have seen. What the migration actually tests is whether the organization's assumptions - about its own cabling plant, its power budget, its thermal envelope - match the machine room they describe. The modules are fine. The checklists are what's missing, and unlike the modules, checklists are free.
Sources
- IEEE 802.3df Task Force - the 800G PMD amendments (DR8, SR8 and the 2xFR4-class options) referenced in the migration paths above.
- QSFP-DD MSA - cage, power class and CMIS management specifications behind the per-port limits discussed.
- OSFP MSA - mechanical and thermal envelope for OSFP deployments.
- Ethernet Alliance - multi-vendor interoperability testing, a useful external sanity check on module/cage combinations.
This article reflects accumulated rollout experience, anonymized and generalized. Site specifics - plant, ambient, platform qualification - always override general guidance.
FAQ
- Can we migrate to 800G without recabling?
- Often yes, with the 2xFR4-class modules that run 800G over existing duplex LC pairs to about 2 km. But check the optical budget first: CWDM lane grouping adds loss inside the module, and a link that was comfortable at 400G FR4 can be marginal at 800G. Measure, do not assume.
- What is the first thing to verify on an '800G-capable' switch?
- The exact cage type and the vendor's qualified module list at your real inlet temperature. '800G-capable' covers at least three mechanically different cages, and the per-port power limit can depend on module population and airflow in ways the headline datasheet does not show.
- Why do MPO problems reappear during an 800G migration?
- Because DR8 shifts traffic onto 8- and 16-fiber trunks that many plants have never run before. A new batch of trunks or cassettes is a new chance to introduce a polarity mismatch, and the failure is silent: the link simply does not come up.
- Is mixing 400G and 800G ports on the same switch acceptable?
- Yes, and it is usually the bridge strategy: 800G-capable cages running 400G modules while the rest of the fabric catches up. Just keep one vendor per rail so DDM baselines and alarm behavior stay comparable.
- How long should a migration soak before declaring success?
- At least one full weekly load cycle including the thermal peak. Several of the failures we describe here appear only when the chassis is fully populated, at afternoon inlet temperatures, with production traffic - none of which exist on commissioning day.