Boot-time USB-C host mode was bench-proven on 2026-05-08: set the hw.dwc3.force_host tunable in loader.conf, the chip comes up as xhci, the Anker hub enumerates at 5.0 Gbps, and axge ue0 carries traffic. That essay isn’t written yet — but the recipe is in appendix/usb-c-pd-recipe. Useful, but a tunable means a reboot to change roles.
This essay is about doing the same thing at runtime. Write 1 to hw.dwc3.role_swap, and the running gadget tears down, the chip soft-resets, an xhci child attaches alongside, and usbus5 shows up — all without losing wifi, without rebooting, and without wedging the system.
That last clause was the entire fight.
The shape of the swap
snps_dwc3_sysctl_role_swap in src/sys/dev/usb/controller/dwc3/dwc3.c does three things in order:
1. dwc3_gadget_quiesce(dev) — stop the gadget without freeing it
2. snps_dwc3_reset(sc) — GCTL.CORESOFTRESET, wait for clear
snps_dwc3_configure_host(sc) — write PRTCAPDIR=HOST, configure mode
snps_dwc3_do_quirks(sc)
3. snps_dwc3_attach_xhci(dev) — alloc IRQ, add usbus child, xhci_init
dwc3_gadget_quiesce is the new piece. Detaching the entire gadget driver (freeing DMA tags, killing the usbus child, joining the kproc) is dozens of lines of fragile code in a driver that we’ve been explicitly told not to touch. Quiescing — drop D+, wait for DSTS.DEVCTRLHLT, mask GEVNTSIZ, zero DEVTEN, drain the TX watchdog, mark every endpoint stopped, mark the ifnet down — is a much smaller surface. The chip soft-reset will trash the chip-side DMA descriptors anyway; we just need the OS side to stop expecting them.
That worked. The bench evidence is at the top of this page. What follows is the four hours it took to get there.
The bring-up arc
[WAR STORY]
A wedge in three personalities
▸ symptom
Build the role_swap sysctl naively: have it call snps_dwc3_reset + snps_dwc3_attach_xhci with the gadget still running. Write 1 to hw.dwc3.role_swap. Phone wedges instantly with this on the framebuffer console:
WARNING list_empty(&lock->head) failed at .../drm_modeset_lock.c:268
WARNING drm_modeset_is_locked(&crtc->mutex) failed at .../atomic_helper.c:617
WARNING drm_modeset_is_locked(&dev->mode_config.connection_mutex)
failed at .../atomic_helper.c:667
WARNING drm_modeset_is_locked(&plane->mutex) failed at .../atomic_helper.c:892
*ERROR* [CRTC:33:crtc-0] hw_done timed out
*ERROR* [CRTC:33:crtc-0] flip_done timed out
*ERROR* [CONNECTOR:35:DSI-1] hw_done timed out
*ERROR* [CONNECTOR:35:DSI-1] flip_done timed out
*ERROR* [PLANE:31:plane-0] hw_done timed out
... etcWiFi dies, ssh dies, USB-net dies. eMMC pull, mount on the build host, copy kernel.prev back over kernel, reboot. Every iteration is 5–10 minutes of recovery.
▸ hypothesis 1
The shape of the wedge — hw_done timed out, drm_modeset_is_locked failures — is identical to a wedge documented from 2026-04-27 in the cross-driver audit. That older wedge was theorised to be DMA stomping a live driver during chip reset. Same theory here: snps_dwc3_reset does GCTL.CORESOFTRESET, which clobbers the chip-side gadget DMA descriptors. The gadget kproc is still running, the gadget IRQ handler is still installed, and there’s a TX watchdog callout — all looking at descriptors that just got rugged.
Write 5db9a9c dwc3: park gadget before chip soft-reset on role swap : dwc3_gadget_quiesce(device_t) — drops D+ via DCTL.RUN_STOP=0, waits for DSTS.DEVCTRLHLT, masks GEVNTSIZ.INTMASK, zeros DEVTEN, drains the TX watchdog, walks sc->eps[] setting started=0, brings the ifnet down. DMA tags and the usbus child stay attached — the cheap version of detach.
Update role_swap to call dwc3_gadget_quiesce before snps_dwc3_reset. Build, deploy, fire.
Same wedge. dwc3_gadget_quiesce: parked makes it to the screen, then the same DRM cascade hits. The DMA-stomping theory was wrong, or at least insufficient.
▸ hypothesis 2
If quiesce is doing its job, then the wedge happens between quiesce returning and the next visible printf. That gap is snps_dwc3_reset (∼5 ms of DELAYs) + snps_dwc3_configure_host + snps_dwc3_do_quirks + snps_dwc3_attach_xhci. None of those should take 10 seconds. But hw_done timed out is a 10 second timeout. So somebody is blocking the vblank IRQ for ten seconds, or something inside attach_xhci is hanging long enough that an in-flight DRM atomic commit times out waiting for a flip.
A photo from the bench shows a cur_vblank != vblank->last failed at .../drm_vblank.c:348 line right above the role_swap output — vblank counters disagreeing. That fires when the kernel reads the hardware vblank counter and finds it didn’t advance the way the software state expected. Vblank IRQs are being lost. Combined with a separate patch we’d landed earlier ( fb81d0f rk_vop: mask vblank interrupts when idle — rk_vop: mask vblank interrupts when idle), the theory becomes: sway is mid-flip when role_swap fires, the role_swap thread blocks the vblank handler somehow, the flip times out 10 s later, the helper logs the cascade.
Test: SIGSTOP the entire sway tree before triggering role_swap. No compositor → no in-flight commits → no hw_done timeout.
Same wedge. Compositor ruled out. The DRM warnings are a symptom, not the cause.
▸ hypothesis 3
If it’s not DMA and not the compositor, then it’s something inside the kernel role_swap path itself, blocking long enough that vt(4)‘s own console writes (every WARN, every ERROR) eventually trip the framebuffer atomic-commit path because the console output queue is what’s actually wedged.
Add device_printfs around every step:
device_printf(sc->dev, "role_swap: step 1/5 snps_dwc3_reset\n");
snps_dwc3_reset(sc);
device_printf(sc->dev, "role_swap: step 2/5 configure_host\n");
snps_dwc3_configure_host(sc);
/* ... step 3/5, step 4/5 ... */And inside snps_dwc3_attach_xhci:
device_printf(dev, "attach_xhci: alloc_irq\n");
sc->sc_irq_res = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid,
RF_SHAREABLE | RF_ACTIVE);
device_printf(dev, "attach_xhci: add_child usbus\n");
sc->sc_bus.bdev = device_add_child(dev, "usbus", DEVICE_UNIT_ANY);
device_printf(dev, "attach_xhci: setup_intr\n");
/* ... */That’s ecedc69 dwc3: instrument role_swap so the next wedge tells us where it hangs . Build, deploy, fire. The bench photo:
role_swap: parking gadget, resetting chip, attaching xhci
dwc3_gadget_quiesce: parking gadget for chip reset
dwc3_gadget_quiesce: parked
role_swap: step 1/5 snps_dwc3_reset
role_swap: step 2/5 configure_host
role_swap: step 3/5 do_quirks
role_swap: step 4/5 attach_xhci
attach_xhci: alloc_irq ← LAST PRINTF
WARNING list_empty(&lock->head) failed at .../drm_modeset_lock.c:268
... cascade ...The wedge is between the alloc_irq printf (which runs before bus_alloc_resource_any) and the add_child usbus printf (which runs after it). Two candidates: bus_alloc_resource_any is hanging, or bus_alloc_resource_any returned and device_printf("add_child usbus") itself is hanging because the console path is already blocked behind something.
Tighter: add matching “end” prints ( 7cda476 dwc3: tighter instrumentation — print before AND after each call ):
device_printf(dev, "attach_xhci: alloc_irq begin\n");
sc->sc_irq_res = bus_alloc_resource_any(...);
device_printf(dev, "attach_xhci: alloc_irq end res=%p\n", sc->sc_irq_res);Bench photo, 7cda476 dwc3: tighter instrumentation — print before AND after each call kernel:
... step 4/5 attach_xhci ...
attach_xhci: alloc_irq begin ← LAST PRINTF
WARNING ...alloc_irq end never prints. bus_alloc_resource_any itself is the wedge.
▸ breakthrough
Why would bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid=0, RF_SHAREABLE | RF_ACTIVE) block forever?
Because the gadget already holds it.
When dwc3_gadget_attach ran at boot, it did:
sc->irq_res = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid,
RF_SHAREABLE | RF_ACTIVE);
bus_setup_intr(dev, sc->irq_res, INTR_TYPE_BIO | INTR_MPSAFE,
NULL, dwc3_gadget_interrupt, sc, &sc->irq_hdl);Same dev. Same rid=0. RF_SHAREABLE | RF_ACTIVE. Now snps_dwc3_attach_xhci calls bus_alloc_resource_any on the same dev with the same rid and the same flags. FreeBSD’s rman_reserve_resource_bound blocks a second RF_ACTIVE reservation on the same rid (even with RF_SHAREABLE) until the first reservation is released. The role_swap thread calls into rman, rman calls mtx_sleep waiting for a reservation that nobody’s coming back to release, and that’s the wedge.
Everything else is downstream noise. The console path eventually backs up because vt(4)‘s commits go through rk_vop’s atomic helper and that helper is waiting on a vblank that’s getting starved by the mtx_sleep-blocked role_swap thread holding bus topology locks. The drm_modeset_is_locked warnings fire because the helper, ten seconds in, gives up and tries to log diagnostic state from a context that no longer holds the locks the helper expects. The whole “DRM is broken” surface is one held IRQ resource three layers down.
The gadget was never going to release that resource on its own. We had to do it in quiesce.
▸ fix
Extend dwc3_gadget_quiesce ( 8d886ad dwc3: release gadget IRQ in quiesce so xhci can claim it ) with a fifth step:
/*
* 5. Release the IRQ resource. This is the load-bearing step for
* runtime role-swap: snps_dwc3_attach_xhci does
* bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_SHAREABLE|RF_ACTIVE)
* on the SAME rid=0 IRQ that we already own. rman blocks a second
* RF_ACTIVE allocation on the same rid (even with RF_SHAREABLE)
* until the existing reservation is released — so without this
* teardown, attach_xhci wedges in bus_alloc_resource_any forever.
*/
if (sc->irq_hdl != NULL) {
bus_teardown_intr(dev, sc->irq_res, sc->irq_hdl);
sc->irq_hdl = NULL;
}
if (sc->irq_res != NULL) {
bus_release_resource(dev, SYS_RES_IRQ,
rman_get_rid(sc->irq_res), sc->irq_res);
sc->irq_res = NULL;
}Order matters: tear the handler down first (otherwise the GIC may still route an interrupt to a freed cookie), then release the resource. Once released, sc->irq_res and sc->irq_hdl are both NULL and the gadget ISR is gone — which is fine because all gadget event sources are masked above and we’re not bringing the gadget back in this scope.
Build, deploy, fire. Bench evidence:
snps_dwc3_fdt0: dwc3_gadget_quiesce: parked, irq released
snps_dwc3_fdt0: 64 bytes context size, 32-bit DMA
snps_dwc3_fdt0: xECP capabilities <LEGACY,PROTO,PROTO,DEBUG>
usbus5: 5.0Gbps Super Speed USB v3.0
snps_dwc3_fdt0: role_swap: xhci attached, look for new usbus child
ugen5.1: <Synopsys XHCI root HUB> at usbus5
uhub5 on usbus5
uhub5: <Synopsys XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1>
uhub5: 2 ports with 2 removable, self poweredhw.dwc3.prtcap reports 1 (HOST). WiFi is up the whole time. 5fdec51 dwc3: drop role_swap diagnostic printfs (root cause fixed) drops the diagnostic printfs now that the cause is known and documented.
▸ lesson
Three lessons, in order of how hard they were to learn.
Console output is in the wedge. Every device_printf writes to the kernel console. On a system with a framebuffer console backed by DRM, the path from printf to pixels runs through vt(4) → DRM atomic commit → rk_vop → vblank wait. If anything on the role_swap thread blocks a vblank — even indirectly, by holding a bus lock — the next device_printf blocks behind the queued console write, and you stop seeing diagnostic output exactly when you need it most. The DRM warnings look like the bug because they’re the only thing left that can print, but they’re the symptom of a console path that can’t drain. The fix is matching “begin” + “end” printfs around every suspect call, so even when the after-printf doesn’t render, the absence of it tells you something.
RF_SHAREABLE is not “two RF_ACTIVE reservations may coexist.” It means the IRQ line can be shared between devices, not that the same (dev, rid) may be activated twice. The second RF_ACTIVE allocation on the same (dev, rid) blocks in mtx_sleep until the first releases. There’s no error return. There’s no log message. The thread just goes to sleep. If you’re going to re-allocate an IRQ that another consumer on the same device owns, you have to release it first — even if “the same device” includes a separate child (like a soft-attached xhci alongside a soft-attached gadget).
Quiesce is cheaper than detach and almost always sufficient. The temptation, when the chip is about to be soft-reset, is to free everything: DMA tags, callouts, the usbus child, the ifnet. That’s a lot of teardown surface for a fragile driver where wrapping a single debug printf was previously documented as breaking USB networking. What we actually needed was to (a) stop the chip from generating events, (b) drain anything in flight on the OS side, and (c) release the one resource that conflicts with the new owner. The DMA buffers and the usbus child stay attached the entire time. They don’t need to go away — they just need to stop doing things.
What’s still TODO
The runtime swap works, but it’s one-way. There’s no dwc3_gadget_unquiesce that re-allocates the IRQ, re-enables DEVTEN, re-arms GEVNTSIZ, and brings the gadget back. For true OTG — flip back to gadget when the cable goes away — we’d need that. Not blocking anything immediate.
The trigger today is manual. sysctl hw.dwc3.role_swap=1 from a shell. The right model is: fusb302(4) watches the CC-line role detection, and when the partner asserts itself as a Source (DFP_DATA / Rd vs Rd), the policy engine fires role_swap automatically. That’s plumbing on top of what’s working — wire fusb302’s notification path to dwc3’s sysctl handler.
And the bench partner is still the limit. With the chip in HOST mode at SuperSpeed and the Anker hub plugged in, the hub doesn’t enumerate downstream devices unless we also set dev.fusb302.0.source_role=1 (so the phone provides VBUS) and the typec phy’s flip=1 for CC2 orientation — the same recipe that worked for boot-time host mode on 2026-05-08. Not a role_swap bug; the same hardware-partner issue documented in appendix/usb-c-pd-recipe.
The original ambition for this work was video-out over USB-C: phone → hub → HDMI monitor. That requires the chain fusb302 (PD VDM Discover, Enter Mode for DisplayPort) → rk_typec_phy (lane assignment) → cdn_dp (Cadence DisplayPort controller, Linux’s cdn-dp-core.c ported) → rk_drm (scanout). The cdn_dp skeleton has been bringing up over the last two days — see project_cdn_dp_status in memory. Runtime role_swap is one piece. The next is auto-triggering it from PD events.