https://darrenqu.net/tags/infiniband/Technical writing on network engineering: campus and WAN, hybrid cloud interconnect, automation and observability, and the data center fabrics — InfiniBand, RoCE, Spectrum-X — underneath GPU clusters.Hugoen-USDarren QuDarren Qu2026 Darren QuSat, 07 Mar 2026 00:00:00 +0000https://darrenqu.net/ai-networking/infiniband-routing-engines/Sat, 07 Mar 2026 00:00:00 +0000https://darrenqu.net/ai-networking/infiniband-routing-engines/AI NetworkinginfinibandThe subnet manager picks the routing engine, and the routing engine decides whether your fabric deadlocks. Min-hop, Up/Down, fat-tree and DragonFly+, and what each one is actually protecting you from.https://darrenqu.net/ai-networking/infiniband-fabric-diagnostics/Sun, 22 Feb 2026 00:00:00 +0000https://darrenqu.net/ai-networking/infiniband-fabric-diagnostics/AI Networkinginfinibandibdiagnet is the first command to run on a sick fabric. What its output files contain, which counters actually matter, and how to get from a slow training job to a specific port.https://darrenqu.net/ai-networking/mlxlink-mlxcables-troubleshooting/Sun, 08 Feb 2026 00:00:00 +0000https://darrenqu.net/ai-networking/mlxlink-mlxcables-troubleshooting/AI NetworkinginfinibandWhen a fabric link flaps or runs at the wrong width, the answer is usually in the optics. How to read PCIe and fabric port state, per-lane signal quality and cable EEPROM with the NVIDIA firmware tools.