Firedancer Unveils fd_quic: A High-Performance Implementation of QUIC and Solana Transaction Ingest Network Protocols

Firedancer, an independent validator client for Solana built by Jump, recently announced their latest technical milestone, fd_quic, a high-performance implementation of the QUIC and Solana transaction ingest network protocols. This breakthrough comes with significant improvements in scalability and robustness, making the Solana network more resilient against spam attacks.

QUIC Protocol and Solana

The Solana network utilizes the QUIC protocol for various purposes, including transaction propagation. Firedancer's custom networking stack is designed from scratch to leverage receive-side scaling (RSS), a hardware-accelerated form of load balancing. This allows for a shared-nothing parallel architecture, where each CPU core manages a portion of the incoming traffic with minimal overhead.

RSS offers better throughput and simplifies implementation compared to software-based load balancing. It eliminates the need for complex global schedulers, async runtimes, locks, and atomics. Firedancer's decision to bypass the Linux networking stack using XDP/AF_XDP, recent kernel technologies, further optimizes performance.

The Complexity of QUIC

QUIC's 151 pages of specifications bring considerable complexity. Unable to find an existing C library that met their licensing, performance, and reliability requirements, Firedancer built their own. Their implementation, fd_quic, boasts optimized data structures and algorithms, with (almost) zero heap allocations while handling traffic, ensuring the validator won't run out of memory.

Live Demo Highlights

Firedancer celebrated their months-long journey of making QUIC quick with a live demo showcasing three design goals: Compatibility, Performance, and Scalability.

• Compatibility: A spam tool built from modules in the Solana Labs validator sent numerous transactions to the Firedancer validator via QUIC, demonstrating that their QUIC implementation was correct.
• Performance: A test pushed the limits of a single CPU core running fd_quic using multiple spammers. The result was 5.8 Gbps or 270k TPS, reaching 1.4 million TPS with small transactions. Eight CPU cores were required to generate this rate.
• Scalability: The same performance test was run with RSS enabled on a 25 Gbps network interface and fd_quic running on four CPU cores. This showcased linear scaling properties, with each core handling roughly the same rate for a near 4x improvement in throughput.

During the test, a Firedancer QUIC server running on just four CPU cores on a single server reliably ingested 21.8 Gbps of incoming transactions (around 1.08M TPS with ~1kB txn size) from Solana Labs clients. This test was limited by the available network hardware (a 25 Gbps link), but Firedancer welcomes users to push performance further with the available documentation.

As work on more components continues, the blockchain community eagerly anticipates Firedancer's future developments.