Confucius: Achieving Consistent Low Latency with
Practical Queue Management for Real-Time Communications

While the Internet has always been shared among multiple applications, the proliferation of real-time communication applications (e.g., videoconferencing, cloud gaming, virtual reality) has made sharing of bottleneck links particularly challenging. Real-time applications require not just low latency but consistently low latency while sending at moderate to high throughputs (ranging from tens to hundreds of Mbps) (sigcomm2022zhuge, ; sigcomm2020tack, ; imc2022measurement, ). For real-time applications, latency consistency is extremely critical to user experiences. For example, a transient increase in latency to 200 ms might cause cloud gaming users to lose (qomex2015cgsteam, ). Therefore, controlling the latency fluctuation and achieving a consistent low latency for real-time applications is essential.

To better illustrate the problem and the limitations of existing approaches, we revisit the example of Fig. 1 in detail. Consider a user who is on a video call, and their housemate (with whom they share the home router) decides to load a Web page. Technically, one existing real-time flow on the bottleneck will compete with the new flows from one Web page. We simulate the real-time flow’s delay of each video frame using NS-3 and present the results in Fig. 2 (details in § 6). Before considering other queue management mechanisms, let us focus on the performance of FIFO (square markers).

Before t=0s, the sending rate of the real-time flow has converged, with the video frame delay fluctuating around 60ms, which is much lower than the stall threshold (190ms³³3 This is the recommended network delay for video chats by ITU (itu-ddl, ). ) required by the application. However, when flows from loading the homepage ofamazon.com join the competition on the bottleneck, the end-to-end delay for the real-time flow sharply increases. Using FIFO, the delay goes up to more than 400 ms, and stays above the threshold for almost one second, during which a stall occurs and the user experience is impaired. When using FqCoDel, the delay of the real-time flow is even worse since fair queueing shifts more bandwidth away and CoDel drops more packets. We find that the delay always spikes regardless of the underlying CCA (§ 7.2).

We argue that the delay spike is caused by (i) the burst of flows and packets from the competing Web page; (ii) the abrupt reallocation of the available bandwidth by queue management; and (iii) the gradual reaction from the congestion control. Next, we will elaborate on how these common factors result in performance degradation, and explain the limitations of existing works.

This triggering of multiple flows to load one page is shared across different websites. We measured the homepage of Top 1000 websites in November 2023 from the saved Alexa list and presented the distribution in Fig. 3(b). We find that the median number of concurrent ACTIVE flows is 8 while the 90th percentile is 19. The highest one in the Top 200,dailymail.co.uk, has up to 50 active flows and 250 open sockets at the same time. We present the structure and list some famous websites in Appx. A. Moreover, for some websites (e.g., Wikipedia and Google), loading other pages triggers more flows compared to the almost blank home page, which will further exacerbate the degradation experienced by the real-time flow.

Such a sharp decrease in the available bandwidth causes a delay spike to the real-time flow. This is because the CCA needs to gradually probe and match its sending rate to the new available bandwidth, which takes several RTTs (dashed green line in Fig. 5). While the number of in-flight packets is converging to the new bandwidth-delay product, the excessive in-flight packets will cause bufferbloat and result in high end-to-end latency for the real-time flow.

Active queue management (AQM) algorithms, which notify the sender about the network conditions by dropping or marking ECN on packets, cannot prevent stalls either. This is mainly because flows driven by different congestion control algorithms (CCAs) have different perceptions of congestion (e.g., delay, loss, rate). Therefore, as shown in Fig. 2, CoDel (cacm2012codel, ) leads to a latency spike even higher than that of FIFO. We observe similar limitations in other AQMs (§ 7.2).

Some proposals are designed to help the CCA to quickly converge to the new available bandwidth, such as XCP (sigcomm2002xcp, ), RCP (infocom2008rcp, ), Kickass (icnp2016kickass, ), and ABC (nsdi2020abc, ). However, none of these proposals work unless both end hosts and routers collaboratively deploy these protocols and offer no improvement otherwise. This poses significant barriers to deployment on the Internet (sigcomm2022zhuge, ). Moreover, during the convergence of the CCA, the excessive in-flight packets also inflate the RTT. For most CCAs using RTT to update (e.g., adjust the sending rate every RTT), the update period will, in turn, inflate after the first several packets. In the example in Fig. 2, before the Web flows join, the RTT for the real-time flow is around 40 ms. However, during the competition, the RTT inflates to hundreds of milliseconds. Putting all factors together, we can see that for all baselines that do not require labels, the delay spike of the real-time flow goes up to at least 400 ms.

One line of solution is DiffServ (rfc4594diffserv, ), which labels the flows of interest in advance and schedules them differently on the router using StrictPriority or weighted class as shown in Fig. 2. This also includes the recent proposal L4S (rfc9330l4s, ) which we will later evaluate in § 7.2. While this is deployable in datacenters (sigcomm2016karuna, ), it is not practical on the Internet. End hosts have the incentive to fake their labels if that could help their flows have better performance. It is also challenging to coordinate the end host and router on the Internet in the real world since they usually belong to different entities. Even with perfect labels, achieving optimal performance requires optimal allocation of bandwidth across the different classes of traffic. To understand why this is challenging, consider some canonical solutions. StrictPriority, albeit guaranteeing the latency for the real-time flow, will drastically harm the performance of competing Web flows (§ 7.2). Allocating bandwidth for different classes using pre-defined weights needs accurate estimation of the bandwidth demands from both classes, where inaccurate estimation easily leads to unfairness or latency spikes. For example, if we set the ratio between the real-time flows and Web flows to 1:1, the Web flows will suffer from degraded PLT since they cannot obtain their fair shares (Fig. 4(c)), while 1:5 will lead to the latency spike to the real-time flow as well (Fig. 2).

There are further mechanisms as below, which, unfortunately, still reactively respond to network changes. Zhuge (sigcomm2022zhuge, ) reduces the feedback loop between the router and the endpoint from one RTT to sub-RTT levels, but CCA convergence still requires multiple RTTs (§ 2.3). Using the example in Fig. 5, Zhuge tightens the turning point of the green dashed line, but the dominant contributor to delay – the time it takes for the green dashed line to converge to the blue line – persists. FEC is designed for loss recovery (nsdi2023tambur, ; nsdi2024hairpin, ) and is hardly helpful in our example since most of them have no loss at all. Multipath transport will switch to the new path (sigcomm2023converge, ; sigcomm2023cellfusion, ; nsdi2024augur, ), but this also occurs after the sender observes drastic degradation in the current path. Real-time flows still have to suffer from stalls during the adaptation period. Bandwidth estimations from the wireless link layer and below (sigcomm2022zhuge, ; sigcomm2020pbecc, ) are not effective either since the link capacity does not change in the competition.

At a high level, Confucius classifies flows to queues and strategically assigns a portion of the link capacity to each of them, as illustrated in Fig. 6.

To address the goal R1 and R2, Confucius leverages a simple yet powerful insight from § 3.1: Upon the arrival of competitors, the reduction of the available bandwidth of existing flows is inevitable if we want to preserve long-term throughput fairness. Yet, we can gradually and cautiously control the reduction of the available bandwidth during the transient period. Thus, we can eliminate the mismatch between the sending rate of the CCA and the service rate at the bottleneck link for existing real-time flows, thereby taming the latency fluctuation. We will extend our insight of using the EWMA reweight mechanism in § 4.

For R3, by grouping flows with similar queue occupancy into the same queue, flows with different queue occupancies will not affect each other. Meanwhile, with a fixed number of queues to schedule between (instead of per-flow queues such as FQ), latency-sensitive flows will have a consistent latency. Thus, Confucius uses a set of queues ($Q_{1},Q_{2},Q_{3}$), each designed to accommodate old flows with different buffer occupancies, and a separate queue ($Q_{NEW}$) dedicated to new flows. It then adopts a Deficit-Weighted Round-Robin (DWRR) algorithm to schedule between these queues. Finally, Confucius periodically measures flow characteristics and reclassifies flows using a hysteresis-based mechanism to further increase robustness in practice (§ 5).

We first quantitatively demonstrate the advantage of gradually controlling the real-time flow’s bandwidth allocation compared to directly cutting its available bandwidth to its fair share. We measured the stall duration $y$ for the real-time flow in the scenario of a sudden reduction of available bandwidth for four low-latency CCAs (§ 4). Concretely, $y$ denotes the stall duration defined by more than 190 ms of end-to-end delay. We plot $y$ as a function of the Available Bandwidth Reduction Factor (ABRF, the factor we will reduce the available bandwidth) for different CCAs in Fig. 7(a). We find that CCAs respond poorly to sudden, large reductions in bandwidth. For instance, reducing GCC’s available bandwidth to 1/16 of its initial value (i.e., $ABRF=16$) results in a $y>10$ seconds stall. The relationship between the stall duration and ABRF ($y=f_{CCA}(ABRF)$) is super-linear.

To avoid such stalls, Confucius gradually reduces the available bandwidth for the real-time flow. For instance, to achieve a final ABRF of 16, we can reduce the available bandwidth four times, each by half. Fig. 7(b) demonstrates, in the ideal case, the value proposition of this approach. Compared to the super-linear stall duration (solid line copied from Fig. 7(a)), exponentially reducing the sending rate will only increase the stall duration logarithmically with the ABRF (modulated by $f_{CCA}(2)$, a small constant).

Such a smooth reallocation of available bandwidth allows the CCA to learn the reduced bandwidth allocation, and is also robust to the number or size of competing flows. No matter how many flows compete with the real-time flow, the curve of the available bandwidth of the real-time flow is fixed so the delay will remain the same. Meanwhile, adjusting the bandwidth share exponentially yields fast convergence to the fair share, satisfying requirements R1 and R2 together. We prove in § 4.3, that Confucius guarantees that the long-term fairness will not be impaired, and the degradation of the performance for new flows will always be within a constant, additive factor of the FCT under a strictly fair allocation.

To assign service rates to queues, Confucius uses the following process. For each flow, $f$, Confucius computes a weight, $w_{f}$, to represent its share of the bandwidth (service rate). Confucius groups new flows into a separate queue called $Q_{new}$ (depicted in Fig. 6). All existing flows which are mapped to other queues are assigned flow weights of $w_{f}=1$, and are collectively denoted as set $\mathcal{F}_{ext}$. The flow weights of all flows in $Q_{new}$ are computed as follows:

Then, for a given queue, $Q$, the weight is the sum of weights of all flows in $Q$. There are several considerations in Eq. 1:

Moreover, different CCAs have different response times to congestion. For example, Copa needs 5 RTTs to reduce its sending rate, while BBR’s response time is dictated by its probing interval of 6-8 RTTs. To deal with the heterogeneity of CCAs on the Internet (sigmetrics2020gordon, ), we set $\lambda$ as the inverse of the response time of the least responsive CCA among common latency-sensitive CCAs. This ensures that even the least responsive CCA can smoothly react to bandwidth changes. Recall that we measure how different CCAs respond to bandwidth reductions in Fig. 7(a), which shows BBR being the least responsive CCA: When the ABRF is 2, BBR suffers from the longest stall compared with other CCAs due to its a long probing period of 6-8 RTTs. Thus, given a typical RTT of 30-50 ms for Web services (www2021wisetrans, ), we set $\lambda$=0.004 (ms${}^{-1}$) to have a doubling interval of $\frac{1}{\lambda}$=250 ms, matching BBR’s probing period. Experiments in § 7.2 demonstrate satisfactory results for not only BBR but also other CCAs.

We still follow the same example in § 2.2. Consider one real-time flow running by itself on a bottleneck link. At $t=0$, $N$ new flows, each with size $B$, join the same bottleneck link and compete with the existing flow. $B_{0}$ is the initial congestion window for Web flows. We show that Confucius guarantees bounded stall for the existing real-time flow while yielding FCTs for Web flows within a constant additive factor of what FQ provides. For simplicity, we summarize the results in Tab. 1 and leave the analytical details to Appx. B.

For FQ and FIFO, we observe that the stall duration ($q^{max}_{P}$) scales linearly with the number of new flows, $N$, and is therefore unbounded, where $N$ can go to more than 100 in some Web pages (Fig. 3). This is quite straightforward – when $N$ flows start to compete with the real-time flow, the available bandwidth of the real-time flow drops to $1/N$. Intuitively, as $N$ increases, the more the available bandwidth for the real-time flow drops, resulting in drastic delay fluctuation.

For class-based queues (CBQ, weighted class), pre-labeling the real-time flow enables the scheduler to allocate the real-time flow with a fixed bandwidth share, resulting in a constant stall. However, if the weights are not accurate (i.e., not matching the traffic ratio), CBQ converges unfairly, and the FCT degradation for new flows becomes unbounded (§ 2.4).

Finally, Confucius yields bounded performance degradation for both sets of flows. On one hand, Confucius ensures that the stall for real-time flows is constant only depending on the CCA’s latency sensitivity (denoted by $q_{0}$), the responsiveness ($k$), the feedback loop ($\tau$), and Confucius’s decay parameter ($\lambda$)⁴⁴4 When using Copa with an RTT of 40ms, $q_{\textsf{Confucius}}^{max}$ is $\approx$640 ms. As we show experimentally in § 7.2, the actual delay using Confucius is much lower. . On the other hand, Confucius can also ensure the FCT degradation for new flows is bounded by an additive constant factor to the decay parameter ($\lambda$), which goes to negligible with the increase of the flow sizes.

Confucius puts short flows into a separate queue $Q_{new}$ and classifies long flows with different buffer occupancy aggressiveness into $Q_{1},Q_{2},\cdots,Q_{n}$. Queue indices increase with buffer target i.e., $Q_{1}$ will be shorter than $Q_{3}$, as shown in Fig. 6. Each queue $Q_{i}$ targets a buffer occupancy of $q_{0}^{(i)}$. We robustly classify flows as follows:

In practice, Confucius has two following considerations.

Schedulers requiring labels (marked in green) protect prelabeled RT flows, but considerably degrade the PLT for the Web traffic. DualQ+Prague improves DualQ since the CCA on the end-host can react more effectively, but still incur considerable penalty on Web flows since it incurs packet drops to the Web flows (non-scalable from DualQ’s design). Note that even when using StrictPriority, the real-time flow still suffers from 60 ms degradation on average due to bandwidth fluctuation. Confucius is almost on par with schemes relying on end-host labels in terms of the delay of the real-time flows. Remember that it is unrealistic to assume that an end-host will correctly label all traffic (§ 2.4).

Most importantly, Confucius does not incur too much penalty on the PLT of Web pages, and pushes the Pareto front of the schedulers not requiring labels (the dashed blue line) forward. Confucius reduces the PLT compared to CoDel, RED, FQ+CoDel, and HHF since Confucius gently adjusts the bandwidth share for the Web flows. Even compared to FQ+FIFO, Confucius only increases the average PLT by up to 8% (up to 88 ms) in three subfigures, which is much smaller than the improvement on the stall duration. The improvement when using BBR is not as significant as the other two CCAs – this is due to the suboptimal performance of BBR in controlling the delay for the real-time flow, as we also saw in Fig. 9.

We also measures the number of websites suffering from a long PLT of longer than 2 seconds, which is the threshold for good user experience (sigcomm2019e2e, ). When using FQ+CoDel, 227 websites will suffer from long PLT, while Confucius can reduce this number by half to 127. Confucius’s results is comparable to FQ+FIFO, demonstrating the fairness of Confucius for competing Web flows. For baselines requiring labels that behave well in Fig. 11(a), at least 198 websites suffer from long PLTs. The closest label-based baseline is CBQ (1:1), which we can also see from Fig. 10. Besides the unrealistic label requirement, we will later show in § 7.3 that CBQ (1:1) does not scale to the variation of workloads. Fig. 10 evaluates with Web flows, but if the competing flows are not Web flows but FTP flows instead, the performance will be drastic since CBQ (1:1) adopts a fixed ratio between classes.

In this subsection, we test our theoretical analysis by investigating whether Confucius can provide consistent performance in controlled workloads. We vary the workload by changing the number of flows in a Web page and the size of Web flows. We measure the stall duration in different scenarios and the degradation on the PLT against FQ.

We further change the size of Web flows (from short flows to long flows) and see if Confucius is capable of handling all types of competing traffic. With the increase of the size of flows, the competing flows are changing from short flows (e.g., Web) to long flows (e.g., FTP). We vary the size of Web flows from 15KB to 9MB, and run 5 flows with the same size to compete with the HRT flow. When using FIFO, the real-time flow will suffer from drastic stall due to failure to provide inter-CCA fairness across flows, as shown in Fig. 15(a). The real-time flow using FQ also has a long stall of hundreds of milliseconds. In contrast, Confucius is still able to achieve negligible stall for the real-time flow and bounded PLT degradation for Web flows at the same time.

In this subsection, we zoom in on Confucius’s flow classification mechanism. We find that Confucius can accurately group flows of the same/similar CCA together without any prior knowledge or labels from end hosts, which in turn leads to better performance compared to the baselines.

We simultaneously run four long flows of different CCAs: one Cubic flow, one BBR flow, one GCC flow, and one Copa flow for 100 seconds. We plot the network delay for each flow over time in Fig. 13(a). We can clearly see that Copa and GCC enjoy a consistent low latency around 40-60 ms, even when they are competing with BBR and Cubic flows.

To understand Confucius’s superior performance, we look at its classification results over time and present them in Fig. 13(b). Four bars represent the classification results of Confucius for four flows over time, while three colors indicate which queue the flow is classified into. Confucius can correctly classify flows using different CCAs into the correct queues: Copa and GCC flows can be stably put into the low occupancy queue ($Q_{1}$, blue), the BBR flow into the median occupancy queue ($Q_{2}$, yellow), and the Cubic flow into the high occupancy queue ($Q_{3}$, green). This follows our previous observation in Fig. 9 – Copa and GCC both demonstrate similar low buffer occupancy, while Cubic occupies the buffer aggressively, and BBR in the middle. Moreover, we notice that the Cubic flow can temporarily be in the same queue as BBR, as shown in the yellow lines in the green bar in Fig. 13(b). This is expected as the Cubic flow has (at times) a low queue occupancy in its probing period. Second, flows with different CCAs can co-exist in the same queue as long as they have similar buffer occupancy. In this experiment, Copa and GCC flows are put into the same queue since they have similar buffer occupancy. As we can see in Fig. 13(a), these two flows still have consistent low latency all the time.

We also measure the Jain’s fairness index (JFI) in Fig. 13(c) to present the fairness when using different schemes. We compare the results (the delay of the Copa and GCC flow, and the JFI of all flows) in the same experiment with other schedulers in Fig. 13(c). With Confucius, the Copa and GCC flows also enjoy a reasonable fair share of the bandwidth as FQ – the JFI in this experiment is 0.98 in Fig. 13(c), where JFI close to one indicates a better fairness.

We implement Confucius as a kernel module of queue disciplines (qdisc) in traffic control in Linux kernel 4.4.0 (1.4k LoCs) and evaluate the performance of Confucius on a software router based on Intel Xeon E5-2620 v4 CPU. We run the official implementation of Copa (copa-ccp, ). We find that Confucius achieves significant benefits in kernel implementations while only adding marginal processing delay.

We stream the video frames using socket and TCP Copa, and measure the end-to-end delay for the real-time flow. We then set up an HTTP server based on Python to replay the Web traces we collected. We also measure the computational overhead of Confucius and the baselines. We record the processing time for the enqueue and dequeue operation in Linux tc using printk, where the reweight and reclassification in Confucius are both implemented.

As shown in Fig. 16(a), Confucius reduces the stall duration by more than 60% without the need for labels on each packet. This result is similar to our simulation in Fig. 10(a). Moreover, from our experiments, 86% of websites when using Confucius do not suffer from stall at all. In contrast, this number is only 56% and 30% for FIFO and FQ.

We vary the number of long-running flows to measure the overhead of Confucius. Note that the processing time of Confucius is insensitive to the number of short flows, as they all belong to the new-flow queue. As shown in Fig. 16(b), Confucius slightly increases the processing time for each packet compared to FQ. Even for 100 concurrent long-running flows, the per-packet processing time of Confucius is still 5 $\mu s$, indicating a processing rate of 200 kpps and is at the same magnitude as FQ. Since Confucius is mainly designed for last-mile routers such as home routers, this can satisfy the daily usage of home access points or last-mile routers. We stress that the kernel implementation of Confucius can be further optimized for high-performance execution in the future. We leave the further exploration of Confucius over numerous flows (e.g., in the routers in the core network) in the future.

We further evaluate the performance of Confucius in a series of microbenchmarking settings. In Appx. C.3, we demonstrate that the hysteresis mechanism of Confucius (§5.1) can work with bandwidth-probing CCAs (e.g., BBR) and stably and correctly classify flows. We further show that Confucius will not have side effects on the fairness aspect in Appx. C.2, and investigate if the bottleneck is not the router where Confucius is deployed in Appx. C.4. Even if multiple real-time flows are competing simultaneously, Confucius can still handle those flows and provide significant performance improvements against baselines (Appx. C.5).