Calico eBPF Source IP Preservation: The Unexpected Story of High Tail Latency
The Calico eBPF data plane is your choice if latency is your primary concern. It was very disturbing that some benchmarking brought to our attention that eBPF had higher tail latency than iptables. The 99+% percentiles were higher by as much as a few hundred milliseconds. We did a whole bunch of experiments and we could not crack the nut until we observed that there are some occasional and unexpected TCP reset (RST) packets, but no connections were reset.