Spiderpool Performance Testing¶

Spiderpool is a high-performance IPAM CNI plugin for underlay networks. This report will compare its performance with the mainstream underlay IPAM CNI plugins (such as Whereabouts, Kube-OVN) and the widely used overlay IPAM CNI plugin calico-ipam under the "1000 Pod" scenario.

Background¶

Why do we need to do performance testing on the underlay IPAM CNI plugin?

1. The speed at which IPAM allocates IP addresses largely determines the speed of application publishing.
2. Underlay IPAM often becomes a performance bottleneck when a large-scale Kubernetes cluster recovers from failures.
3. Under underlay networks, private IPv4 addresses are limited. Within a limited range of IP addresses, concurrent creation of Pods can involve IP address preemption and conflict, and it is challenging to quickly adjust the limited IP address resources.

ENV¶

• Kubernetes: v1.25.4
• container runtime: containerd 1.6.12
• OS: CentOS Linux 8
• kernel: 4.18.0-348.7.1.el8_5.x86_64

Objects¶

This test is based on the CNI Specification 0.3.1, using macvlanwith Spiderpool, and selecting several other underlay network solutions in the open source community for comparison:

Plan¶

During the testing, we will follow the following agreement:

• IPv4/IPv6 dual-stack.
• When testing the underlay IPAM CNI plugins, ensure that the number of available IP addresses is 1:1 to the number of Pods as much as possible. For example, if we plan to create 1000 Pods in the next step, we should limit the number of available IPv4/IPv6 addresses to 1000.

Specifically, we will attempt to create a total of 1000 Pods on the Kubernetes cluster in the following two ways, and record the time taken for all Pods to become Running:

• Create only one Deployment with 1000 replicas.
• Create 100 Deployments with 10 replicas per Deployment.

Next, we will use the following command to delete these 1000 Pods at once, and record the time it took for all Pods to become Running again:

kubectl get pod | grep "prefix" | awk '{print $1}' | xargs kubectl delete pod

Finally, we delete all Deployments and record the time taken for all Pods to completely disappear.

Result¶

1 Deployment with 1000 replicas¶

During the testing of macvlan + Whereabouts, in the creation scenario, 922 Pods became Running at a relatively uniform rate within 14m25s. After that, the growth rate of Pods significantly decreased, and ultimately it took 25m18s for 1000 Pods to become Running. As for the re-creation scenario, after 55 Pods became Running, Whereabouts basically stopped working, and the time consumption was close to infinity.

100 Deployments with 10 replicas¶

Summary¶

Although Spiderpool is an IPAM CNI plugin suitable for underlay networks, it faces more complex IP address preemption and conflict issues than mainstream overlay IPAM CNI plugins, but its performance in most use cases is not inferior to the latter.

Comments