Cilium CNI Support with AKO
The Cilium Container Network Interface (CNI) plugin is supported on Kubernetes. Cilium can be configured to use either Cluster Scope mode or Kubernetes Host Scope mode for IPAM, and AKO is capable of supporting both.
Viewing IPAM Mode
To see the IPAM mode, check the
ipam field in the
cilium-config configmap in the
In cluster scope mode, the IPAM value is cluster-pool.
In Kubernetes host scope mode, the IPAM value is kubernetes.
Configuring AKO to use Cilium CNI
AKO reads the per-node
PodCIDRs to be able to sync the static route configurations. With Cilium CNI, there are two modes to configure the per-node PodCIDRs.
Cluster Scope IPAM mode
By default, Cilium uses the
Cluster Scope mode for IPAM. To use Cilium in the cluster scope IPAM mode with AKO, the
AKOSettings.cniPlugin value in the AKO Helm chart values.yaml should be set to cilium. For more information, see Configuring AKO and Parameters in Install Avi Kubernetes Operator.
In the cluster scope mode, the podCIDRs range are made available via the
CiliumNode (cilium.io/v2.CiliumNode) CRD and AKO reads this CRD to determine the Pod CIDR to Node IP mappings. The CiliumNode CRD object is created with the same name as the node name (one per node) and specifies the podCIDRs range in the
Kubernetes Host Scope IPAM Mode
In Kubernetes host scope mode, podCIDRs are allocated out of the PodCIDR range associated to each node by Kubernetes. This
PodCIDR range is available in the Node
spec.podCIDRs field. By default, when the
cniPlugin flag is empty, AKO determines the Pod CIDR to Node IP mappings from Node
spec.podCIDRs field and configures the static routes accordingly. Hence, the
cniPlugin flag should be left empty for Kubernetes Host Scope IPAM mode.
AKO supports the SCTP protocol for L4 services.
Starting with version 1.13 Cilium CNI also provides basic SCTP support. However, SCTP support is not enabled by default and needs to be enabled in the Cilium configuration before the SCTP protocol can be used in port definitions.
To enable SCTP support, set the
enable-sctp field in
cilium-config configmap to
NOTE: The Kernel version on the Kubernetes nodes should be 5.2 or higher for Cilium to support SCTP. Otherwise, the Cilium daemon may crash with the following error.
level=fatal msg=”failed to start: daemon creation failed: SCTP support needs kernel 5.2 or newer” subsys=daemon.