RKE2 Ansible Role

This Ansible role will deploy RKE2 Kubernetes Cluster. RKE2 will be installed using the tarball method.

The Role can install the RKE2 in 3 modes:

• RKE2 single node

• RKE2 Cluster with one Server(Master) node and one or more Agent(Worker) nodes

• RKE2 Cluster with Server(Master) in High Availability mode and zero or more Agent(Worker) nodes. In HA mode you should have an odd number (three recommended) of server(master) nodes that will run etcd, the Kubernetes API (Keepalived VIP or Kube-VIP address), and other control plane services.

• Additionally it is possible to install the RKE2 Cluster (all 3 modes) in Air-Gapped functionality with the use of local artifacts.

It is possible to upgrade RKE2 by changing rke2_version variable and re-running the playbook with this role. During the upgrade process the RKE2 service on the nodes will be restarted one by one. The Ansible Role will check if the node on which the service was restarted is in Ready state and only then proceed with restarting service on another Kubernetes node.

Requirements for Ansible Controller

• Ansible 2.10+
• netaddr Python package

Tested on

• Rocky Linux 9
• Ubuntu 24.04 LTS

Role Variables

This is a copy of defaults/main.yml

---
Determines whether downgrades of the RKE2 version are allowed.
If set to false, the role will prevent downgrades unless explicitly permitted.
Set to true to allow downgrades of the RKE2 version.
Note: This setting is ignored in Ansible check mode, and the related prevention task will be skipped.
rke2allowdowngrade: false
The node type - server or agent
rke2type: "{{ 'server' if inventoryhostname in groups[rke2serversgroupname] else 'agent' if inventoryhostname in groups[rke2agentsgroup_name] }}"
Deploy the control plane in HA mode
rke2hamode: false
Install and configure Keepalived on Server nodes
Can be disabled if you are using pre-configured Load Balancer
rke2hamode_keepalived: true
Install and configure kube-vip LB and VIP for cluster
rke2hamode_keepalived needs to be false
rke2hamode_kubevip: false
Kubernetes API and RKE2 registration IP address. The default Address is the IPv4 of the Server/Master node.
In HA mode choose a static IP which will be set as VIP in keepalived.
Or if the keepalived is disabled, use IP address of your LB.
rke2apiip: "{{ hostvars[groups[rke2serversgroupname].0]['ansibledefaultipv4']['address'] | default(hostvars[groups[rke2serversgroupname].0]['ansibledefaultipv6']['address'] ) }}"
optional option for RKE2 Server to listen on a private IP address & port
rke2apiprivate_ip:
rke2apiprivate_port: 9345
optional option for kubevip IP subnet
rke2apicidr: 24
optional option for kubevip
rke2_interface: eth0
optional option for IPv4/IPv6 addresses to advertise for node
rke2bindaddress: "{{ hostvars[inventoryhostname]['ansible' + rke2_interface]['ipv4']['address'] }}"
kubevip load balancer IP range
rke2loadbalancerip_range: {}
 range-global: 192.168.1.50-192.168.1.100
 cidr-finance: 192.168.0.220/29,192.168.0.230/29
Install kubevip cloud provider if rke2hamode_kubevip is true
rke2kubevipcloudproviderenable: true
Enable kube-vip to watch Services of type LoadBalancer
rke2kubevipsvc_enable: true
Specify which image is used for kube-vip container
rke2kubevipimage: ghcr.io/kube-vip/kube-vip:v0.9.2
Specify which image is used for kube-vip cloud provider container
rke2kubevipcloudproviderimage: ghcr.io/kube-vip/kube-vip-cloud-provider:v0.0.12
Enable kube-vip IPVS load balancer for control plane
rke2kubevipipvslbenable: false
Enable layer 4 load balancing for control plane using IPVS kernel module
Must use kube-vip version 0.4.0 or later
rke2kubevipserviceelectionenable: true 
By default ARP mode provides a HA implementation of a VIP (your service IP address) which will receive traffic on the kube-vip leader.
 
To circumvent this kube-vip has implemented a new function which is "leader election per service",
 
instead of one node becoming the leader for all services an election is held across all kube-vip instances and the leader from that election becomes the holder of that service. Ultimately,
 
this means that every service can end up on a different node when it is created in theory preventing a bottleneck in the initial deployment.
 
minimum kube-vip version 0.5.0

(Optional) Change parameters for leader election - see upstream install flags link below
rke2kubevipleaseduration: 5
rke2kubeviprenewdeadline: 3
rke2kubevipretryperiod: 1
rke2kubeviploglevel: 4
(Optional) A list of kube-vip flags
All flags can be found here https://kube-vip.io/docs/installation/flags/
rke2kubevipargs: []
- param: lb_enable
  value: true
- param: lb_port
  value: 6443
Prometheus metrics port for kube-vip
rke2kubevipmetrics_port: 2112
Enable ARP for kube-vip load balancer
rke2kubeviparp_enable: true
Enable kube-vip control plane load balancer for RKE2
rke2kubevipcp_enable: true
Namespace for kube-vip control plane load balancer
rke2kubevipcp_namespace: "kube-system"
Enable kube-vip DDNS for control plane load balancer
rke2kubevipddns_enable: false
Enable kube-vip UPnP for control plane load balancer
rke2kubevipupnp_enable: false
Enable kube-vip leader election for control plane load balancer
rke2kubevipleaderelection_enable: true
Resources for kubevip DaemonSet Pods.
rke2kubevipdaemonset_resources: {}
 requests:
   memory: 40Mi
Resources for kubevip cloud controller Pods.
rke2kubevipcloudcontrollerresources: {}
 requests:
   memory: 40Mi
Add additional SANs in k8s API TLS cert
rke2additionalsans: []
Configure cluster domain
rke2clusterdomain: cluster.example.net
API Server destination port
rke2apiserverdest_port: 6443
Server nodes taints
rke2servernode_taints: []
  # - 'CriticalAddonsOnly=true:NoExecute'
Agent nodes taints
rke2agentnode_taints: []
Pre-shared secret token that other server or agent nodes will register with when connecting to the cluster
rke2_token: defaultSecret12345
RKE2 version
rke2_version: v1.25.3+rke2r1
URL to RKE2 repository
rke2channelurl: https://update.rke2.io/v1-release/channels
URL to RKE2 install bash script
e.g. rancher Chinese mirror http://rancher-mirror.rancher.cn/rke2/install.sh
rke2installbash_url: https://get.rke2.io
Local data directory for RKE2
rke2datapath: /var/lib/rancher/rke2
Default URL to fetch artifacts
rke2artifacturl: https://github.com/rancher/rke2/releases/download/
Local path to store artifacts
rke2artifactpath: /rke2/artifact
Airgap required artifacts
rke2_artifact:
  - sha256sum-{{ rke2_architecture }}.txt
  - rke2.linux-{{ rke2_architecture }}.tar.gz
  - rke2-images.linux-{{ rke2_architecture }}.tar.zst
Timeout for fetching artifacts in seconds
rke2artifactfetch_timeout: 30
Changes the deploy strategy to install based on local artifacts
rke2airgapmode: false
Airgap implementation type - download, copy or exists
- 'download' will fetch the artifacts on each node,
- 'copy' will transfer local files in 'rke2_artifact' to the nodes,
- 'exists' assumes 'rke2artifact' files are already stored in 'rke2artifact_path'
rke2airgapimplementation: download
Local source path where artifacts are stored
rke2airgapcopysourcepath: localartifacts
Tarball images for additional components to be copied from rke2airgapcopy_sourcepath to the nodes
(File extensions in the list and on the real files must be retained)
rke2airgapcopyadditionaltarballs: []
Destination for airgap additional images tarballs ( see https://docs.rke2.io/install/airgap#tarball-method )
rke2tarballimagespath: "{{ rke2data_path }}/agent/images"
Enable Conditional Image Imports - create a .cache.json file in the images directory so that
RKE2 only reimports tarballs that have changed (by size or mtime) since the last import, even
across restarts. Speeds up startup at the cost of guaranteed image reimport on every boot.
See https://docs.rke2.io/install/airgap?airgap-load-images=Manually+Deploy+Images#enable-conditional-image-imports
rke2conditionalimage_imports: false
Architecture to be downloaded, currently there are releases for amd64 and s390x
rke2_architecture: amd64
Destination directory for RKE2 installation script
rke2installscript_dir: /var/tmp
RKE2 channel
rke2_channel: stable
Do not deploy packaged components and delete any deployed components
Valid items: rke2-canal, rke2-coredns, rke2-ingress-nginx, rke2-metrics-server
rke2_disable: []
Option to disable kube-proxy
disablekubeproxy: false
Option to disable builtin cloud controller when working with aws, azure, gce etc
For onprem environment, this should remain false and keep rke2cloudprovider_name as "external"
https://docs.k3s.io/networking/networking-services#deploying-an-external-cloud-controller-manager (same for RKE2)
rke2disablecloud_controller: false
Cloud provider to use for the cluster (aws, azure, gce, openstack, vsphere, external)
applicable only if rke2disablecloud_controller is true
rke2cloudprovider_name: "external"
Path to custom manifests deployed during the RKE2 installation
It is possible to use Jinja2 templating in the manifests
rke2custommanifests: []
Path to static pods deployed during the RKE2 installation
rke2staticpods: []
Configure custom Containerd Registry
rke2customregistry_mirrors: []
  # - name:
  #   endpoint: {}
  rewrite: '"^rancher/(.*)": "mirrorproject/rancher-images/$1"'
Configure custom Containerd Registry additional configuration
rke2customregistry_configs: []
  - endpoint:
    config:
Path to Container registry config file template
rke2customregistry_path: templates/registries.yaml.j2
Path to RKE2 config file template
rke2_config: templates/config.yaml.j2
Etcd snapshot source directory
rke2etcdsnapshotsourcedir: etcd_snapshots
Etcd snapshot file name.
When the file name is defined, the etcd will be restored on initial deployment Ansible run.
The etcd will be restored only during the initial run, so even if you will leave the file name specified,
the etcd will remain untouched during the next runs.
You can either use this or set options in rke2etcdsnapshots3options
rke2etcdsnapshot_file: ""
Etcd snapshot location
rke2etcdsnapshotdestinationdir: "{{ rke2datapath }}/server/db/snapshots"
(Optional) Etcd snapshot schedule
The schedule is in cron format, e.g. "0 /12   "
rke2etcdsnapshot_schedule: "0 /12   "
Etcd snapshot s3 options
Set either all these values or rke2etcdsnapshotfile and rke2etcdsnapshotsource_dir
rke2etcdsnapshots3options:
  # https://docs.rke2.io/datastore/backuprestore?highlight=restore&_highlight=sna&etcdsnap=Multiple+Servers#s3-compatible-object-store-support
  # s3_endpoint: "" # required
  # access_key: "" # required
  # secret_key: "" # required
  # bucket: "" # required
  # snapshot_name: "" # optional - if specified, etcd will be restored upon the first initialization, that is, when starting from a clean slate
  # skipsslverify: false # optional
  # endpoint_ca: "" # optional
  # region: "" # optional - defaults to us-east-1
  # folder: "" # optional - defaults to top level of bucket
  # proxy: "" # optional - Proxy server to use when connecting to S3, overriding any proxy-related environment variables
  # insecure: false # optional - Disables S3 over HTTPS
  # timeout: "" # optional - S3 timeout (default: 5m0s)
  # s3_retention: 5 # optional - Number of snapshots in S3 to retain (default: 5)
Override default containerd snapshotter
rke2snapshotter: "{{ rke2snapshooter }}"
rke2_snapshooter: overlayfs # legacy variable that only exists to keep backward compatibility with previous configurations
Deploy RKE2 with default CNI canal
rke2_cni: [canal]
Validate system configuration against the selected benchmark
(Supported value is "cis-1.23" or eventually "cis-1.6" if you are running RKE2 prior 1.25 or "cis" for rke2 1.30+)
rke2cisprofile: ""
Download Kubernetes config file to the Ansible controller
rke2downloadkubeconf: false
Name of the Kubernetes config file will be downloaded to the Ansible controller
rke2downloadkubeconffilename: rke2.yaml
Destination directory where the Kubernetes config file will be downloaded to the Ansible controller
rke2downloadkubeconf_path: /tmp
Default Ansible Inventory Group name for RKE2 cluster
rke2clustergroupname: k8scluster
Default Ansible Inventory Group name for RKE2 Servers
rke2serversgroup_name: masters
Default Ansible Inventory Group name for RKE2 Agents
rke2agentsgroup_name: workers
(Optional) A list of Kubernetes API server flags
All flags can be found here https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver
rke2kubeapiserver_args: []
(Optional) List of Node labels
k8snodelabel: []
(Optional) Additional RKE2 server configuration options
You could find the flags at https://docs.rke2.io/reference/server_config
rke2serveroptions:
  - "option: value"
  - "node-ip: {{ rke2bindaddress }}"  # ex: (agent/networking) IPv4/IPv6 addresses to advertise for node
(Optional) Additional RKE2 agent configuration options
You could find the flags at https://docs.rke2.io/reference/linuxagentconfig
rke2agentoptions:
  - "option: value"
  - "node-ip: {{ rke2bindaddress }}"  # ex: (agent/networking) IPv4/IPv6 addresses to advertise for node
(Optional) Configure Proxy
All flags can be found here https://docs.rke2.io/advanced#configuring-an-http-proxy
rke2environmentoptions: []
  - "option=value"
  - "HTTP_PROXY=http://your-proxy.example.com:8888"
(Optional) Customize default kube-controller-manager arguments
This functionality allows appending the argument if it is not present by default or replacing it if it already exists.
rke2kubecontrollermanagerarg:
  - "bind-address=0.0.0.0"
(Optional) Customize default kube-scheduler arguments
This functionality allows appending the argument if it is not present by default or replacing it if it already exists.
rke2kubescheduler_arg:
  - "bind-address=0.0.0.0"
Ingress controller selection - single value or list of values. Supported: ingress-nginx, traefik, none
NOTE: The default value will change to traefik in role release 1.52.0
rke2ingresscontroller: ingress-nginx
(Optional) Configure nginx via HelmChartConfig: https://docs.rke2.io/networking/networking_services#nginx-ingress-controller
rke2ingressnginx_values:
  controller:
    config:
      use-forwarded-headers: "true"
rke2ingressnginx_values: {}
(Optional) Configure traefik via HelmChartConfig: https://docs.rke2.io/networking/networking_services
rke2traefikvalues:
  providers:
    kubernetesIngressNginx:
      enabled: true
      ingressClass: "rke2-ingress-nginx-migration"
      controllerClass: 'rke2.cattle.io/ingress-nginx-migration'
rke2traefikvalues: {}
Cordon, drain the node which is being upgraded. Uncordon the node once the RKE2 upgraded
rke2drainnodeduringupgrade: false
Additional args that will be passed to the kubectl drain command e.g. --pod-selector
rke2drainadditional_args: ""
Wait for all pods to have a status of running or succeeded after rke2-service restart during rolling restart.
rke2waitforallpodstobe_ready: false
Wait for all pods to be ready after rke2-service restart during rolling restart.
Named "healthy" to keep backwards compatibility with existing variable names.
rke2waitforallpodstobe_healthy: false
The args passed to the kubectl wait command
rke2waitforallpodstobehealthyargs: --for=condition=Ready -A --all pod --field-selector=metadata.namespace!=kube-system,status.phase!=Succeeded
Enable debug mode (rke2-service)
rke2_debug: false
(Optional) Customize kubelet config using KubeletConfiguration - https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/
rke2kubeletconfig:
  imageGCHighThresholdPercent: 80
  imageGCLowThresholdPercent: 70
Note that you also need to add the following to kubelet args:
rke2kubeletarg:
  - "--config=/etc/rancher/rke2/kubelet-config.yaml"
rke2kubeletconfig: {}
(Optional) Customize default kubelet arguments
rke2kubeletarg:
  - "--system-reserved=cpu=100m,memory=100Mi"
(Optional) Customize default kube-proxy arguments
rke2kubeproxy_arg:
  - "proxy-mode=ipvs"
(Optional) Customize default kube-proxy extra mounts
rke2kubeproxyextramount:
  - "/lib/modules:/lib/modules:ro"
The value for the node-name configuration item
rke2nodename: "{{ inventory_hostname }}"
default pod network range for rke2
rke2clustercidr:
  - 10.42.0.0/16
default service network range for rke2
rke2servicecidr:
  - 10.43.0.0/16
Enable SELinux for rke2
rke2_selinux: false

Inventory file example

This role relies on nodes distribution to masters and workers inventory groups. The RKE2 Kubernetes master/server nodes must belong to masters group and worker/agent nodes must be the members of workers group. Both groups has to be the children of k8s_cluster group.

[masters]
master-01 ansible_host=192.168.123.1
master-02 ansible_host=192.168.123.2
master-03 ansible_host=192.168.123.3
[workers] worker-01 ansible_host=192.168.123.11 worker-02 ansible_host=192.168.123.12 worker-03 ansible_host=192.168.123.13
[k8s_cluster:children] masters workers

Playbook example

This playbook will deploy RKE2 to a single node acting as both server and agent, using the RKE2 version defined in the role defaults.

- name: Deploy RKE2
  hosts: node01
  become: yes
  roles:
     - role: lablabs.rke2

This playbook will update an already deployed RKE2 cluster (inventory contains one server and multiple workers) to a specific version.

- name: Deploy RKE2
  hosts: all
  become: yes
  vars:
    rke2_version: v1.35.1+rke2r1
  roles:
     - role: lablabs.rke2

This playbook will deploy RKE2 to a cluster with one server(master) and several agent(worker) nodes in air-gapped mode using the copy implementation, which transfers local artifact files from the Ansible controller to the target nodes. It will use Multus and Calico as CNI

- name: Deploy RKE2
  hosts: all
  become: yes
  vars:
    rke2airgapmode: true
    rke2airgapimplementation: copy
    rke2_cni:
      - multus
      - calico
    rke2_artifact:
      - sha256sum-{{ rke2_architecture }}.txt
      - rke2.linux-{{ rke2_architecture }}.tar.gz
      - rke2-images.linux-{{ rke2_architecture }}.tar.zst
    rke2airgapcopyadditionaltarballs:
      - rke2-images-multus.linux-{{ rke2_architecture }}
      - rke2-images-calico.linux-{{ rke2_architecture }}
  roles:
     - role: lablabs.rke2

This playbook will deploy RKE2 to a cluster with HA server(master) control-plane and several agent(worker) nodes. The server(master) nodes will be tainted so the workload will be distributed only on worker(agent) nodes. The role will install also keepalived on the control-plane nodes and setup VIP address where the Kubernetes API will be reachable. it will also download the Kubernetes config file to the local machine.

- name: Deploy RKE2
  hosts: all
  become: yes
  vars:
    rke2hamode: true
    rke2apiip : 192.168.123.100
    rke2downloadkubeconf: true
    rke2servernode_taints:
      - 'CriticalAddonsOnly=true:NoExecute'
  roles:
     - role: lablabs.rke2

This playbook will deploy RKE2 to a cluster with HA server(master) control-plane and several agent(worker) nodes using kube-vip for VIP management and LoadBalancer services, and Traefik as the ingress controller instead of the default ingress-nginx. It will also download the Kubernetes config file to the local machine.

- name: Deploy RKE2
  hosts: all
  become: yes
  vars:
    rke2hamode: true
    rke2hamode_keepalived: false
    rke2hamode_kubevip: true
    rke2apiip: 192.168.123.100
    rke2loadbalancerip_range:
      range-global: 192.168.123.200-192.168.123.250
    rke2ingresscontroller: traefik
    rke2traefikvalues:
      logs:
        general:
          level: "DEBUG"
    rke2downloadkubeconf: true
  roles:
     - role: lablabs.rke2

Having separate token for agent nodes

As per server configuration documentation it is possible to define an agent token, which will be used by agent nodes to connect to cluster, giving them less access to cluster than server nodes have. Following modifications to above configuration would be necessary:

• remove rke2_token from global vars
• add to group_vars/masters.yml:

rke2_token: defaultSecret12345 rke2agenttoken: agentSecret54321

• add to group_vars/workers.yml:

rke2_token: agentSecret54321

While changing server token is problematic, agent token can be rotated at will, as long as servers and agents have the same value and the services (rke2-server and rke2-agent, as appropriate) have been restarted to make sure the processes use the new value.

Troubleshooting

Playbook stuck while starting the RKE2 service on agents

If the playbook starts to hang at the Start RKE2 service on the rest of the nodes task and then fails at the Wait for remaining nodes to be ready task, you probably have some limitations on your nodes' network.

Please check the required Inbound Rules for RKE2 Server Nodes at the following link: .

RKE2 upgrade playbook failed due to an interrupted upgrade

In case the new RKE2 version was installed but not started, rerun the playbook with the variable rke2allowdowngrade: true to bypass the downgrade prevention check.

License

MIT

Author Information

Created in 2021 by Labyrinth Labs