Running Krustlet on Amazon Elastic Kubernetes Service (EKS)
Currently, EKS does not support running managed node groups with custom Amazon Machine Images (AMI).
However, it does appear the feature might be coming soon.
Until that time, we can use eksctl to create and manage a node group with a custom Krustlet-based AMI.
The following tools are needed to complete this walkthrough:
Building the Krustlet-based AMI
We will be using Packer to spin up an EC2 instance to build the AMI.
There is a Makefile in
docs/howto/assets/eks that will run
packer for you.
It will use a
c5.2xlarge EC2 instance to build the AMI with. Use the
instance_type variable to
make to change the type of the EC2 instance used.
make to build the AMI:
$ cd docs/howto/assets/eks $ make
You can also build the AMI with a different version of Krustlet from a forked repo. For example:
$ cd docs/howto/assets/eks $ KRUSTLET_VERSION=$(git rev-parse --short HEAD) KRUSTLET_SRC=https://github.com/jingweno/krustlet/archive/$(git rev-parse --short HEAD).tar.gz make krustlet
This command will take a while to build Krustlet from source on the EC2 instance. In the future, a prebuilt binary for Amazon Linux 2 might be available that would speed up the AMI creation process.
If everything works correctly, you should see the command complete with output similar to:
... ==> Builds finished. The artifacts of successful builds are: --> amazon-ebs: AMIs were created: us-west-2: ami-07adf9ce893885a3d --> amazon-ebs:
Make note of the AMI identifier (in the example output above it would be
ami-07adf9ce893885a3d) as it will be used to create the EKS cluster.
Creating the EKS cluster
We will be using eksctl to deploy the EKS cluster.
Create a file named
cluster.yaml with the following contents, replacing the
ami fields with your values:
apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: krustlet-demo region: <YOUR_AWS_REGION_HERE> version: "1.15" nodeGroups: - name: krustlet ami: <YOUR_AMI_HERE> instanceType: t3.small minSize: 1 maxSize: 3 desiredCapacity: 2 ssh: allow: true overrideBootstrapCommand: /etc/eks/bootstrap.sh --krustlet-node-labels "alpha.eksctl.io/cluster-name=krustlet-demo,alpha.eksctl.io/nodegroup-name=krustlet"
This will create a EKS cluster named
krustlet-demo with a single unmanaged
node group named
krustlet with two
Be aware that the
overrideBootstrapCommand setting is required to properly
boot the nodes. Without it, the Krustlet service will not be started and the
nodes will not automatically join the cluster.
eksctl to create the cluster:
$ eksctl create cluster -f cluster.yaml
This command will take a long time to run as it provisions the EKS cluster and nodes.
Eventually, the command will be stuck on the following output:
... [ℹ] waiting for at least 1 node(s) to become ready in "krustlet"
With another shell, ensure the nodes have joined the cluster:
$ kubectl get nodes NAME STATUS ROLES AGE VERSION ip-192-168-24-34.us-west-2.compute.internal Ready agent 23s v1.17.0 ip-192-168-44-27.us-west-2.compute.internal Ready agent 17s v1.17.0
You should see two nodes with different names in the output.
Running a WebAssembly application
Let’s deploy a demo WebAssembly application to the cluster:
$ kubectl apply -f demos/wasi/hello-world-rust/k8s.yaml
Check that the pod ran to completion:
$ kubectl get pod hello-world-wasi-rust NAME READY STATUS RESTARTS AGE hello-world-wasi-rust 0/1 ExitCode:0 0 7s
This output shows the pod completed with an exit code of 0.
Take a look at the log to see the output of the application:
$ kubectl logs hello-world-wasi-rust hello from stdout! hello from stderr! POD_NAME=hello-world-wasi-rust FOO=bar CONFIG_MAP_VAL=cool stuff Args are: 
Congratulations! You’ve run a WebAssembly program on an EKS cluster!
Deleting the cluster
eksctl to delete the cluster and the nodes:
$ eksctl delete cluster --name krustlet-demo
Deleting the Krustlet AMI
Determine the snapshot identifier of the AMI, where
$AMI_ID is the identifier
of your Krustlet AMI:
$ aws ec2 describe-images --image-ids $AMI_ID | grep SnapshotId
aws to deregister the AMI, where
$AMI_ID is the identifier of your
$ aws ec2 deregister-image --image-id $AMI_ID
Next, delete the snapshot, where
$SNAPSHOT_ID is the previously determined
$ aws ec2 delete-snapshot --snapshot-id $SNAPSHOT_ID