Kubernetes is API driven. Anything you can do from the Kubernetes command-line interface (kubectl
), you can do through the Kubernetes API. At Revelry, we love Kubernetes and we love working with APIs, so it’s no surprise that we’ve built apps using the Kubernetes API.
Many popular programming languages have Kubernetes API clients. We’ve worked with k8s for Elixir, kubeclient for Ruby, and kubernetes-client for Node.
You can use the Kubernetes API to create, configure, inspect, and delete any kind of resource from Kubernetes. The sky is the limit.
How to Build an App with Kubernetes API
If building an application on the Kubernetes API interests you, here’s a broad outline of the process:
1. Service Account and Token Secret
Set up a service account to use for your application with a bearer token secret:
apiVersion: v1
kind: ServiceAccount
metadata:
name: my-kube-app
namespace: default
secrets:
- name: my-kube-app-secret
---
apiVersion: v1
kind: Secret
metadata:
name: my-kube-app-secret
annotations:
kubernetes.io/service-account.name: my-kube-app
type: kubernetes.io/service-account-token
You can use the default
service account for your application’s namespace, but creating a service account for this specific purpose allows you to fine-tune the permissions that this account has. This is more secure.
Note: most Kubernetes resources (including service accounts, secrets, roles, role bindings, pods, jobs, etc) are namespaced. A common source of errors when setting up a Kubernetes API app is placing your resources such as service accounts or secrets in the wrong namespace. This leads to your application not being able to see those resources in the API or not receiving the proper permissions to operate on them.
2. Permissions and Role Binding
You will need to set up a role (set of permissions) and role binding (associates the service account to the role).
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: my-kube-app-role
rules:
# if you are bringing in resources that are not part of the base API
# (for example, jobs are in the `batch` group), you need to add
# those groups to this array
- apiGroups: [""]
resources: ["pods"] # adjust as needed
verbs: ["get", "delete", "list", "patch", "update"] # adjust as needed
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: my-kube-app-role-binding
roleRef:
kind: ClusterRole
name: my-kube-app-role
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: my-kube-app
namespace: default
3. Copy Secret Bearer Token
The secret bearer token will be available to the app. Once your application is deployed to your cluster, you can either mount the secret token as an environment variable or read it from the file system. For now, copy the secret token value so that you have it for local development.
- See this documentation for information on reading the token from the file system.
- See this documentation for information on making the token available as an environment variable.
4. Auth the API
Use the bearer token or service account certificate to auth to the API. Make sure you can see the right resources. If you copied the token value from the prior steps for local development, and point your application at the cluster’s API, you can test against the live cluster from your development environment (be careful!).
- Instructions for Kubeclient (Ruby)
- Instructions for Kubernetes (Elixir)
- Instruction for Kubernetes-client (Node)
5. Create Resources
To create resources, you will need to write a specification for it, as if you were writing a YAML config for kubectl.
If you are creating deployments, jobs, pods, etc, remember to specify the right service account! You can use the serviceAccountName
under the template spec for your resource.
Here’s an example of specifying a YAML for a job using Ruby syntax:
<<-YAML
apiVersion: batch/v1
kind: Job
metadata:
name: myjob-#{SecureRandom.hex}
spec:
template:
spec:
serviceAccountName: my-kube-app
containers:
- name: lint
image: myjob-docker-image:1.2.3
restartPolicy: Never
backoffLimit: 4
YAML
6. API Library
Using the API library of your choice, and the specification you made, you can create, update, or delete resources. For example, using Ruby, this might look like:
job_yaml = <<-YAML
apiVersion: batch/v1
kind: Job
metadata:
name: myjob-#{SecureRandom.hex}
spec:
template:
spec:
serviceAccountName: my-kube-app
containers:
- name: lint
image: myjob-docker-image:1.2.3
restartPolicy: Never
backoffLimit: 4
YAML
client = Kubeclient::Client.new('http://localhost:8080/apis/batch', 'v1')
job_resource = Kubeclient::Resource.new(YAML.safe_load(job_yaml), nil)
client.create_job job_resource
7. Deploy Your Application into the Custer
Reminder: if you host your docker images in a private docker repo, you will need to make sure the proper service account has imagePullSecrets
for that repo. If you set up a special service account for this app, you’ll need to set an imagePullSecret
for that. If you are using default
, make sure it has the right image pull secrets.
8. Test It
It should be able to list and manipulate the proper kinds of objects.
That’s a broad overview of integrating with the Kubernetes API to automate Kubernetes tasks. I hope you find it helpful. Please feel free to reach out with questions or suggestions.
We're building an AI-powered Product Operations Cloud, leveraging AI in almost every aspect of the software delivery lifecycle. Want to test drive it with us? Join the ProdOps party at ProdOps.ai.