How to use kubectl

kubectl is a command-line utility used to manage kubernetes clusters. This page exists to provide an introduction to its basic usage and some of our most frequently used commands.

This page will not cover the initial installation process or the process of connecting kubectl to a cluster, as that will change depending on:

• How you’re running kubectl

• Where your cluster is

Concepts

Before getting into any commands, it’s helpful to have a good understanding of kubernetes. In this section, we’ll go over some of the core concepts that will make kubectl make sense.

What is a Kubernetes installation?

Broken down into its most basic components, a kubernetes installation is just a collection of resource definitions (generally defined with .yaml files) and the corresponding resources that those definitions control.

Each resource has a type. There are many resource types, including but not limited to:

• Deployments

• ReplicaSets

• Pods

• Ingresses

• Jobs

• PersistentVolumes

• PersistentVolumeClaims

• Services

• Secrets

Many kubectl commands expect references to one or more of these resources. The general format of a reference is:

<resource-type>/<resource-name>

So a pod might be referenced as pod/asr-en-7c67b59c74-hnzfh, a deployment as deployment/asr-en, and so on.

Namespaces

Objects in kubernetes all live within specific namespaces. These namespaces are determined when resources are created and typically correspond with how the resources are grouped conceptually.

For example, the lumenvox helm charts are typically installed in the lumenvox namespace. We typically use the ingress-nginx helm chart to manage our ingresses, and those resources live in the ingress-nginx namespace. Other common namespaces include linkerd for linkerd components, and kube-system for the components used to manage kubernetes itself. To see a list of all namespaces, run kubectl get namespaces (or kubens, if you’ve already installed it).

kubectl commands are aware of namespaces, so depending on the resource you want to reference in a command, you will need to specify the namespace in one way or another.

For example, the kubectl command to list pods is:

kubectl get pods

However, if you run this with a freshly connected kubectl, you’ll likely get the following output:

No resources found in default namespace.

This is because kubectl uses the default namespace by default, and we generally don’t install anything in that namespace.

One way to specify the namespace is with the -n option:

jb@DEV-JACK-2:~$ kubectl get pods -n lumenvox
NAME                                      READY   STATUS      RESTARTS       AGE
admin-portal-5c8bbdb464-ll7kr             2/2     Running     0              14d
archive-96ff6db8f-ms422                   2/2     Running     0              14d
asr-en-7c67b59c74-hnzfh                   2/2     Running     0              2d20h
asr-es-746567bcc8-2m28h                   2/2     Running     0              28h
asr-fr-5f6586794-4nn5b                    2/2     Running     0              28h
binary-storage-85489db79-kxxzt            2/2     Running     50 (13d ago)   14d
configuration-66b9796478-q4fpz            2/2     Running     0              14d
deployment-77444b9f-bc4fm                 2/2     Running     0              14d
deployment-portal-697ddb5dbc-xgcps        2/2     Running     0              14d
diag-tools-87f6dfc9f-xxqp4                1/1     Running     0              20d
grammar-857d5bdf9f-vf2wr                  2/2     Running     42 (13d ago)   14d
itn-en-7c58f6fdf8-r2vph                   2/2     Running     40 (13d ago)   14d
itn-es-687c9b56b-zwt7r                    2/2     Running     40 (13d ago)   14d
itn-fr-6686594bbb-wfkxl                   2/2     Running     40             14d
license-85d4f74b7-fg4d4                   2/2     Running     0              2d11h
lumenvox-api-795bdf4bb5-8pbmd             2/2     Running     0              3h25m
management-api-5d8c55fb9d-vdqpj           2/2     Running     0              34h
neural-tts-en-us-6cfb58db9f-kx687         2/2     Running     0              13d
neural-tts-es-us-77b4964587-h2pkt         2/2     Running     0              13d
nlu-f4df9845f-n64n8                       2/2     Running     0              47h
persistent-volume-directory-setup-w9nc8   0/1     Completed   0              34h
reporting-api-6cdd55d65d-bm8jq            2/2     Running     48 (13d ago)   14d
resource-856458f96c-qlsxx                 2/2     Running     0              14d
session-969c7c7c9-pf29s                   2/2     Running     50 (13d ago)   14d
vad-77f7b879dc-5rbtj                      2/2     Running     0              13d

This shows us all the pods installed in the lumenvox namespace. 

Using -n in this way gets the job done, but if you find yourself frequently running commands, it’s often more efficient to set lumenvox as kubectl’s default namespace. This can be done in 2 ways:

• (Recommended) install the kubens utility from GitHub: ahmetb/kubectx: Faster way to switch between clusters and namespaces in kubectl

  • Once this is installed, you can run kubens lumenvox to set the default namespace to lumenvox. From this point, the command kubectl get pods should return all the lumenvox containers. If you want to specify another namespace for just 1 command, you can still use the -n option to override this new default. And of course, if you want to change the default to a different namespace, you can do that too.

• kubectl config set-context --current --namespace=YOUR_NAMESPACE

  • This does the same thing, but it’s harder to remember. kubens is the easy button.

Additionally, some kubectl commands can select all namespaces with the -A option. For example, to list all pods in all namespaces, you would run kubectl get pods -A.

Deployments, ReplicaSets, Pods, and Containers

The majority of our work involves containers, but we don’t define containers directly in kubernetes. Instead, we largely rely on Deployments, which are a higher-level abstraction than containers.

• Deployments manage ReplicaSets, generally 1 ReplicaSet per Deployment

• ReplicaSets manage Pods. As you might guess from the name, they can manage multiple copies of the same Pod, if the deployment has been scaled appropriately (more on this later).

• A Pod is an individual unit that contains one or more containers.

• Containers here are the same as in docker: a sort of ultra-lightweight VM where programs are run.

There are various kubectl commands that require references to these sorts of resources. Depending on the command, the behavior can change based on whether you reference a pod, a deployment, or a replica set. Some examples of this will be provided below.

Useful Commands

It is highly recommended to add an alias for kubectl in your working environment:

1. edit ~/.bashrc, creating it if it doesn’t already exist

   1. add the line alias k="kubectl" to the bottom

2. use the updated bashrc by running 

   
   
   
   

   . ~/.bashrc

   The remainder of this section will use k in place of kubectl

“Get” Commands

• List all running pods: k get pods

• List all ingresses: k get ingress

• List all deployments: k get deployments

• Get a deployment definition in YAML format: k get deployment <deployment-name> -o yaml

The -A option (for “all namespaces”) is valid for the first 3, and for most other operations where we’re listing potentially multiple objects. It does not work with the 4th command, which is getting the details from a specific object.

“Describe” Commands

k describe is another way to get details about a specific resource. While k get <resource> -o yaml mainly returns the YAML that defines the object, k describe will return more of the circumstantial information: the current status, any related replica sets, any related events, etc.

• Describe a deployment: k describe deployment <deployment-name>

• Describe a pod: k describe pod <pod-name>

Logs

There are a few ways to access the logs from a container. The most basic way is to reference a specific pod and container.

For example, if you want the English ASR logs, you could run k get pods to find the pod name. Once you have that, you can get the ASR logs with:

k logs pod/asr-en-7c67b59c74-hnzfh -c asr

Note the -c asr, to specify the asr container within the asr-en-7c67b59c74-hnzfh pod. To see a list of containers within a pod, you can run k describe for that pod.

If you haven’t scaled anything past 1, or if you don’t care about getting the logs from a specific pod and instead just want the logs from any English ASR pod, this process can be made a bit more efficient by referencing the deployment instead of the pod:

k logs deployment/asr-en -c asr

This approach involves one fewer command and no copy/pasting.

To get the live logs, you can use the -f option:

k logs deployment/asr-en -c asr -f

Editing Live Resources

If you’re testing changes to resource definitions and don’t want to uninstall/reinstall the helm charts every time, you can modify live definitions with kubectl. You won’t generally edit pods directly, but deployments and ingresses are common use cases.

To edit a deployment:

k edit deployment/asr-en

This will open the definition within your configured editor (probably defaulting to vim). Make changes, save, and quit. Quit without making changes to cancel the operation.

After editing a deployment, kubernetes should automatically spin up a new pod and bring down the old pod.

After editing an ingress, the ingress-nginx ingress controller should automatically pick up the changes and update its configuration (you can see this in its logs).

Restarting Pods

In kubernetes, we don’t really restart pods. Instead, we delete pods and let their ReplicaSets recreate them. To delete a pod:

k delete pod <pod-name>

Be aware that unlike the logs command, using a deployment reference here instead of a pod reference will result in drastically different behavior:

• When we delete a Pod but the ReplicaSet that manages it still exists, the ReplicaSet will create a new Pod as configured.

• When we delete a Deployment, that will delete the ReplicaSets it manages, which will delete the Pods it manages, and so on. They will not respawn.

In general, the delete subcommand can be used on almost any resource, but not every resource type will respawn. For that reason, you will mostly use this against pods.

Creating Resources

There are a few options to create resources.

For some resources, kubectl provides a create command. The syntax differs for each resource type - see k help create for more info.

If you have a YAML with the appropriate definitions, you can also use the apply subcommand to create an appropriate resource in the cluster:

k apply -f some-resource-definition.yaml

Scaling Resources

We often need multiple copies of pods, especially work-heavy pods like the ASR. This is managed by ReplicaSets, which are managed by Deployments. To scale a deployment:

k scale deployment deployment/asr-en --replicas=3

Bashing into a Container/Running Processes in a Container

It’s occasionally useful to bash into a container.

Container selection is the same as in the logs command: you can use a pod reference with the -c option, or you can use a deployment reference with the -c option, depending on what level of specificity you need. The command is:

k exec -it deployment/asr-en -c asr -- bash

You don’t have to run bash here - it could be any binary in the container. If you want to run a noninteractive command, you can remove the -it options:

k exec deployment/asr-en -c asr -- ls -1 /var

Note that you are limited by what binaries exist on the container. We have multiple containers that not only don’t have bash, but don’t even have ls.

Copying Files from a Container

For this operation, you must use a pod reference. Ex:

k cp asr-en-7c67b59c74-hnzfh:/zoneinfo.zip -c asr zoneinfo.zip

This will only work for containers that have tar installed.

Watching Pods Initialize

Instead of running k get pods over and over, consider using the watch command:

watch kubectl get pods

Shortcuts

While using kubectl, you’ll find yourself frequently including resource types in commands: “deployment”, “ingress”, “replicaset”, etc. That will quickly add up to lots of typing, so it’s highly recommended to learn the abbreviations. Here are a few: