Kubernetes Pod IP addresses: How to find and use them

Last month, while troubleshooting a production issue in our OpenShift cluster, I needed to track down why our service mesh wasn’t routing traffic correctly. This led me down a rabbit-hole of Pod IP addressing, that I wish I’d understood better when I first started with Kubernetes.

So I thought it was a good excuse for a blog. In this article, I’ll share what I learned about working with Pod IP addresses in Kubernetes, focusing on two key scenarios:

Finding a Pod’s IP address from outside (for troubleshooting)
Accessing the IP address from inside the Pod (for application configuration)

Find a Pod’s IP address from outside the Pod

So you want to find out the IP address of a Pod from your terminal? It’s quite straightforward.

You can get most information about the state of the Kubernetes cluster using the kubectl command, which fetches this info from the Kubernetes API.

Specifically, the command kubectl get pod will give you information about Pods.

Let’s have a look:

By the way, if you want a quick Kubernetes cluster to follow these instructions, I’m using k3s, which is a small Kubernetes cluster that you can download and run on your laptop.

Use kubectl to get a Pod’s IP address

Create a Pod, either directly or through a Deployment.

I’m creating an Nginx pod with this YAML:

                  apiVersion: v1
kind: Pod
metadata:
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always

                

On your laptop or terminal, type kubectl get pod -o wide:

kubectl get pod -o wide

This will show you the pod’s name and its IP address.

                  NAME    READY   STATUS    RESTARTS   AGE    IP           NODE           NOMINATED NODE   READINESS GATES
nginx   1/1     Running   0          103s   10.42.0.13   toms-laptop   <none>           <none>

If you just want to get the Pod IP address by itself, then use a template to format the output:
```
kubectl get pod nginx --template '{{.status.podIP}}'
```
This uses a “Go template” to format the information of the Pod, and return only the Pod’s IP address.
```
10.42.0.13
```

If you have more than one container in a single Pod, the containers will all have the same IP address.

Common gotcha: Pod IPs aren’t externally accessible

One of the first things that might trip you up is assuming that because a Pod has an IP address, that it’s reachable from outside the cluster. They’re not - and this is by design.

Pod IP addresses are only accessible from inside the Kubernetes cluster.

This is because Kubernetes creates an internal private network for Pods. If you want to access your Pods from outside the cluster, you’ll need to expose your Pod through:

A Service (NodePort or LoadBalancer)
An Ingress or Route (which I use in production)

Now let’s look at how to access Pod IPs from inside the cluster itself.

Find a Pod’s IP address from inside the Pod

Finding the IP address of a Pod from inside the Pod itself, is a little different.

Why would you want to do this?

Perhaps you’re running some kind of application inside the Pod, which needs to know its own IP address.

We can use the Kubernetes Downward API for this.

Getting Pod info with the Downward API

The Kubernetes Downward API lets you access metadata about a Pod, from inside the Pod definition itself.

You can use this data to populate an environment variable, which you can then access from inside the container.

Let’s take a look at an example!

Finding a Pod’s IP address from inside the Pod

First, we need to define an environment variable inside the Pod. We want to populate the environment variable from a field called status.podIP.

To use the Downward API to fetch this, use the valueFrom, fieldRef and fieldPath in your environment variable definitions.

Take a look at this simple Pod definition which populates the IP address into the env var POD_IP:
```
apiVersion: v1
kind: Pod
metadata:
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - env:
    - name: POD_IP
      valueFrom:
        fieldRef:
          fieldPath: status.podIP
    image: nginx
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
```
To add this configuration to a Deployment (instead of a Pod), you’d just need to add the environment variable in the equivalent location in your Pod spec.
Now run your Pod, or restart your Pod if it’s already running.
Start a shell inside a Pod with kubectl exec - this is a really common way to debug Pods in Kubernetes (assuming that your container image has a shell installed):
```
kubectl exec -it nginx -- /bin/sh
```
Type env and you’ll see that the Pod’s IP address is now stored in an environment variable.

I’ve added grep here – it will filter the output from env to show only lines that contain our environment variable, POD_IP:
```
env | grep POD_IP
# should output something like: 
# POD_IP=10.42.0.14
```
So to print the Pod’s IP address from inside the Pod, you can now use:
```
echo $POD_IP
```

Ta-da! But how is this useful to us?

How to use the Pod’s IP address in your application

How can you use this value that’s stored in an environment variable?

Well, we can now read this value from our application inside the Pod. Setting environment variables is a very common way to configure applications in containers.

Most programming languages have a way to read environment variables.

For example:

in a Java application you might use System.getenv("POD_IP")
while in Python, you might use os.environ.

But, here are a few things to be aware of:

My lessons learned

After working with Kubernetes, I’ve learned some pretty important things about Kubernetes networking which I think you should be aware of, too:

Expect Pod IPs to change

In production, Pod IPs can change frequently, due to restarts or rescheduling. Don’t hardcode your Pod IPs, anywhere! - use Kubernetes Services instead.

Network providers really matter

Clusters can behave quite differently, depending on which CNI plugin you’re using (OpenShift SDN, Calico, etc.). When troubleshooting, always check which one you’re dealing with. If you’re using K8s on public cloud, check the docs.

Beware the NetworkPolicies!

These are rules which can prevent Pods from talking to each other - kind of like a firewall. If you can’t reach a Pod by its IP, check if NetworkPolicies are blocking access. They are beloved by administrators, but many hours can be easily wasted here :-)