How to Configure Kubernetes Health Checks and Readiness Probes Properly

Kubernetes gives you powerful building blocks for keeping applications available, but those building blocks only help if you configure them with care. Health checks are one of the most important pieces of that puzzle. They influence whether traffic reaches a Pod, whether a container gets restarted, and how safely your application rolls out during updates. Kubernetes uses probes to decide when a container is alive, ready, or still starting up, and those signals directly affect Services and EndpointSlices, which in turn control traffic routing. (kubernetes.io)

Used well, probes improve resilience. Used poorly, they can create cascading restarts, traffic flapping, and rollout failures. Kubernetes documentation explicitly warns that incorrect liveness behavior can cause cascading failures and that liveness checks should be used carefully to detect unrecoverable problems such as deadlocks. Readiness probes, by contrast, are meant to keep traffic away from Pods that are not yet able to serve requests safely. (kubernetes.io)

This post walks through how to design probes that are practical, low-noise, and production-friendly. It covers the differences between startup, liveness, and readiness checks; how to build the right endpoints; how the probe types interact with Services and rolling updates; and how to validate that your configuration behaves the way you expect.

General illustration of Kubernetes probes and traffic flow

1. Why health checks matter in Kubernetes: traffic routing, self-healing, and safer rollouts

Health checks in Kubernetes do three jobs at once: they help route traffic, they help Kubernetes recover from failure, and they help you ship changes more safely. The biggest misconception is that “health” is a single thing. In reality, Kubernetes cares about different kinds of health. A container may be running but not ready, ready but not stable, or still starting up. Kubernetes uses different probe types to distinguish those states. (kubernetes.io)

For traffic routing, readiness is the key signal. When a Pod’s readiness probe fails, Kubernetes marks the Pod as not ready and removes it from the backends used by Services. The EndpointSlice controller maps Pod readiness into endpoint conditions, so a not-ready Pod should stop receiving regular traffic without needing to be killed. This is essential for avoiding requests to Pods that are still warming caches, waiting on dependencies, or performing maintenance. (kubernetes.io)

For self-healing, liveness is the signal that matters. A liveness probe is meant to detect whether the process is truly stuck or broken in a way that only a restart can fix. If the kubelet sees repeated liveness failures, it restarts the container. That makes liveness useful for deadlocks and similar unrecoverable states, but dangerous if your check is too strict or depends on fragile external systems. (kubernetes.io)

For safer rollouts, readiness helps reduce blast radius. During deployments, new Pods should only receive traffic when they are genuinely able to serve. If your readiness probe checks more than “the process is up” and instead verifies that critical dependencies are available, you can avoid sending requests to a Pod that would otherwise return errors. Kubernetes explicitly recommends readiness checks for applications with strict backend dependencies. (kubernetes.io)

In short: use readiness to protect users, use liveness to recover from true failures, and use startup probes to give slow applications time to initialize before the other probes begin. That separation is what makes Kubernetes health checks useful rather than noisy. (kubernetes.io)

2. The three probe types explained: startup, liveness, and readiness—and what each one should do

Kubernetes supports three probe types, and each has a different purpose. Understanding that distinction is the foundation of a good configuration. A startup probe is for the initialization phase, a liveness probe is for detecting whether a container is stuck or broken, and a readiness probe is for deciding whether a container should receive traffic. (kubernetes.io)

A startup probe is the simplest to understand: it tells Kubernetes when the application has finished starting. While the startup probe is running, Kubernetes does not execute liveness or readiness probes. That means it can protect slow-starting applications from being killed by probes that would otherwise fire too early. If your app takes longer to boot than a normal liveness window allows, a startup probe is the correct fix. (kubernetes.io)

A liveness probe asks, “Should this container be restarted?” Kubernetes recommends using it only for situations that indicate the process is unhealthy in a way a restart can fix, such as a deadlock. If a liveness probe fails too aggressively, you can end up in a restart loop under load, which often makes the problem worse rather than better. (kubernetes.io)

A readiness probe asks, “Should this Pod receive traffic right now?” It does not restart the container when it fails. Instead, Kubernetes removes the Pod from load balancing until the probe succeeds again. This makes readiness ideal for dependency checks, warmups, maintenance mode, and temporary overload conditions. Kubernetes also notes that readiness probes run throughout the container lifecycle. (kubernetes.io)

The most important rule is that these probes should not all test the same thing. A startup probe can often reuse the same endpoint as liveness, but readiness usually needs to be more nuanced. Liveness should stay cheap and reliable. Readiness should be dependency-aware. Startup should be generous enough to cover real initialization time without masking true failures forever. (kubernetes.io)

Comparison table of startup, liveness, and readiness probes

3. How to design the right endpoint: keep liveness cheap, make readiness dependency-aware

Designing good probe endpoints is mostly about restraint. The best probes answer a narrow question quickly and consistently. They should not become full diagnostic endpoints, and they should not perform expensive work. Kubernetes docs recommend a low-cost HTTP endpoint for liveness and often for readiness as well, but with different semantics and thresholds. (kubernetes.io)

For liveness, the endpoint should confirm that the application event loop, request handler, or main worker is functioning at a basic level. It should not call databases, external APIs, message brokers, or third-party services unless the application truly cannot function without them and a restart is the correct remedy. If the dependency is down, killing the container usually does not fix the dependency. It only adds churn. (kubernetes.io)

For readiness, the endpoint can be smarter. It may check whether the app has loaded config, completed migrations it depends on, established internal caches, connected to critical dependencies, or finished background warmups. Kubernetes explicitly recommends readiness for scenarios where an app depends on backend services and should not receive traffic until those services are available. (kubernetes.io)

A common pattern is:

/live or /healthz for liveness: quick internal check only.
/ready or /readyz for readiness: internal check plus critical dependency state.
/startup when initialization takes long enough that normal liveness would be too aggressive. (kubernetes.io)

The most important design principle is predictability. A readiness endpoint should fail only when the app truly should not serve traffic. If it flaps because a noncritical dependency blips, Kubernetes will keep removing and re-adding the Pod from the Service, which can create instability. Liveness should be even more conservative; it should only fail when the process is effectively broken. (kubernetes.io)

If you need maintenance mode, do not hack around it with liveness failures. Use readiness to withdraw traffic cleanly while keeping the Pod alive long enough to finish in-flight work or drain connections. Kubernetes notes that readiness failures remove Pods from EndpointSlices and Service backends, which is the right mechanism for traffic shedding. (kubernetes.io)

4. HTTP, TCP, and gRPC probe options: when to use each and what to avoid

Kubernetes supports HTTP, TCP, and gRPC probes, and each has a valid use case. Choosing the right one matters because the probe mechanism should match what you are trying to verify. (kubernetes.io)

HTTP probes are the most flexible choice for web applications and APIs. They let you define an explicit path, and your application can return structured status information. HTTP probes are usually the easiest way to express readiness and liveness separately, since you can expose different endpoints for each. They are also the easiest to understand operationally. (kubernetes.io)

TCP probes are useful when you only need to know whether a port is accepting connections. They are simple and lightweight, but they tell you less than HTTP. A TCP socket can be open even if the app is not ready to process useful work. Because of that, TCP probes are best when transport availability is all you care about, not application semantics. (kubernetes.io)

gRPC probes are a newer option and are well suited to services that already use gRPC as their application protocol. Kubernetes documents gRPC probing as a native option, which can avoid awkward HTTP wrappers for gRPC-native services. That said, the probe should still be minimal and should not become a full RPC dependency graph check. (kubernetes.io)

What to avoid:

Don’t use TCP probes when you need to know whether the application is truly ready.
Don’t make HTTP probes expensive.
Don’t make readiness depend on too many external systems unless traffic really must stop when one of them fails.
Don’t use exec probes when a simple HTTP or gRPC check would do; exec probes can add overhead and operational complexity. Kubernetes also highlights the importance of reducing exec-probe overhead in current best practices. (kubernetes.io)

If you are using gRPC services, gRPC health checking can be a good fit because it aligns probe behavior with the service protocol. If you are running a traditional web app, HTTP remains the most readable and maintainable option. If you merely need to know whether a socket is listening, TCP is sufficient. The best probe is the one that answers the smallest useful question. (kubernetes.io)

5. Tuning probe timing: initialDelaySeconds, periodSeconds, timeoutSeconds, successThreshold, and failureThreshold

Timing is where many otherwise good probe designs go wrong. A correct endpoint can still behave badly if the probe window is too short, the retry count is too low, or the timeout is unrealistic for the workload. Kubernetes defines these fields clearly, and the defaults are not always right for real systems. (kubernetes.io)

initialDelaySeconds controls how long Kubernetes waits after container start before beginning the probe. It defaults to 0. For startup probes, liveness, and readiness, this is often the first knob people reach for. However, if your application has variable startup time, a startup probe is usually a better solution than simply stretching liveness delays. Kubernetes also notes that if a startup probe is configured, liveness and readiness do not begin until it succeeds. (kubernetes.io)

periodSeconds controls how often the probe runs. The default is 10 seconds. A shorter period gives quicker detection, but it increases probe traffic and can make flapping more visible. Readiness probes may also run at times other than the configured interval while a Pod is not ready, in order to make the Pod ready faster. (kubernetes.io)

timeoutSeconds sets how long Kubernetes waits for a probe response before considering that attempt failed. The default is 1 second. That is often too aggressive for services under load, for services with cold caches, or for network paths that occasionally spike. A timeout should reflect how quickly the endpoint can reliably respond under normal worst-case conditions. (kubernetes.io)

successThreshold is the number of consecutive successes required after a failure. For liveness and startup probes, it must be 1. For readiness probes, it can be greater than 1, which is useful when you want to avoid flapping. (kubernetes.io)

failureThreshold is the number of consecutive failures allowed before Kubernetes acts on the probe result. For startup and liveness probes, repeated failures trigger a restart. For readiness, repeated failures simply mark the Pod not ready. Kubernetes defaults this to 3. (kubernetes.io)

A useful rule of thumb is:

Slow startup? Use a startup probe.
Occasional transient dependency issues? Give readiness a slightly higher failureThreshold.
Real deadlocks? Keep liveness strict enough to recover quickly.
Under load? Make sure timeoutSeconds is not unrealistically low. (kubernetes.io)

The real goal is not “fastest possible detection.” It is “fast enough to help, slow enough to avoid noise.” That balance varies by workload, but the tuning knobs are the same.

6. Common misconfigurations that cause cascading restarts, flapping, or premature traffic cuts

Probe failures are not always application failures. Very often they are configuration failures. The most common mistakes are subtle, and they tend to show up only under load or during deployment. Kubernetes documentation explicitly warns that incorrect liveness configuration can lead to cascading failures, reduced scalability, and increased load on remaining Pods. (kubernetes.io)

One frequent mistake is making liveness too smart. If liveness checks databases, queues, or other external systems, a dependency outage can cause the kubelet to restart otherwise healthy containers. That creates extra pressure at exactly the moment your system is already struggling. In most cases, dependency checks belong in readiness, not liveness. (kubernetes.io)

Another common issue is using the same endpoint for liveness and readiness without adjusting behavior. Kubernetes notes that a low-cost HTTP endpoint can be reused, but the semantics should still be appropriate. If both probes fail for the same transient reason, your app may be restarted when it only needed to stop receiving traffic briefly. (kubernetes.io)

A third mistake is forgetting about slow startup. If your app needs time to load data, compile assets, connect to services, or warm caches, a plain liveness probe can kill it before it ever becomes useful. That is exactly what startup probes were designed to prevent. (kubernetes.io)

Probe flapping often comes from over-tight timing. A 1-second timeout, a 1-second period, and a low failure threshold can create noisy false positives in real clusters. Likewise, readiness flapping can cause Pods to be added and removed from Services repeatedly, leading to poor user experience and uneven load distribution. (kubernetes.io)

Another issue is premature traffic cuts during rollout. If readiness depends on every noncritical dependency, a small transient blip can prevent new Pods from ever becoming ready. That can stall deployments or leave too few replicas serving traffic. Readiness should be strict about user safety, but not so strict that it becomes brittle. (kubernetes.io)

The safest mindset is to treat probe failures as operational signals, not as a proxy for “anything bad happened.” Make the signal narrow, stable, and intentional.

7. Probe patterns for real workloads: slow startups, cache warmups, sidecars, databases, and microservices

Different workloads need different probe strategies. The right pattern depends on whether your app is CPU-heavy at startup, stateful, proxy-dependent, or highly distributed. Kubernetes documentation gives the primitives; the pattern choice is yours. (kubernetes.io)

For slow startups, use a startup probe first, then a simple liveness probe afterward. This is the cleanest pattern for apps that spend a long time loading data, running migrations, or initializing runtime state. The startup probe gives the app breathing room, and once it succeeds, liveness takes over to catch later deadlocks. (kubernetes.io)

For cache warmups, readiness is the right place to hold traffic back. A Pod can be running but still not ready if key caches, indexes, or in-memory structures are not populated. Once warmup is complete, readiness can flip to true and traffic will begin. This works especially well when the app is functional but would serve poor results or slow responses until warmed. (kubernetes.io)

For sidecars, especially service mesh or log-forwarding sidecars, think carefully about which container is actually required for serving traffic. If the application container is ready but the sidecar is not, readiness may need to reflect the whole Pod’s ability to serve. If the sidecar is noncritical, do not block readiness on it unnecessarily. The Pod-level Ready condition depends on container readiness and other conditions, so be explicit about what “ready” means in your architecture. (kubernetes.io)

For databases, probes should be especially conservative. A database process may still be alive while replication is lagging, recovery is in progress, or maintenance is underway. Liveness should usually confirm that the process is healthy enough to keep running, while readiness should reflect whether the node should accept client traffic. In stateful systems, readiness is often more important than liveness. (kubernetes.io)

For microservices, readiness often needs to include only truly blocking dependencies. A service that depends on a primary backend may need to stay unready when that backend is unavailable, but a service that can degrade gracefully should not withdraw from Service discovery just because a nonessential dependency is missing. The key is matching the readiness rule to what users actually experience. (kubernetes.io)

The best probe pattern is rarely the most complicated one. It is the one that mirrors your operational reality without overfitting to edge cases.

8. Readiness beyond the pod: how probes interact with Services, EndpointSlices, and rolling updates

Readiness is not just a Pod-local concern. It changes how the rest of the cluster sees that Pod. Kubernetes uses the Pod’s Ready condition to determine whether the Pod should be included in Service load balancing, and that readiness state is reflected in EndpointSlices. When readiness fails, the Pod’s IP is removed from the Service’s backend set. (kubernetes.io)

This matters because Services are what most clients actually talk to. If readiness is accurate, users avoid traffic to Pods that are not prepared to serve. If readiness is wrong, the Service may route traffic to Pods that are only partially initialized or temporarily unhealthy. EndpointSlices are the modern mechanism for representing that endpoint membership and readiness state. (kubernetes.io)

Readiness also affects rolling updates. During a Deployment rollout, new Pods should become ready before old Pods are terminated. If readiness is too strict, the rollout may stall because the new ReplicaSet never accumulates enough ready Pods. If readiness is too loose, Kubernetes may route traffic to a Pod before it can actually serve correctly. The rolling update is only as safe as the readiness signal it trusts. (kubernetes.io)

A useful nuance is that Pod deletion itself can also drive endpoint removal. Kubernetes documents that when a Pod is deleted, the corresponding endpoint readiness is set to false so load balancers stop using it for regular traffic. That means readiness probes are important for steady-state traffic control, but termination handling also plays a role during shutdown and draining. (kubernetes.io)

There is also a broader ecosystem shift toward EndpointSlices. Kubernetes v1.33 officially deprecated the Endpoints API in favor of EndpointSlices, reinforcing that readiness is now represented in a more scalable endpoint model. For operators, that means understanding probes is increasingly tied to understanding endpoint conditions. (kubernetes.io)

In practice, readiness is the bridge between your application and the cluster’s routing layer. If that bridge is solid, rolling updates are smoother, traffic is safer, and failures are less visible to users.

9. Observability and validation: using events, logs, metrics, and kubectl to verify probe behavior

A probe configuration should never be considered “done” until you have observed it under real conditions. Kubernetes gives you multiple ways to validate probe behavior, and you should use more than one of them. (kubernetes.io)

Start with kubectl describe pod. This will show events that often reveal probe failures, timing issues, or repeated restarts. If a probe is failing, the event stream is usually the quickest way to see whether the issue is connection refusal, timeout, HTTP status mismatch, or startup delay. You can then correlate those events with container logs. While logs live in your application, probe failures frequently have corresponding log signatures in the same time window. (kubernetes.io)

Next, watch the Pod status over time. A Pod that repeatedly flips between ready and not ready usually indicates either a brittle readiness check or a real dependency problem. A container that restarts due to liveness may show a crash loop pattern if the probe is too strict or the app is actually deadlocked. Kubernetes’ probe model makes these states visible, but only if you watch them long enough to see the pattern. (kubernetes.io)

Metrics are the next layer. Probe failure rates, restart counts, and readiness transitions are all useful signals for dashboards and alerts. A sudden rise in readiness failures may indicate a backend incident, while a rise in liveness failures may point to a software regression or a bad rollout. Even if you do not have a dedicated probe dashboard, container restarts and endpoint membership changes are worth tracking. (kubernetes.io)

Finally, test your assumptions. Deliberately slow startup, temporarily block a dependency, or simulate overload in a staging environment. Validate that readiness behaves as expected, that liveness only restarts when it should, and that startup probes truly protect slow initialization. Kubernetes even notes that readiness probes may run more frequently while a Pod is not ready so that traffic can begin as soon as possible; that detail is useful when you are trying to understand “why did it become ready so quickly?” (kubernetes.io)

The rule is simple: if you cannot explain a probe failure from events, logs, and rollout behavior together, you do not yet understand the configuration well enough.

10. Current best practices and trends: startup probes, gRPC health checks, and reducing exec-probe overhead

The current direction in Kubernetes probe design is clear: prefer explicit startup handling, use protocol-native checks when available, and keep probe execution lightweight. Kubernetes documentation strongly supports startup probes for long initialization, native gRPC probing for gRPC services, and caution with liveness to avoid cascading failures. (kubernetes.io)

Startup probes are now a standard best practice for slow starters. Instead of inflating liveness delays and risking false positives, you can let the startup probe own the initialization window. Once it succeeds, liveness and readiness begin as normal. This is cleaner and easier to reason about than trying to stretch one probe to fit every phase of the lifecycle. (kubernetes.io)

gRPC health checks are increasingly attractive for services built around gRPC. They reduce protocol mismatch and let the probe speak the same language as the workload. That said, they should still be simple and stable. The goal is not to recreate full request validation inside the probe; it is to determine whether the service is ready to handle real RPC traffic. (kubernetes.io)

Another trend is reducing exec-probe overhead. Exec probes can be useful, but they are often unnecessary if HTTP or gRPC can express the same signal more directly. Lower overhead means fewer moving parts, lower runtime cost, and less chance of probe behavior diverging from application behavior. In practice, if your app already exposes a lightweight health endpoint, that is usually preferable to shelling out inside the container. (kubernetes.io)

There is also a growing emphasis on dependency-aware readiness and minimal liveness. Rather than trying to encode every operational concern into one probe, modern practice is to separate concerns: keep liveness narrowly focused on unrecoverable process health, let readiness model traffic safety, and let startup absorb initialization complexity. That separation is one of the simplest ways to make Kubernetes more predictable at scale. (kubernetes.io)

In other words, the best probe strategy today is less about cleverness and more about clarity. Clear signals lead to stable routing, stable rollouts, and fewer surprises.

Conclusion

Kubernetes health checks are not just box-ticking configuration. They are part of the control plane logic that decides whether your Pods receive traffic, whether containers restart, and whether rollouts happen safely. The most effective setups use the three probe types for distinct purposes: startup for initialization, liveness for unrecoverable process failures, and readiness for traffic safety. (kubernetes.io)

If you remember only a few principles, make them these: keep liveness cheap, make readiness reflect real serving ability, use startup probes for slow boots, and tune timing to the real behavior of your application rather than the default values. Also remember that readiness affects Services and EndpointSlices directly, so probe design is really traffic management design. (kubernetes.io)

Well-designed probes make systems calmer. Poorly designed probes make systems noisier. The difference is usually not Kubernetes itself, but how thoughtfully the endpoints and thresholds are chosen. If you treat probes as part of your application contract, you will get better rollouts, fewer false restarts, and more reliable user experience.