Keycloak at 100% CPU After Deploy: Root Cause and Fix

Jul 15, 20265 min read

The Symptom: Recognizing This CPU Spike

You deploy a containerized Keycloak instance — even the official quay.io/keycloak/keycloak image, unmodified — to a managed container platform, and within seconds CPU utilization climbs to 100% and stays there — regardless of traffic. No requests are being served, no realm imports are running, and nothing in the application logs looks wrong. Health checks pass. HTTP endpoints respond normally.

This pattern shows up most often on:

  • DigitalOcean App Platform
  • Google Cloud Run
  • AWS ECS Fargate
  • Heroku
  • Self-managed or managed Kubernetes clusters without a multicast-aware CNI

Root Cause: Infinispan, JGroups, and the Multicast Problem

The Default: Distributed Caching via Infinispan

By default, Keycloak ships with a distributed cache mode built on Infinispan, an embedded data grid used to replicate session state, authentication sessions, and other transient data across cluster nodes. This is the ispn cache provider, and it's the default unless explicitly overridden — either at build time (kc.sh build --cache=<mode>) or at runtime (KC_CACHE).

Why JGroups Can't Discover Peers

Infinispan's clustering layer relies on JGroups, a toolkit for reliable group communication. JGroups needs a discovery protocol to find other cluster members, and out of the box, Keycloak's default JGroups stack uses UDP-based discovery (PING/MPING), which depends on IP multicast to broadcast "who else is out there?" messages across the network.

Managed container platforms don't expose multicast-capable networking. Their networking layers are software-defined, ephemeral, and built around dynamic scheduling of containers across shared infrastructure — a model that's fundamentally incompatible with IP multicast, which assumes a flat, broadcast-capable network segment. This holds almost universally: PaaS offerings, serverless container runtimes, and even many managed Kubernetes configurations without a multicast-aware CNI all impose the same restriction.

The Result: A Silent, Load-Independent CPU Burn

JGroups doesn't fail gracefully when discovery is unreachable. Instead, it enters an aggressive retry loop — broadcasting discovery packets, waiting for a timeout, retrying, re-forming view negotiations, and repeating indefinitely. Each cycle consumes CPU on packet construction, timeout scheduling, and thread contention, multiplied across the JGroups protocol stack (transport, discovery, failure detection, and flow control protocols all layered on top of one another).

None of this is visible from an application standpoint. A background thread pool is perpetually spinning, trying to solve a problem — finding cluster peers via multicast — that's structurally impossible in this environment. The result is sustained, load-independent CPU burn with no connection to request volume.

Setting KC_CACHE=local eliminates the problem — not by fixing discovery, but by disabling clustering altogether. In local mode, Keycloak uses in-memory, non-replicated caches with no Infinispan clustering, no JGroups stack, and therefore no discovery loop to fail.

Why It Happens After Redeploys, Not on First Boot

First Boot: A Clean, Uncontested Singleton

On the very first deploy, exactly one container exists. When JGroups' discovery protocol fires, it queries whatever peer list mechanism is configured — raw UDP multicast, or more commonly on managed platforms, a DNS-based or static-host discovery protocol that resolves the platform's internal service name to the currently running container IPs. Since this is the first and only container, that resolution returns exactly one address: itself. There's nothing to negotiate and no peer to merge with. JGroups times out once on the discovery request, finds no one else, and installs itself as a singleton cluster view — a cheap, one-shot operation that doesn't manifest as sustained CPU load.

Redeploy or Restart: A Two-Node Race That Never Resolves

A redeploy or restart on most managed platforms is a rolling operation, not an instant swap. The platform starts the new container before fully tearing down the old one, to avoid downtime. For a short window, both the outgoing and incoming containers are alive simultaneously, and depending on how peer discovery is configured, both may be resolvable under the same internal service identity.

This is where things diverge from the first-boot case: JGroups on the new node now believes there is another cluster member to reach. It attempts to establish a channel, negotiate a merged view, and start heartbeat-based failure detection with that peer — but the platform's networking still doesn't permit the actual cache-cluster transport traffic between containers (multicast is blocked, and often the cache port isn't even routed between sibling containers in the first place). The handshake can never complete: the peer is addressable enough to be discovered, but not reachable enough to actually communicate.

The result is a protocol deadlock rather than a clean failure. JGroups cycles through:

  • periodic heartbeat attempts to the discovered peer
  • failure-suspicion timers when heartbeats go unanswered
  • merge-coordinator election attempts to decide which node's view should win
  • state-transfer request/response handshakes that time out without completing

Each sub-protocol retries independently and indefinitely, because there's no terminal "peer confirmed dead, drop it forever" signal — the old container is still technically registered as existing from the discovery layer's point of view, even though it's unreachable at the transport layer. This composite retry storm is considerably more CPU-intensive than the one-shot singleton bootstrap on first boot, which is why the spike reliably appears after a redeploy or restart, but not on the initial one.

How to Fix It: Step-by-Step

  1. Confirm you're hitting this issue. Check CPU metrics with no active traffic — a sustained 100% with zero requests is the signature. Confirm KC_CACHE is unset or set to ispn (the default).
  2. Decide whether you actually need clustering. If you're running a single Keycloak instance, which covers most small and mid-sized deployments, you don't need Infinispan's distributed cache at all.
  3. Single instance → set KC_CACHE=local. Add it as a runtime environment variable, or bake it in at build time with -cache=local. Redeploy and confirm CPU returns to baseline.
  4. Multiple instances / HA → switch to JDBC_PING. Build with -cache-stack=jdbc-ping (or supply a custom cache-ispn.xml), pointing at your existing database. This keeps session replication working without depending on multicast.
  5. Verify after a redeploy, not just first boot. Because the issue specifically reproduces on rolling restarts, don't declare victory after the first boot alone — trigger a redeploy and confirm CPU stays flat afterward too.

The Trade-Offs of KC_CACHE=local

KC_CACHE=local is a valid and common fix, but it comes with real trade-offs:

  • No safe horizontal scaling. Run more than one replica behind a load balancer, and users can be routed to a different instance mid-session — silently logged out, losing SSO state, or hitting inconsistent authorization decisions.
  • Restarts are lossy. A container restart — which managed platforms perform routinely during deploys, scaling events, or health-check failures — wipes all in-memory session state for that instance.

For a single-instance deployment, none of this is a compromise — there's no cluster to form in the first place, so disabling clustering is simply correct.

Need High Availability? Use JDBC_PING Instead

If horizontal scaling or high availability is a requirement, the architecturally correct fix isn't to disable clustering — it's to replace the discovery mechanism with one that doesn't depend on multicast.

Keycloak, via Infinispan/JGroups, supports a JDBC_PING discovery protocol that uses your existing relational database as a shared coordination point: each node writes its cluster address to a database table, and peers discover each other by querying that table instead of broadcasting UDP packets.

This is configured via a custom JGroups stack — --cache-stack=jdbc-ping at build time, or a custom cache-ispn.xml descriptor — and it works reliably on any managed platform, regardless of vendor, because it never depends on network-layer broadcast at all. The cost is a small, constant amount of additional load on your database (heartbeat writes/reads at a configurable interval) in exchange for genuine session replication across nodes.

KC_CACHE Settings: Quick Reference

ScenarioRecommended SettingWhy
Single Keycloak instance, no plans to scale horizontallyKC_CACHE=localNo cluster to form — disables Infinispan/JGroups entirely
Multiple instances, need session replication / HAKC_CACHE=ispn with --cache-stack=jdbc-pingKeeps clustering, swaps multicast discovery for database-backed discovery
Multiple instances, occasional failover session loss acceptable (rare)KC_CACHE=ispn with default stackNot recommended on managed platforms — reproduces the CPU issue

Frequently Asked Questions

Why does my Keycloak container jump to 100% CPU with no traffic?

Keycloak's default Infinispan cluster uses JGroups discovery over multicast; on managed platforms multicast is unavailable, causing discovery to loop and consume CPU.

Will setting `KC_CACHE=local` fix the CPU spike?

Yes — KC_CACHE=local disables Infinispan clustering and JGroups discovery, eliminating the background discovery loop and the CPU burn, but it disables session replication.

Why does the spike appear on redeploys but not on first boot?

On first boot a single node boots as a singleton; during rolling redeploys a new node discovers an unreachable old node, triggering a prolonged merge and handshake loop that consumes CPU.

I need high availability — what's the right fix?

Use a database-backed discovery like JDBC_PING (build with --cache-stack=jdbc-ping) so nodes discover each other via the DB instead of multicast, preserving clustering on managed platforms.

Are there trade-offs to using `KC_CACHE=local`?

Yes — local mode prevents safe horizontal scaling and loses in-memory session state on restarts, so it's best for single-instance deployments only.

Need help stabilizing Keycloak

u11d can review your Keycloak deployment, implement an auth solution from scratch, or support your team with DevOps expertise and best practices.

A person sitting and typing on a laptop keyboard
RELATED POSTS
Maciej Łopalewski
Maciej Łopalewski
Senior Software Engineer

CloudFront Response Headers Policies Explained: Managed Policies, CORS, and the Override Flag

Jul 08, 202615 min read
Article image
Robert Szczepanowski
Robert Szczepanowski
Senior Software Engineer

Faster Geospatial Animations with PMTiles: Multi-Layer MVT Time Series in OpenLayers

Jun 17, 20267 min read
Article image
Paweł Swiridow
Paweł Swiridow
Senior Software Engineer

Argo CD GitLab Authentication: A Complete GitOps-Friendly Setup

Jun 10, 20266 min read
Article image