How to Implement Hot-Reload Configuration in Kubernetes Microservices Without Restarts

Mar 30, 2026

I pushed a configuration change to our production Kubernetes cluster. The ConfigMap updated successfully. I checked the application logs - still using the old config. I waited five minutes. Nothing changed. My microservice was stuck with stale configuration, and the only way to apply the new settings was to restart the pod manually.

This was a fundamental gap in my understanding of how Kubernetes ConfigMaps work. The changes I made to the ConfigMap weren’t propagating to running pods. I needed to figure out how to implement hot-reload configuration without manually restarting services every time.

The Problem: ConfigMap Changes Don’t Reach Running Pods

I was building a Go-based config server that needed to support hot-reload. My Reddit post asked the question: “Do you restart microservices automatically on the change in the config maps?”

The answers revealed three approaches:

Use Stakater Reloader - Automatically restart pods when ConfigMaps change
Use hash annotations - Native Kubernetes pattern for rollout triggers
Implement in-process hot-reload - Watch files and reload without pod restart

Each approach has trade-offs. Let me walk through what I learned implementing each one.

Understanding Why ConfigMaps Don’t Auto-Update

First, I needed to understand the root cause. Kubernetes ConfigMaps have two mounting modes:

Volume-mounted ConfigMaps:

ConfigMap --kubelet updates--> /etc/config/app.yaml
                                        |
                                        v
                            Application reads file
                            (but doesn't know it changed)

When you update a ConfigMap mounted as a volume, kubelet eventually updates the file (within ~1 minute). But your application has no idea the file changed unless it actively watches for changes.

Environment variable ConfigMaps:

ConfigMap --injected into--> Pod spec
                                  |
                                  v
                    Environment variables (immutable)

Environment variables are set at pod creation time. Changing the ConfigMap does nothing until the pod restarts.

This is intentional Kubernetes design. Immutable infrastructure advocates for explicit rollouts, not automatic config propagation.

Approach 1: Stakater Reloader (Restart-Based, Simplest Solution)

The most common answer I received was: “With stakater reloader. Yes.”

Stakater Reloader watches for ConfigMap and Secret changes, then triggers rolling updates for associated Deployments, StatefulSets, and DaemonSets.

Installation

I installed it with a single command:

kubectl apply -k github.com/stakater/Reloader/deployments/kubernetes

Or with Helm:

helm repo add stakater https://stakater.github.io/stakater-charts
helm repo update
helm install reloader stakater/reloader --namespace reloader --create-namespace

Auto Mode: Reload on Any ConfigMap Change

The simplest approach is auto mode. Any ConfigMap or Secret referenced by the deployment triggers a rollout:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  annotations:
    reloader.stakater.com/auto: "true"
spec:
  template:
    spec:
      containers:
        - name: app
          image: my-app:latest
          env:
            - name: APP_CONFIG
              valueFrom:
                configMapKeyRef:
                  name: my-config  # Changes trigger rollout
                  key: config.yaml

When my-config changes, Reloader updates an annotation on the deployment, triggering a rolling restart.

Search Mode: Selective Reloading

For more control, I used search mode. Only ConfigMaps with matching annotations trigger rollouts:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  annotations:
    reloader.stakater.com/search: "true"
spec:
  template:
    spec:
      containers:
        - name: app
          volumeMounts:
            - name: config
              mountPath: /etc/config
      volumes:
        - name: config
          configMap:
            name: my-config

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config
  annotations:
    reloader.stakater.com/match: "true"
data:
  app.conf: |
    key=value

Now only ConfigMaps with reloader.stakater.com/match: "true" trigger rollouts for this deployment.

How Reloader Works Under the Hood

1. Reloader watches Kubernetes API for ConfigMap/Secret changes
                    |
                    v
2. Finds Deployments/StatefulSets/DaemonSets referencing changed resource
                    |
                    v
3. Updates annotation: reloader.stakater.com/last-reload: <timestamp>
                    |
                    v
4. Kubernetes detects annotation change and triggers rolling update

This approach works for any language and requires zero code changes. The trade-off is brief downtime during the rolling update.

Approach 2: Hash Annotation (Restart-Based, Native Kubernetes)

I also explored a native solution without installing additional controllers. The pattern is simple: include a hash of the ConfigMap content in the deployment annotation.

Helm Implementation

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}
spec:
  template:
    metadata:
      annotations:
        # Pod restarts when ConfigMap content changes
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
    spec:
      containers:
        - name: app
          volumeMounts:
            - name: config
              mountPath: /etc/config
      volumes:
        - name: config
          configMap:
            name: {{ .Release.Name }}-config

When the ConfigMap content changes, the SHA256 hash changes, which changes the deployment annotation. Kubernetes detects the change and triggers a rolling update.

Kustomize Implementation

Kustomize makes this even easier with configMapGenerator:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

configMapGenerator:
  - name: my-config
    files:
      - config.yaml

resources:
  - deployment.yaml

Kustomize automatically:

Adds a hash suffix to the ConfigMap name (e.g., my-config-7k9m2t)
Updates all references to use the new name
When config content changes, a new hash is generated, triggering rollout

Pros and Cons

Pros:

Native Kubernetes, no extra tools
GitOps-native (works with ArgoCD, Flux)
Simple implementation

Cons:

Pod restart required
Must manage hash computation
Less automated than Stakater Reloader

Approach 3: In-Process Hot-Reload with fsnotify (Zero-Downtime)

For applications requiring instant config updates without restarts, I implemented file watching with fsnotify. This is the approach my config-server-go uses.

Understanding Kubernetes ConfigMap Volume Updates

ConfigMap update
       |
       v
kubelet detects change (~1 min)
       |
       v
Updates mounted file in pod
       |
       v
fsnotify detects WRITE event
       |
       v
Application reloads config in-process
       |
       v
Zero downtime, instant update

Go Implementation with fsnotify

Here’s my implementation for a hot-reload config manager:

package main

import (
    "log"
    "sync"

    "github.com/fsnotify/fsnotify"
    "gopkg.in/yaml.v3"
    "os"
)

type Config struct {
    Database DatabaseConfig `yaml:"database"`
    Features FeatureFlags   `yaml:"features"`
}

type DatabaseConfig struct {
    Host     string `yaml:"host"`
    Port     int    `yaml:"port"`
    Database string `yaml:"database"`
}

type FeatureFlags struct {
    EnableNewUI   bool `yaml:"enableNewUI"`
    MaxConnection int  `yaml:"maxConnection"`
}

type ConfigManager struct {
    config     Config
    configPath string
    mu         sync.RWMutex
    watchers   []chan Config
}

func NewConfigManager(path string) (*ConfigManager, error) {
    cm := &ConfigManager{
        configPath: path,
        watchers:   make([]chan Config, 0),
    }

    if err := cm.load(); err != nil {
        return nil, err
    }

    go cm.watch()

    return cm, nil
}

func (cm *ConfigManager) load() error {
    data, err := os.ReadFile(cm.configPath)
    if err != nil {
        return err
    }

    cm.mu.Lock()
    defer cm.mu.Unlock()

    return yaml.Unmarshal(data, &cm.config)
}

func (cm *ConfigManager) watch() {
    watcher, err := fsnotify.NewWatcher()
    if err != nil {
        log.Printf("Failed to create watcher: %v", err)
        return
    }
    defer watcher.Close()

    // Watch the config file
    if err := watcher.Add(cm.configPath); err != nil {
        log.Printf("Failed to watch config file: %v", err)
        return
    }

    log.Printf("Watching config file: %s", cm.configPath)

    for {
        select {
        case event, ok := <-watcher.Events:
            if !ok {
                return
            }

            // Handle write events
            if event.Op&fsnotify.Write == fsnotify.Write {
                log.Println("Config file modified, reloading...")

                if err := cm.load(); err != nil {
                    log.Printf("Error reloading config: %v", err)
                } else {
                    log.Println("Config reloaded successfully")
                    cm.notifyWatchers()
                }
            }

        case err, ok := <-watcher.Errors:
            if !ok {
                return
            }
            log.Printf("Watcher error: %v", err)
        }
    }
}

func (cm *ConfigManager) notifyWatchers() {
    cm.mu.RLock()
    config := cm.config
    cm.mu.RUnlock()

    for _, ch := range cm.watchers {
        select {
        case ch <- config:
        default:
            log.Println("Watcher channel full, skipping notification")
        }
    }
}

func (cm *ConfigManager) Subscribe() <-chan Config {
    cm.mu.Lock()
    defer cm.mu.Unlock()

    ch := make(chan Config, 10)
    cm.watchers = append(cm.watchers, ch)

    return ch
}

func (cm *ConfigManager) Get() Config {
    cm.mu.RLock()
    defer cm.mu.RUnlock()
    return cm.config
}

Using the Config Manager in Your Application

package main

import (
    "context"
    "log"
    "net/http"
    "time"
)

func main() {
    // Initialize config manager with hot-reload
    cm, err := NewConfigManager("/etc/config/app.yaml")
    if err != nil {
        log.Fatalf("Failed to load config: %v", err)
    }

    // Subscribe to config changes
    configCh := cm.Subscribe()

    // Handle config changes in background
    go func() {
        for newConfig := range configCh {
            log.Printf("Config updated: %+v", newConfig)

            // Reinitialize connections, update feature flags, etc.
            // For example, update database connection pool
            updateDatabasePool(newConfig.Database)

            // Update feature flags
            updateFeatureFlags(newConfig.Features)
        }
    }()

    // Start HTTP server
    http.HandleFunc("/config", func(w http.ResponseWriter, r *http.Request) {
        config := cm.Get()
        // Return current config
    })

    log.Println("Server starting on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

func updateDatabasePool(config DatabaseConfig) {
    log.Printf("Updating database pool: %s:%d", config.Host, config.Port)
    // Reinitialize connection pool with new config
}

func updateFeatureFlags(flags FeatureFlags) {
    log.Printf("Updating feature flags: EnableNewUI=%v", flags.EnableNewUI)
    // Update in-memory feature flags
}

Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: config-server
spec:
  template:
    spec:
      containers:
        - name: app
          image: config-server:latest
          volumeMounts:
            - name: config
              mountPath: /etc/config
              readOnly: true
      volumes:
        - name: config
          configMap:
            name: app-config

When you update app-config, kubelet updates the mounted file, fsnotify detects the change, and your application reloads the config in-process - all without a pod restart.

Java/Spring Boot Alternative: WatchService

For Java applications, I implemented a similar pattern using WatchService:

import java.io.IOException;
import java.nio.file.*;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class ConfigWatcher {

    private final WatchService watchService;
    private final Path configDir;
    private final ConfigReloadHandler reloadHandler;
    private final ExecutorService executor;

    public ConfigWatcher(String configPath, ConfigReloadHandler reloadHandler) throws IOException {
        this.watchService = FileSystems.getDefault().newWatchService();
        this.configDir = Paths.get(configPath).getParent();
        this.reloadHandler = reloadHandler;
        this.executor = Executors.newSingleThreadExecutor();

        this.configDir.register(
            watchService,
            StandardWatchEventKinds.ENTRY_MODIFY
        );
    }

    public void start() {
        executor.submit(() -> {
            while (!Thread.currentThread().isInterrupted()) {
                try {
                    WatchKey key = watchService.take();

                    for (WatchEvent<?> event : key.pollEvents()) {
                        if (event.kind() == StandardWatchEventKinds.ENTRY_MODIFY) {
                            Path changedFile = (Path) event.context();
                            if (changedFile.toString().endsWith(".yaml") ||
                                changedFile.toString().endsWith(".properties")) {
                                System.out.println("Config file modified: " + changedFile);
                                reloadHandler.onConfigReload();
                            }
                        }
                    }

                    key.reset();
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                    break;
                }
            }
        });
    }

    public void stop() throws IOException {
        executor.shutdown();
        watchService.close();
    }

    @FunctionalInterface
    public interface ConfigReloadHandler {
        void onConfigReload();
    }
}

Approach 4: Spring Cloud Config Refresh (For Spring Boot)

If you’re already using Spring Cloud Config, the /actuator/refresh endpoint provides built-in hot-reload capability.

Setup

Add the dependencies:

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-config</artifactId>
    </dependency>
</dependencies>

Enable the refresh endpoint:

management:
  endpoints:
    web:
      exposure:
        include: refresh,health,info

Use @RefreshScope on beans that should reload:

@RestController
@RefreshScope
public class FeatureController {

    @Value("${feature.enabled:false}")
    private boolean featureEnabled;

    @Value("${feature.max-connections:100}")
    private int maxConnections;

    @GetMapping("/feature")
    public Map<String, Object> getFeature() {
        return Map.of(
            "enabled", featureEnabled,
            "maxConnections", maxConnections
        );
    }
}

Trigger Refresh

When the ConfigMap or config server is updated:

curl -X POST http://my-service/actuator/refresh

Spring Cloud refreshes all @RefreshScope beans with new values. No pod restart required.

Combining with Kubernetes ConfigMaps

For Spring Boot apps on Kubernetes, use Spring Cloud Kubernetes Config:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-kubernetes-fabric8-config</artifactId>
</dependency>

spring:
  application:
    name: my-service
  cloud:
    kubernetes:
      config:
        name: my-service-config
        namespace: default
      reload:
        enabled: true
        mode: event

This watches ConfigMaps and triggers refresh automatically when they change.

Comparison Matrix

After implementing all four approaches, I created this comparison:

Aspect	Stakater Reloader	Hash Annotation	fsnotify	Spring Cloud Refresh
Restart Required	Yes (rolling)	Yes (rolling)	No	No
Downtime	Brief	Brief	None	None
Code Changes	None	None	Required	Minimal
Language Agnostic	Yes	Yes	No	No (Spring only)
Complexity	Low	Medium	High	Medium
GitOps Compatible	Yes	Yes	Yes	Yes
Instant Update	No	No	Yes (~1 min)	Yes
Additional Components	Reloader controller	None	None	Config Server (optional)

Decision Guide: Which Approach to Use

Use Stakater Reloader when:

You want the simplest solution
Brief downtime during rollout is acceptable
You’re using GitOps (ArgoCD, Flux)
You can’t modify application code
You need language-agnostic solution

Use Hash Annotation when:

You prefer native Kubernetes solutions
You’re already using Helm/Kustomize
You don’t want additional controllers
Brief downtime is acceptable

Use In-Process Hot-Reload (fsnotify) when:

Zero downtime is critical
You can modify application code
You need instant config updates
You need multiple config versions with runtime switching

Use Spring Cloud Refresh when:

You’re already using Spring Boot
You have Spring Cloud Config Server deployed
You need selective bean refresh
You want declarative @RefreshScope

Common Mistakes I Made

Mistake 1: Expecting ConfigMap changes to auto-propagate

I assumed updating a ConfigMap would automatically reach running pods. It doesn’t - that’s a fundamental Kubernetes design choice.

Fix: Implement one of the hot-reload approaches above.

Mistake 2: Mounting ConfigMap as environment variable instead of volume

Environment variables are set at pod creation and can’t be updated without restart.

env:
  - name: DATABASE_URL
    valueFrom:
      configMapKeyRef:
        name: app-config
        key: database-url

volumeMounts:
  - name: config
    mountPath: /etc/config
volumes:
  - name: config
    configMap:
      name: app-config

Fix: Use volume mounts if you need hot-reload capability.

Mistake 3: Not handling ConfigMap update latency

Kubelet updates mounted ConfigMap files within approximately 1 minute. My tests showed anywhere from 10 seconds to 90 seconds.

Fix: Account for this delay in your hot-reload implementation. Add logging to track when updates occur.

Mistake 4: Forgetting to add Reloader annotations

I installed Reloader but forgot the annotations. Pods didn’t restart on config changes.

Fix: Always add reloader.stakater.com/auto: "true" to deployments.

Mistake 5: Over-engineering for non-critical configs

Not every config needs hot-reload. Some configs (like database URLs) should require explicit pod restart for safety.

Fix: Be selective about which configs need hot-reload. Use immutable ConfigMaps for critical configs.

ConfigMap Update Behavior

When you update a ConfigMap, several things happen:

API server updates the ConfigMap object
kubelet periodically syncs with API server (default: 1 minute)
For volume-mounted ConfigMaps, kubelet updates the mounted files
For environment variable ConfigMaps, nothing happens until pod restart

Immutable ConfigMaps

Kubernetes 1.21+ supports immutable ConfigMaps:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
immutable: true
data:
  config.yaml: |
    key: value

Immutable ConfigMaps can’t be changed after creation, preventing accidental updates. Use them for configs that should require explicit rollout.

fsnotify Platform Support

fsnotify uses different mechanisms on different platforms:

Linux: inotify
macOS: FSEvents
Windows: ReadDirectoryChangesW

All support the same API, but behavior may vary slightly.

Rolling Update Strategy

When using restart-based approaches, configure your deployment for smooth rollouts:

apiVersion: apps/v1
kind: Deployment
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0

This ensures zero pods are unavailable during rollout.

Final Thoughts

The Reddit discussion I started had a clear winner for most cases: Stakater Reloader. It’s the simplest solution that works for any language and integrates seamlessly with GitOps workflows.

However, for my Go config server, I implemented in-process hot-reload with fsnotify because I needed:

Zero downtime
Instant config updates
Multiple config versions with runtime switching

The choice depends on your requirements. For most microservices, brief downtime during rolling update is acceptable. For critical services requiring zero downtime, invest in in-process hot-reload.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

👨‍💻 Stakater Reloader GitHub
👨‍💻 fsnotify GitHub
👨‍💻 Spring Cloud Config Refresh
👨‍💻 Reddit Discussion: ConfigMap Hot Reload

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!