Preventing multiple instances of the same job from running simultaneously is important for scheduled tasks, deployments, database processing, backups, and other workloads where duplicate execution can cause race conditions or inconsistent results.
The best approach depends on where the job is running.
Use Concurrency Controls in Kubernetes
For a Kubernetes CronJob, the simplest solution is concurrencyPolicy: Forbid. Kubernetes allows concurrent executions by default, but Forbid skips a new execution when the previous Job is still running. Replace can be used when the running Job should be replaced by the newer execution.
apiVersion: batch/v1
kind: CronJob
metadata:
name: database-backup
spec:
schedule: "0 * * * *"
concurrencyPolicy: Forbid
jobTemplate:
spec:
template:
spec:
containers:
- name: backup
image: backup-job:latest
restartPolicy: Never
This is usually the cleanest solution when the workload is specifically a Kubernetes CronJob.
Use Distributed Locks for Multiple Application Instances
If a scheduled task runs inside multiple application replicas, every replica may attempt to execute it. In this situation, use a distributed lock backed by a shared system such as Redis or a database.
The basic pattern is:
Acquire Lock → Check Ownership → Execute Job → Release Lock
If another instance already owns the lock, the second instance should skip or wait rather than execute the same job. Distributed locking systems are specifically designed to provide mutual exclusion across processes.
Libraries such as ShedLock can also coordinate scheduled tasks across multiple application nodes using external stores such as JDBC databases, Redis, MongoDB, or Hazelcast.
Use CI/CD Concurrency Controls
For CI/CD pipelines, use the concurrency or deployment-locking mechanism provided by the platform. For example, a production deployment should normally prevent two workflows from modifying the same environment simultaneously.
This is especially important when deployments involve database migrations, infrastructure changes, or shared environments.
Make Jobs Idempotent
Concurrency control should not be the only protection. Jobs should also be designed to be idempotent, so that retrying the same operation does not create duplicate or corrupted results.
For example, instead of blindly inserting a record every time a job runs, use a unique business key or idempotency key so that repeated execution produces the same intended result.
Handle Lock Failures Carefully
A production lock should have a safe expiration or lease mechanism so that a crashed worker does not permanently block the job. Redis documentation, for example, describes locks using an expiration time and unique ownership value so that a lock can eventually be released and cannot simply be removed by an unrelated process.
Monitoring should also track:
- Job currently running
- Lock acquisition failures
- Job duration
- Lock expiration
- Failed executions
- Retry count
- Stale locks
Which Method Should You Choose?
For Kubernetes CronJobs, use concurrencyPolicy: Forbid.
For multiple application replicas, use a distributed lock.
For CI/CD pipelines, use native pipeline concurrency controls.
For critical business operations, combine locking with database constraints, transactions, and idempotency.
Final Thought
The strongest solution is not simply to “block the second job.” A reliable design combines concurrency control, distributed locking where required, idempotent processing, failure recovery, and monitoring. This ensures that even when a worker crashes, a deployment overlaps, or a network failure occurs, the system can recover without creating duplicate or inconsistent work.