Published on PyPI · v0.6.1
A library that stops background jobs from running twice or writing stale results after a crash.
Python · Redis · Celery · Hypothesis · GitHub Actions · PyPI Trusted Publishing
63 automated tests · Python 3.10–3.13 in CI
What it is
fencekit is a Python library on PyPI for background jobs that use Redis and Celery. It stops duplicate starts, issues fencing tokens with locks, and gives Django/Postgres a one-statement way to reject stale writes. I extracted it from analysis pipelines on ChessMate after worker crashes produced duplicate work and ambiguous progress rows.
The problem
Celery with late acknowledgements redelivers work when a worker dies. That is the right reliability default. The same batch could start twice and burn CPU. Worse: a paused worker could resume after its lock TTL and overwrite progress a newer worker had already written. Storage side fencing is still your job.
What I built
IdempotencyGuard with owner-checked completion and optional result memoization on redelivery.DistributedLock that increments a fencing token in the same Lua script as acquire.FenceGate.set_if_fresh for Redis and fenced_update for Django rows.Flow
Celery task (redelivered)
IdempotencyGuard.try_begin_or_reclaim
SET NX · reclaim when lock free
DistributedLock.acquire
Lua · lock + fencing token
FenceGate
Redis progress strings
fenced_update
Postgres / Django rows
What we do not claim
Exactly-once delivery. Redlock across Redis failover. Safety when application code writes without presenting the token. Limits are in DESIGN.md.
Status