A repeatable deployment process for an existing Compose fleet, from reviewed updates to recovery when the deployer itself fails.
The Problem
Managing Docker Compose across multiple hosts had gradually become a collection of host-specific deployment procedures. I wanted the reviewability and reproducibility I already had with Kubernetes GitOps while keeping these workloads in Compose. By combining Doco-CD, SOPS with age, Renovate, and Gitea Actions, I moved deployments into a shared Git-driven workflow with explicit targets, encrypted configuration, independently reviewable image updates, and documented recovery paths.
An image update, a configuration edit, and a secret update needed the same traceable path to production. Recording the image artifact, configuration, and deployment scope together made the intended state reviewable before it reached a running service, with a clear path back to the previous configuration when something went wrong.
Scope at the time of writing: two repositories, eight deployment targets, 15 application stacks, and 27 application service definitions.
Design Constraints
I introduced this workflow across two Compose repositories: one organized by application, and another organized by deployment target and stack. Both use the same basic contract:
- Git records deployment intent. Compose files, image pins, encrypted secrets, and target lists belong in version control.
- Updates arrive for review. Renovate proposes changes; a merge to
mainmakes them eligible for deployment. - Deployment is explicit. Adding a directory does not automatically enroll it in the deployment process.
- Recovery remains possible from the host. Initial bootstrap and deployer repair have documented manual procedures.
Architecture Diagram
01 / Propose a change
02 / Review in Git
03 / Apply on each target
Local inputs: deploy key + age key. Local state: decrypted artifacts + application data. Verify through deployment labels, logs, and application checks.
Pull request + CI
CI scans for secrets, checks SOPS metadata and image pins, and parses Compose files. The fleet workflow also cross-checks target lists against stack directories. Human review considers compatibility and rollout scope.
Required checks must be configured in branch protection. The workflow file alone cannot enforce merge policy.
Git Owns the Deployment Intent
Doco-CD runs alongside Docker and polls the repository over SSH. Each agent uses a read-only deploy key scoped to its repository and a pinned Git server host key. Polling lets the deployment path work without an inbound webhook listener; the tradeoff is a delay between a merge and its discovery. This follows Doco-CD’s polling model.
The application-oriented repository uses one explicit deployment list. The fleet repository selects a target-specific list, with each entry naming a Compose project and its working directory. A simplified version looks like this:
# Illustrative .doco-cd.target-a.yml
name: example-service
working_dir: hosts/target-a/stacks/example-serviceThat small file is an operational boundary: a stack can be staged in Git before an agent is told to deploy it. A CI script cross-checks target lists against the directory layout, catches duplicate project names, and checks that listed stacks exist. It also catches an ordinary path mix-up that points a target at another target’s stack.
Target selection is a routing convention, not a security sandbox. The agent still has access to the local Docker socket, so changes to its deployment configuration deserve the same care as other privileged automation.
SOPS: Version Secrets Alongside Configuration
SOPS lets me commit encrypted secret values beside the Compose files that consume them. I use age recipients for the editing workstation, the relevant deployment target, and offline recovery. In the fleet repository, application secret files are scoped to the corresponding target’s key rather than one decryption key shared across every machine.
Compromising one target’s age key does not by itself unlock other targets’ application secrets. The shared workstation and offline recovery recipients are broader trust anchors: either key can decrypt application secrets across targets, so its compromise has a larger potential impact.
Each target gets two creation rules: one for structured config files that encrypts only matching fields, and a fallback that encrypts everything else. Both list the same three recipients:
# Illustrative .sops.yaml (fictional targets, placeholder keys)
keys:
- &workstation age1workstation...
- &offline_backup age1offlinebackup...
- &target_a age1targeta... # /etc/sops/age/keys.txt on target-a
- &target_b age1targetb... # /etc/sops/age/keys.txt on target-b
creation_rules:
# --- target-a ---
# Structured config: encrypt only secret fields so the rest stays diffable
- path_regex: ^hosts/target-a/stacks/.*/(config|.*\.config)\.sops\.ya?ml$
encrypted_regex: ^(password|.*_token|.*_key|.*secret.*)$
key_groups:
- age: [*workstation, *target_a, *offline_backup]
# Everything else (*.sops.env, secrets.sops.yaml, ...): fully encrypted
- path_regex: ^hosts/target-a/.*\.sops\.(env|ya?ml|json)$
key_groups:
- age: [*workstation, *target_a, *offline_backup]
# --- target-b: same two rules with *target_b ---The recipient policy has explicit exceptions for shared configuration. For example, a shared notification secret must be decryptable by the agents that use it. That distinction matters: the isolation claim applies to target-specific application secrets, not every encrypted file in the repository.
Environment-file values are encrypted, while selected structured configuration files encrypt matching sensitive fields so the remaining settings stay readable in a diff. The field rules need maintenance whenever an application introduces a new kind of credential.
Doco-CD uses the locally provisioned age key during deployment. Decrypted files are written into its data volume, so encryption in Git does not remove the need to protect the machine. Root and Docker access can reach that plaintext. The age key stays out of Git, but it is stored on the host, so the same access can read the key itself.
Renovate: Make Image Changes Reviewable
Images are pinned as tag@sha256:digest. The tag helps me recognize the release; the digest identifies the artifact I reviewed. Renovate’s digest pinning and update support turns upstream image changes into a visible diff, including rebuilds that retain the same tag.
Application update branches are scoped by stack in one repository and by target plus stack in the fleet repository. If several targets use the same image, I can merge one target’s update, check its behavior, and then merge the others. That supports a deliberate canary rollout without pretending there is an automatic promotion system behind it.
In the fleet repository, the scoping comes from one package rule that derives the branch prefix, commit suffix, and a target label from the Compose file’s directory:
{
"matchManagers": ["docker-compose"],
"matchFileNames": ["hosts/**"],
"additionalBranchPrefix": "{{{replace '^hosts/([^/]+)/stacks/([^/]+)$' '$1-$2' packageFileDir}}}-",
"commitMessageSuffix": "({{{replace '^hosts/([^/]+)/stacks/([^/]+)$' '$1/$2' packageFileDir}}})",
"addLabels": ["{{{replace '^hosts/([^/]+)/.*$' 'host:$1' packageFileDir}}}"]
}An update to the same image in hosts/target-a/stacks/proxy and hosts/target-b/stacks/proxy becomes two PRs on two branches, each labelled with its target.
Major updates receive an extra label for closer review. Renovate’s own image publishes several releases a day, so after the first few days I moved it, and a dev-branch image rebuilt on every upstream push, to a weekly window. Application updates still open as soon as they are released. The Renovate job runs on a schedule or manual dispatch, rather than on pull requests, keeping its write credentials out of the PR validation job.
Shared bootstrap changes are a wider operation. A merged update to the common deployer definition can reach every agent that manages that definition. Those PRs carry a self-update label and an operational note, rather than inheriting the assumption that an application update affects one target.
Validation Before Merge
Local pre-commit hooks catch common mistakes early. Gitea Actions repeats the checks on pull requests and pushes to main, alongside Compose validation and digest checks. The validation tools are downloaded with pinned checksums, and Compose parsing does not require a running Docker daemon.
Both workflows check out the repository with fetch-depth: 0 and run ./gitleaks git --redact -v .. Gitleaks’s Git mode scans commit patches, including secrets committed earlier and subsequently removed from the current files.
| Check | What it catches | What it cannot prove |
|---|---|---|
| Gitleaks + file rules | Recognized secrets in Git history and tracked plaintext env/key files. | That every possible secret is detectable. |
| SOPS metadata | A SOPS-named file missing its expected metadata marker. | Correct recipients, complete encryption, or successful decryption. |
| Compose parsing | Invalid Compose configuration. | Application health, image compatibility, or a successful data migration. |
| Image pin check | Image declarations missing a digest reference. | Artifact availability or freedom from vulnerabilities. |
| Fleet target check | Missing stacks, duplicate names, and common target/path mistakes. | Host authorization or runtime isolation. |
None of these block a merge unless branch protection requires their statuses. That is a server-side setting; a workflow file alone does not enforce it.
Adopting Existing Stacks
Migration needed to preserve the identity of each existing Compose project. The deployment entry’s name must match the running project name, otherwise adoption can create a second project instead of taking over the first. Containers originally started with docker run need a separate cutover because they do not have that Compose identity.
Each adoption pinned the images already running before introducing later upgrades through Renovate. Databases, caches, and other runtime state stayed in their existing volumes or host paths. After adoption, Compose project listings and Doco-CD’s deployment revision labels provide evidence that the intended project and commit were applied. Application checks still determine whether the service actually works.
A Deploy Key Setting That Did Not Survive the Clone
One concrete migration issue was in the bootstrap instructions. Early checkouts used git -c core.sshCommand=… clone …, which supplied the SSH configuration only for that command. Later manual git pull operations could not rely on inheriting the same deploy-key settings. That matters most when the checkout is needed to repair a broken deployer.
I corrected the bootstrap procedure to use git clone -c core.sshCommand=… …, which persists the setting in the new repository, and documented a repair step for existing checkouts. The saved SSH command selects the deploy key and enforces the pinned host-key checks. This made repeatable authentication part of the recovery procedure, rather than something established only for the first successful clone.
Updating the Deployer, and Recovering It
Doco-CD can manage its own Compose definition, so its image updates follow the same review path. Its self-update mechanism supports a health-gated handover and recovery when a replacement fails to become healthy. Configuration constraints determine whether that handover can overlap two instances or requires a brief restart; zero downtime is not a blanket property of the setup.
The persistent deployer volume carries the repository cache and handover state across updates. Its identity must remain stable. A failed self-update can restore a container, but it cannot reconstruct volume data that was changed or removed.
For an application configuration rollback, the documented path is a new revert commit on main. This preserves an auditable history and gives the agent a new revision to apply. A database migration may still require a separate restore or application-specific recovery procedure.
A subtler failure is a deployer that passes its healthcheck but stops processing deployments. It may be unable to apply its own rollback. The recovery procedure therefore corrects the desired revision in Git first, then reapplies the deployer manually on the affected machine. Otherwise, a repaired agent could immediately readopt the bad configuration.
In the fleet setup, warnings and failures are forwarded through Apprise to a notification service. Successful deployments stay quiet. Notifications are a diagnostic aid rather than a deployment dependency, so their failure must not be mistaken for proof that deployment stopped or succeeded.
What Changed Operationally
Previously, applying a Compose change meant reaching the right host over SSH, finding its local project configuration, and running the update there. The migration moved those local definitions into reviewed repositories and made the normal update process consistent across targets.
A routine image update now starts with a Renovate PR. I review the proposed tag and digest, compatibility notes, CI results, and target scope; merge the change; then verify the applied revision and application behavior. Doco-CD handles fetching the merged configuration and applying Compose on the target. The routine path no longer requires a shell session on each machine to edit the image reference, pull the image, and run Compose manually.
In the first two weeks after adoption, I merged 18 application version updates through Renovate. Most of the remaining PRs were not application churn: 21 were digest-only rebuilds, mostly my own Caddy images being rebuilt during standardization and fanned out per target, and 18 were Renovate’s own CI image, 16 of them before it moved to a weekly schedule. Each agent polls main every 60 seconds, so a merged change is picked up within about a minute. The first unattended deployer upgrade ran on a canary target and completed its handover in about 10 seconds. The same image has also been merged separately for different targets, with each target’s update decided on its own.
The repositories also make the remaining manual responsibilities clear: bootstrap, local key provisioning, host configuration, recovery, and data protection. Git describes the desired application configuration; restoring a machine still requires those other pieces.
What I’d Improve Next
The next useful additions would be stronger validation of SOPS file structure and recipient policy, application-level checks after deployment, and a clearer comparison between desired and applied revisions across targets. I would also make rollout of shared deployer changes more gradual, since their scope is broader than the application update PRs.