S3 backup not configured automatically on first boot when enabled at install time
When S3 backup is enabled during CloudFormation stack creation, the backup timer is silently not configured on first boot. Backups never run despite the option being set.
- Published
- May 29, 2026
- Last Updated
- May 29, 2026
- Affected Versions
- Standard tier AMI builds released before 2026-05-29 Enterprise (ARM64) tier AMI builds released before 2026-05-29
- Fixed In
- Pending AMI release
Overview
On Standard and Enterprise tier deployments, customers can enable automated S3 backups by setting the Enable S3 Backup parameter during CloudFormation stack creation. Due to a race condition in the stack provisioning sequence, the SSM Parameter Store entries required by the bootstrap script were not created before the EC2 instance launched. As a result, the bootstrap script read no backup configuration and silently skipped enabling the backup timer.
Affected customers may believe their library data is being backed up automatically when it is not.
Affected Versions
- Standard tier (x86_64) AMI builds released before this patch
- Enterprise tier (ARM64) AMI builds released before this patch
Free and Basic tier stacks are not affected — those tiers configure backup differently.
How to Check if You Are Affected
Connect to your instance via Session Manager (SSM) and run:
systemctl is-active koha-s3-backup.timer
cat /etc/default/koha-s3-backup 2>/dev/null || echo "MISSING"
If the output is inactive and the file is MISSING, your stack is affected and backups have not been running.
You can also verify whether the SSM parameter was ever created:
aws ssm get-parameter \
--name "/koha/YOUR-STACK-NAME/config/enable-s3-backup" \
--region us-east-1 2>&1
Replace YOUR-STACK-NAME with your CloudFormation stack name. If you see ParameterNotFound, the parameter was never written and the fix was not applied when your stack launched.
Remediation
Option 1: Upgrade to Patched AMI (Recommended)
When a patched AMI is available on AWS Marketplace, upgrade your stack. The fix ensures the SSM parameters are created as part of the CloudFormation stack before the EC2 instance is allowed to launch. On first boot after the upgrade, backup will be configured correctly.
See How to Upgrade Koha to a New AMI Version for step-by-step instructions.
Option 2: Apply Interim Mitigation (Manual Fix)
While waiting for the patched AMI, you can manually configure the backup on your running instance:
- Connect to your instance via Session Manager
- Run:
STACK_NAME="YOUR-STACK-NAME"
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
BUCKET="${STACK_NAME}-koha-backup-${ACCOUNT_ID}"
sudo tee /etc/default/koha-s3-backup > /dev/null <<EOF
KOHA_BACKUP_BUCKET=${BUCKET}
KOHA_BACKUP_SCHEDULE=daily
KOHA_BACKUP_TIME=02:00
KOHA_BACKUP_RETENTION=30
EOF
sudo systemctl enable --now koha-s3-backup.timer
systemctl is-active koha-s3-backup.timer
- Verify the timer is
activeand test with a manual backup run:
sudo koha-s3-backup --bucket "${BUCKET}" --yes
Option 3: Contact Support
If you are unsure how to apply the above steps, contact us at [email protected] or use the contact form. We can walk you through the fix remotely.
Timeline
| Date | Event |
|---|---|
| 2026-05-16 | Issue confirmed — test stack with EnableS3Backup=true had no backup timer running |
| 2026-05-29 | Root cause identified — SSM params not pre-created before ASG launches |
| 2026-05-29 | Fix merged (PR #691) — AWS::SSM::Parameter CF resources with DependsOn on ASG |
| 2026-05-29 | Advisory published |
| 2026-05-29 | Patched AMI published to AWS Marketplace |