Teardown

Delete the stack, then the resources it deliberately leaves behind.

Deleting the stack removes the service, load balancer, network and stack machinery. It deliberately leaves your data and keys behind, and AWS services create a few resources of their own that the stack never owned. This page covers all of them.

Decide first whether to keep the data. To be able to restore it later, keep the database's final snapshot and the KMS key, the data-encryption-key secret and the master secret. A snapshot is unreadable without the key, and its encrypted fields are unreadable without the secret. Otherwise, remove everything below.

Below, <stack> is the stack name and <id> the short id (the first block of the DeploymentId output). Run everything in the stack's Region.

1. Save the stack's resource list

Do this before deleting anything. It's how you find the retained resources afterwards, by their physical ids rather than by guessing from names or tags:

aws cloudformation describe-stacks --stack-name <stack> --query 'Stacks[0].Outputs' > tracemill-outputs.json
aws cloudformation list-stack-resources --stack-name <stack> > tracemill-resources.json

2. Delete old database instances from a restore

A tracemill-<id>-db-old instance left by a restore still uses the stack's subnet, parameter and security groups, so the stack delete in step 4 fails while it exists. Delete it first, deciding whether to keep a final snapshot of it:

aws rds modify-db-instance --db-instance-identifier tracemill-<id>-db-old \
  --no-deletion-protection --apply-immediately
aws rds delete-db-instance --db-instance-identifier tracemill-<id>-db-old \
  --skip-final-snapshot   # or --final-db-snapshot-identifier <name>
aws rds wait db-instance-deleted --db-instance-identifier tracemill-<id>-db-old

3. Prepare the database

If the stack created the database, turn off its deletion protection:

aws rds modify-db-instance --db-instance-identifier tracemill-<id>-db \
  --no-deletion-protection --apply-immediately

If you launched with an existing database, remove the rule you added to its security group for the stack's server security group (ServerSecurityGroupId in tracemill-outputs.json). While that rule exists, the stack can't delete its security group and the stack delete fails:

aws ec2 revoke-security-group-ingress --group-id <your database security group> \
  --protocol tcp --port 5432 --source-group <ServerSecurityGroupId>

4. Delete the stack

aws cloudformation delete-stack --stack-name <stack>
aws cloudformation wait stack-delete-complete --stack-name <stack>

About 13 to 15 minutes in our tests. CloudFormation snapshots the database before deleting it; that final snapshot is named <stack>-snapshot-db-<suffix>, not tracemill-<id>-..., so a search by the tracemill- prefix misses it.

5. Remove the retained resources

These are kept on delete by design. Take each physical id from tracemill-resources.json.

Logical idWhatHow to remove
WorkspaceBucketS3 bucket tracemill-<id>-workspace-<account>Empty it, then aws s3 rb s3://<bucket> --force
ServerLogGroup, TransitionLogGroup, StackLambdaLogGroupLog groups under /tracemill/<stack>-<id>/aws logs delete-log-group --log-group-name <name>
DbMasterSecret, DbAppSecret, DbResolveSecret, DbGrantOperatorSecret, SessionEncKeySecret, DataEncKeySecretSecrets under tracemill/<stack>-<id>/ (the first and last only if the stack created them)aws secretsmanager delete-secret --secret-id <arn> (add --force-delete-without-recovery to skip the recovery window)
UserPoolCognito user pool <stack>-users, with deletion protection onTurn deletion protection off in the Cognito console (or aws cognito-idp update-user-pool --user-pool-id <id> --deletion-protection INACTIVE), then aws cognito-idp delete-user-pool --user-pool-id <id>
DbKeyKMS key for the created databaseLast; see step 7

6. Remove what AWS created outside the stack

None of these are stack resources, so deleting the stack leaves them all:

  • RDS log group /aws/rds/instance/tracemill-<id>-db/postgresql, and one per -db-old instance deleted in step 2.
  • Container Insights log group /aws/ecs/containerinsights/<stack>/performance.
  • Transition snapshots tracemill-<id>-*, taken before each migration-crossing update.
  • The final snapshot <stack>-snapshot-db-<suffix> from step 4.
  • SSM parameters under /tracemill/<stack>-<id>/transition/.
aws logs delete-log-group --log-group-name /aws/rds/instance/tracemill-<id>-db/postgresql
aws logs delete-log-group --log-group-name /aws/ecs/containerinsights/<stack>/performance

aws rds describe-db-snapshots --snapshot-type manual \
  --query "DBSnapshots[?starts_with(DBSnapshotIdentifier, 'tracemill-<id>-') || starts_with(DBSnapshotIdentifier, '<stack>-snapshot-db-')].DBSnapshotIdentifier"
aws rds delete-db-snapshot --db-snapshot-identifier <snapshot>   # for each

aws ssm get-parameters-by-path --path /tracemill/<stack>-<id>/transition --recursive \
  --query 'Parameters[].Name' --output text \
  | xargs -n 10 aws ssm delete-parameters --names

Check each snapshot name before deleting it: <stack>-snapshot-db- can match another stack whose name starts the same way.

Also remove what you created yourself: the DNS record for the console hostname, the app in your identity provider, the ACM certificate if nothing else uses it, and the auto-apply role if you made one.

7. Schedule the KMS key for deletion

Do this last, and only for the key in tracemill-resources.json under DbKey. Never pick a key by its tags or alias: CloudFormation doesn't tag the keys it creates, and another key could look similar.

A key can only go once nothing encrypted with it remains: no database instance, snapshot or automated backup. Check:

KEY=<DbKey physical id>
KEY_ARN=$(aws kms describe-key --key-id $KEY --query KeyMetadata.Arn --output text)
aws rds describe-db-instances --query "DBInstances[?KmsKeyId=='$KEY_ARN'].DBInstanceIdentifier"
aws rds describe-db-snapshots --query "DBSnapshots[?KmsKeyId=='$KEY_ARN'].DBSnapshotIdentifier"
aws rds describe-db-instance-automated-backups --query "DBInstanceAutomatedBackups[?KmsKeyId=='$KEY_ARN'].DBInstanceIdentifier"

When all three are empty:

aws kms schedule-key-deletion --key-id $KEY --pending-window-in-days 7

The key stays recoverable until the waiting period ends.

If you launched with your own VPC or database

The stack never deletes what it didn't create. An existing VPC, subnets and database stay as they are; remove the database's ingress rule before the stack delete (step 3), and decide what to do with the tracemill database inside the instance.