cdk destroy failed

[thomas-roos]

❌ QemuEmbeddedLinuxPipeline: destroy failed Error: The stack named QemuEmbeddedLinuxPipeline is in a failed state. You may need to delete it from the AWS console : DELETE_FAILED (The following resource(s) failed to delete: [BuildProjectSecurityGroup28D27668]. ): resource sg-0d0cd1c97d4aa94b0 has a dependent object (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: d4c7dcf9-f29d-4033-b483-26b264d779dd; Proxy: null)
at destroyStack (/home/ANT.AMAZON.COM/throos/.local/lib/node_modules/aws-cdk/lib/index.js:471:1792)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async CdkToolkit.destroy (/home/ANT.AMAZON.COM/throos/.local/lib/node_modules/aws-cdk/lib/index.js:474:188097)
at async exec4 (/home/ANT.AMAZON.COM/throos/.local/lib/node_modules/aws-cdk/lib/index.js:529:53102)

The stack named QemuEmbeddedLinuxPipeline is in a failed state. You may need to delete it from the AWS console : DELETE_FAILED (The following resource(s) failed to delete: [BuildProjectSecurityGroup28D27668]. ): resource sg-0d0cd1c97d4aa94b0 has a dependent object (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: d4c7dcf9-f29d-4033-b483-26b264d779dd; Proxy: null)

-> this happes just "sometime"
-> a workaround currently is go the aws console and delete manually the eni blocking the sg deletion

[rpcme]

Analysis and Proposed Fix

Root Cause

The issue occurs due to a race condition during stack deletion. When the CodeBuild project is deleted, its ENIs (Elastic Network Interfaces) are not immediately cleaned up by AWS. CDK attempts to delete the Security Group while ENIs are still attached, causing the DependencyViolation error.

Recommended Solution: Custom Resource for ENI Cleanup

Add a Lambda-backed custom resource that waits for ENIs to detach before allowing Security Group deletion:

const eniCleanupLambda = new lambda.Function(
  this,
  "EmbeddedLinuxENICleanup",
  {
    runtime: lambda.Runtime.PYTHON_3_10,
    handler: "index.handler",
    timeout: cdk.Duration.minutes(5),
    code: lambda.Code.fromInline(`
import boto3
import time
import cfnresponse

ec2 = boto3.client('ec2')

def handler(event, context):
  try:
    if event['RequestType'] == 'Delete':
      sg_id = event['ResourceProperties']['SecurityGroupId']
      # Wait for ENIs to detach
      for i in range(30):  # 5 minutes max
        enis = ec2.describe_network_interfaces(
          Filters=[{'Name': 'group-id', 'Values': [sg_id]}]
        )['NetworkInterfaces']
        if not enis:
          break
        time.sleep(10)
    cfnresponse.send(event, context, cfnresponse.SUCCESS, {})
  except Exception as e:
    cfnresponse.send(event, context, cfnresponse.FAILED, {'Error': str(e)})
    `),
  }
);

eniCleanupLambda.addToRolePolicy(
  new iam.PolicyStatement({
    actions: ['ec2:DescribeNetworkInterfaces'],
    resources: ['*'],
  })
);

const eniCleanup = new cdk.CustomResource(this, 'ENICleanupResource', {
  serviceToken: eniCleanupLambda.functionArn,
  properties: {
    SecurityGroupId: projectSg.securityGroupId,
  },
});

projectSg.node.addDependency(eniCleanup);

Files to Update

• lib/codebuild-embedded-linux.ts
• lib/codepipeline-embedded-linux.ts

This solution properly handles the intermittent nature of the issue by ensuring ENIs are detached before Security Group deletion proceeds.