Monitoring the health of your canary allows CodeDeploy to make a decision to whether a rollback is needed or not. If any of the CloudWatch Alarms specified gets to ALARM status, CodeDeploy rollsback the deployment automatically.

Introduce an error on purpose

Lets break the Lambda function on purpose so that the alarm gets triggered during deployment. Update the lambda code in sls-app/lambda/ to throw an error on every invocation, like this.

import json
import os

import boto3

ddb = boto3.resource('dynamodb')
table = ddb.Table(os.environ['TABLE_NAME'])
_lambda = boto3.client('lambda')

def handler(event, context):
    # print('request: {}'.format(json.dumps(event)))
    # response = table.update_item(
    #     Key={'path': event['path']},
    #     UpdateExpression='ADD hits :incr',
    #     ExpressionAttributeValues={':incr': 1},
    #     ReturnValues='UPDATED_NEW'
    # )
    # hit_count = int(response['Attributes']['hits'])
    # return {
    #     'statusCode': 200,
    #     'headers': {'Content-Type': 'text/plain'},
    #     'body': f"You've hit {event['path']}: {hit_count} time(s), heheheeeeeee"
    # }
    raise Exception("Error on purpose")

Push the changes

In the terminal, run the following commands from the root directory of your sls-app project.

git add .
git commit -m "Breaking the lambda function on purpose"
git push

Wait for deployment to start

Deployment start

Invoke the Canary

While the deployment is running, you need to generate traffic to the new Lambda function to make it fail and trigger the CloudWatch Alarm. In a real production environment, your users will likely generate organic traffic to the canary function, so you may not need to do this.

In your terminal, run the following command to invoke the Lambda function:

aws lambda invoke --function-name \
$(aws lambda list-functions | jq -r -c '.Functions[] | select( .FunctionName == "CounterFunction").FunctionName'):Prod \

If you get an error that jq command is not installed, you can install by running sudo yum install -y jq

Lambda invoke

There will be a new file response.json created. It contains the response of the lambda invocation. If you open it, you may see the the response of the old Lambda version, or you may see the new one that causes an error.

Remember: During deployment, only 10% of the traffic will be routed to the new version. So, keep on invoking your lambda many times. 1 out of 10 invocations should trigger the new broken lambda, which is what you want to cause a rollback.

Here is a command that invokes your function 20 times in a loop. Feel free to run it in your terminal.

while [ $counter -le 20 ]
    aws lambda invoke --function-name \
    $(aws lambda list-functions | jq -r -c '.Functions[] | select( .FunctionName == "CounterFunction").FunctionName'):Prod \
    sleep 1