Monitor canary health

Canary deployments are considerably more successful if the code is being monitored during the deployment. You can configure CodeDeploy to automatically roll back the deployment if a specified CloudWatch metric has breached the alarm threshold. Common metrics to monitor are Lambda Invocation errors or Invocation Duration (latency), for example.

Install requirements

Add the following CDK CloudWatch library to the sls-app/requirements.txt.

-e .
pytest

aws-cdk.aws_dynamodb
aws-cdk.aws-lambda
aws-cdk.aws_apigateway
aws-cdk.aws_codecommit
aws-cdk.aws_codedeploy
aws-cdk.aws_codebuild
aws-cdk.aws_codepipeline_actions
aws-cdk.aws_cloudwatch
Then run

pip3 install -r requirements.txt

Define a CloudWatch Alarm

Let’s update the sls_app_stack.py to add the alarm for deployment.

from datetime import datetime

from aws_cdk import (
    core,
    aws_lambda,
    aws_dynamodb,
    aws_apigateway,
    aws_codedeploy,
    aws_cloudwatch
)


class SlsAppStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)
        
        ## 1. Define the table that count number of times the path is hitted.
        table = aws_dynamodb.Table(
            self, 
            "counter-table",
            partition_key={
                'name': 'path',
                'type': aws_dynamodb.AttributeType.STRING
            }
        )
        
        ## 2.1. Defines Lambda resource & API-Gateway request handler
        ## All API requests will go to the same function.
        self.lambda_code = aws_lambda.Code.from_cfn_parameters()
        now = datetime.now().strftime('%Y-%m-%d %H:%M:%S')

        handler = aws_lambda.Function(self, "CounterFunction",
            description=f"Generated on {now}",
            function_name='CounterFunction',
            code=self.lambda_code,
            handler='index.handler',
            runtime=aws_lambda.Runtime.PYTHON_3_8
        )
        
        current_version = handler.current_version
        alias = current_version.add_alias('Prod')
        
        ## 2.2. Pass the table name to the handler through an env variable 
        ## and grant the handler read/write permissions on the table.
        table.grant_read_write_data(handler)
        handler.add_environment('TABLE_NAME', table.table_name)
        
        ## 3. Define the API endpoint and associate the handler
        api = aws_apigateway.LambdaRestApi(self, "CounterAPI",
                                           handler=handler)
                                         
        ## 4. Define new Lambda application                                         
        app = aws_codedeploy.LambdaApplication(
            self,
            "ApplicationDeploy"
        )
        
        ## 5. Create LambdaDeploymentGroup for our lambda
        deployment_group = aws_codedeploy.LambdaDeploymentGroup(self,
            'DeploymentGroup',
            alias=alias,
            application=app,
            deployment_config=aws_codedeploy.LambdaDeploymentConfig.CANARY_10_PERCENT_5_MINUTES
        )
        
        metric = aws_cloudwatch.Metric(
            metric_name='Errors',
            namespace='AWS/Lambda',
            # dimensions=dict(Resource=)
        )
        
        alarm = aws_cloudwatch.Alarm(
            self,
            id="alarm",
            metric=metric,
            threshold=0,
            evaluation_periods=2,
            alarm_description="Lambda function canary errors",
            statistic="sum",
            period=core.Duration.minutes(1),
            comparison_operator=aws_cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD
        )
        
        deployment_group.add_alarm(alarm)

Push the changes

In the terminal, run the following commands from the root directory of your sls-app project.

git add .
git commit -m "Added CloudWatch alarm to monitor the canary"
git push