Creating CloudWatch Log Groups for Lambdas in Cloudformation and the simple reason why

Let’s start with the cloudformation yaml first and dive in to the reasons why this simple resource should not be ignored for your Lambdas.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Comms Engine Stack

Globals:
  Function:
    Runtime: python3.9
    Timeout: 300
    
Resources:

	SimpleLambdaFunction:
        Type: AWS::Serverless::Function
        Properties:
          CodeUri: ./simple-lambda/
          Handler: simple-lambda.lambda_handler
          FunctionName: simple-lambda

	SimpleLambdaLogGroup:
        Type: AWS::Logs::LogGroup
        Properties:
          LogGroupName: !Sub "/aws/lambda/${SimpleLambdaFunction}"
          RetentionInDays: 3

Simple enough.

Now even if you don’t include the Log Groups for all your Lambdas individually – it doesn’t stop anything because AWS will create one for you. But the reason you should at least consider doing so is right there in the last property:

RetentionInDays: 3

Right, AWS created log groups never expire by default and that may not be something you want or need.

Now this isn’t a slam dunk decision though, and there is an obvious cost to following this practice – the most obvious disadvantage is making a cloudformation yaml even more verbose than they already tend to get.

But it is something that should at least be considered as a best practice before consciously deciding against because the decision is usually made easier if we follow the most important best practice for CloudWatch logs in my opinion – never leave them in CloudWatch.

They are a pain to work with directly and can get impossible if your serverless stacks generate large amounts of logs. Ship them out some where else where they can be indexed and collated and be made searchable with monitoring and alerts and all that good stuff, like Elastic/OpenSearch or some other log collation provider.

If you still don’t want to add the little extra verbosity to your stacks, go with a solution that at least updates the retention values for the AWS defaults whenever a new log group is created.

I personally have gotten used to Cloudformation verbosity and like to use this pattern because I can vary the retention period depending on how critical each Lambda is and in case something gets messed up with the Log collation job itself, and yes I have had that happen to me on occasion.