awslogs

You can install awslogs using pip or brew.

For most users:

$ pip install awslogs

If you are on OSX El Capitan, use:

$ pip install awslogs --ignore-installed six

Alternatively, using Homebrew:

$ brew install awslogs

Credentials

awslogs can use credentials in several ways:

• AWS CLI Profiles: If you have aws-cli configured, awslogs will use those credentials. Use --profile [PROFILE_NAME] or set the AWS_PROFILE environment variable to switch profiles.
• Environment Variables: Use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (e.g., via envdir).
• Direct Flags: Use --aws-access-key-id and --aws-secret-access-key.

Endpoints

To use third-party endpoints like LocalStack or FakeS3, use the --aws-endpoint-url flag or set the AWS_ENDPOINT_URL environment variable.

To ensure the code is working correctly before submitting changes, use tox to run the test suite across all locally available Python versions. If you see ERROR: pyXY: InterpreterNotFound: pythonX.Y errors, you can ignore them as long as the tests pass in at least one available Python version.

$ pip install -U tox
$ tox

To function correctly, awslogs requires permissions equivalent to the CloudWatchLogsReadOnlyAccess managed policy. Specifically, it needs:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "logs:Describe*",
                "logs:Get*",
                "logs:List*",
                "logs:StartQuery",
                "logs:StopQuery",
                "logs:TestMetricFilter",
                "logs:FilterLogEvents"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

You can filter events using the --start (-s) and --end (-e) options. awslogs uses dateutil for parsing, allowing human-friendly strings.

Relative time formats:

• Minutes: --start='2m', --start='1 minute', --start='5 minutes'
• Hours: --start='2h', --start='1 hour', --start='5 hours'
• Days: --start='2d', --start='1 day', --start='5 days'
• Weeks: --start='2w', --start='1 week', --start='5 weeks'

Specific date formats:

• --start='23/1/2015 12:00'
• --start='1/1/2015'
• --start='Sat Oct 11 17:13:46 UTC 2003'

Use --filter-pattern to retrieve only logs that match a specific CloudWatch Logs Filter pattern. This reduces the amount of data downloaded.

Example: Retrieve only REPORT events from a Lambda stream:

$ awslogs get my_lambda_group --filter-pattern="[r=REPORT,...]"

Use the following commands to explore your available CloudWatch logs:

• List all existing log groups: awslogs groups
• List all streams within a specific group: awslogs streams GROUP

Note: You must provide a valid AWS region using the --aws-region flag or the AWS_REGION environment variable.

awslogs groups
awslogs streams /var/log/syslog

The get command retrieves logs from a group. You can specify a STREAM_EXPRESSION which can be a regular expression or the wildcard ALL to aggregate all streams in the group.

Example: Get all logs from the /var/log/syslog group from the last day:

$ awslogs get /var/log/syslog ALL -s1d

To watch logs in pseudo-realtime as they are created:

$ awslogs get /var/log/syslog ALL --watch

Similar to the AWS CLI, you can use the --query option to filter JSON log lines and extract specific fields.

Example: Display only the message field from each JSON log line:

$ awslogs get my_lambda_group --query=message