KeyError: 'awslogs' … outEvent = str(event

I get the below error:

Traceback (most recent call last): File "/var/task/lambda_function.py", line 22, in lambda_handler outEvent = str(event['awslogs']['data']) KeyError: 'awslogs'

Code in use:

import boto3
import logging
import json
import gzip
import urllib
import time
from io import StringIO

logger = logging.getLogger()
logger.setLevel(logging.INFO)

s3 = boto3.client('s3')

def lambda_handler(event, context):
#set the name of the S3 bucket
bucketS3 = 'test-flowlogs'
folderS3 = 'ArcSight'
prefixS3 = 'AW1Logs_'

#capture the CloudWatch log data
outEvent = str(event['awslogs']['data'])

#decode and unzip the log data
outEvent = gzip.GzipFile(fileobj=StringIO(outEvent.decode('base64','strict'))).read()

#convert the log data from JSON into a dictionary
cleanEvent = json.loads(outEvent)

#create a temp file
tempFile = open('/tmp/file', 'w+')

#Create the S3 file key
key = folderS3 + '/' + prefixS3 + str(int(time.time())) + ".log"

#loop through the events line by line
for t in cleanEvent['logEvents']:

    #Transform the data and store it in the temp file. 
    tempFile.write("CEF:0|AWS CloudWatch|FlowLogs|1.0|src=" + str(t['extractedFields']['srcaddr']) + "|spt=" + str(t['extractedFields']['srcport']) + "|dst=" + str(t['extractedFields']['dstaddr']) + "|dpt=" + str(t['extractedFields']['dstport'])+ "|proto=" + str(t['extractedFields']['protocol'])+ "|start=" + str(t['extractedFields']['start'])+ "|end=" + str(t['extractedFields']['end'])+ "|out=" + str(t['extractedFields']['bytes'])+"\n")

#close the temp file
tempFile.close()    

#write the files to s3
s3Results = s3.upload_file('/tmp/file', bucketS3, key)
print s3Results

I'm using code found here: https://gist.github.com/mlapida/1166b18651a185e21a08#file-flowlogs-to-s3-lambda-py

Screen shot of error: https://www.screencast.com/t/argjftOeu Trying to get CloudWatch Logs to S3 Buckets

Any help is appreciated! Thanks Shane

You are trying to test your function with a event that does not has the object event['awslogs']['data'].

This event is generated when you lambda function is triggered by CloudWatch, like example below:

{
    "awslogs": {
        "data": "H4sIAAAAAAAAAHWPwQqCQBCGX0Xm7EFtK+smZBEUgXoLCdMhFtKV3akI8d0bLYmibvPPN3wz00CJxmQnTO41whwWQRIctmEcB6sQbFC3CjW3XW8kxpOpP+OC22d1Wml1qZkQGtoMsScxaczKN3plG8zlaHIta5KqWsozoTYw3/djzwhpLwivWFGHGpAFe7DL68JlBUk+l7KSN7tCOEJ4M3/qOI49vMHj+zCKdlFqLaU2ZHV2a4Ct/an0/ivdX8oYc1UVX860fQDQiMdxRQEAAA=="
    }
}

If you want to test your function manually, please make sure to go in "Actions", "Configure event test" and in "Sample event template" select the option "CloudWatch Logs".