I have an AWS SQS queue that I'm going to setup with a Lambda function trigger to run the Lambda function for every item that gets added to the queue to do some processing work.
One step of processing is going to be hitting an API endpoint to get some data back for every item added to the queue, then storing that away in an DynamoDB table. In order to manage costs, and stay in control of this process I'm wanting to throttle the amount of times this Lambda function gets called in a certain period of time.
For example, I want this function to be run a maximum of 100 times per day, in order to not overwhelm the DynamoDB table capacity or the API endpoint. It would also be nice to throttle it, to where it will only have a maximum of 5 concurrent actions being run at once, with a 1 second delay between running again. This type of control would allow me to directly map the function to the DynamoDB table limits to ensure I'm not going over capacity, and to I comply with any API rate limits.
I have looked into AWS Lambda Managing Concurrency. Specifically the "Function Level Concurrent Execution Limit" section. But this section doesn't seem to address the 100 times per day limit, or the 1 second delay between running the next function.
I also know that I could just limit it on the other side, by limiting the number of items I put in the queue at a time (or per day). But because of how I'm planning this system, that would add a lot of complexity that I would love to try to avoid.
Is there a way using AWS SQS and Lambda to achieve this and limit these Lambda functions?