I used to run [my static blog](https://www.const.no/) on AWS with [[Amazon Simple Storage Service]] and Route 53. Early on I also used S3 (Glacier) as a personal backup solution. I have worked with AWS extensively and know how to write clean [[Terraform]] modules. I have experience setting up [[Infrastructure as code]] in a way that makes the code maintainable and easy to understand. I know a lot about [Cloud Custodian](https://cloudcustodian.io/).
It's possible to route events through [[Amazon CloudWatch]] with the help of [[Amazon EventBridge]]. CloudWatch events can be sent to [[Amazon Kinesis Data Streams]]. One of the few applications with a Kinesis input is [[Benthos]].
```mermaid
flowchart TD
subgraph aws [AWS]
ct[/Amazon CloudTrail\] --> cw((Amazon CloudWatch))
cd[/Amazon GuardDuty\] --> cw
cw --->|No latency| k[(Amazon Kinesis Data Streams)]
cw --->|Latency from file batching| s3[(Amazon Simple Storage Service)]
k --> kdf[Amazon Kinesis Data Firehose]
end
kdf -->|Push| lm
s3 -->|Pull| v[Vector]
v --> lm
k --->|Pull| b[Benthos]
b ---> lm((Log management))
class ct internal-link
class cd internal-link
class cw internal-link
class k internal-link
class b internal-link
class lm internal-link
class v internal-link
class s3 internal-link
class kdf internal-link
class db internal-link
style aws stroke:#f66,stroke-width:2px,stroke-dasharray: 5 5
```
[[Benthos]] stores the latest message sequence consumed in a [[Amazon DynamoDB]] table. [[Vector]] uses a [[Amazon Simple Queue Service]] to keep track of files consumed. [[Amazon S3 Event Notifications]] can be used to send notifications to the [[Amazon Simple Queue Service|SQS]]. A drawback of using S3 is that you have to unpack the events from the `Records` JSON list:
```
[sources.in]
type = "stdin"
[transforms.remap]
type = "remap"
inputs = ["in"]
source = """
. |= object!(parse_json!(.message))
. = .Records
"""
[sinks.out]
type = "console"
encoding.codec = "json"
inputs = ["remap"]
[sinks.out]
type = "console"
encoding.codec = "json"
inputs = ["remap"]
```
See [feat(remap transform): Emitting multiple events](https://github.com/timberio/vector/pull/7267).
Reinventing things that are already solved by AWS services? Perhaps a bad idea. The time you save by using AWS services properly is astounding. Zero system maintenance is amazing.