lambda-kinesis-forward
baikonur-oss/lambda-kinesis-forward/aws
Terraform module and Lambda for forwarding JSON logs from one Kinesis stream to another.
Amazon Kinesis to Kinesis log forwarding Terraform module Terraform module and Lambda for transferring JSON log records from one Kinesis Data Streams stream to another, with key filtering. !terraform v0.12.x  Prerequisites 1. Records in Kinesis stream must be valid JSON data. Non-JSON data will be ignored. 1. gzipped JSON, CloudWatch Logs subscription filters log format are supported. 2. Broken JSON logs or logs without log type will be saved to S3 as unknown. 2. JSON data must have the following keys (key names are modifiable via variables): 1. log_type: Log type identifier. Used for filtering data sent to target stream. 3. Recommended keys (necessary if target stream has
| Name | Type | Description | Default |
|---|---|---|---|
| name | string | Resource name | |
| source_stream_name | string | Source Kinesis Data Stream name | |
| lambda_package_url | string | Lambda package URL (see Usage in README) | |
| failed_log_s3_bucket | string | S3 bucket name for saving failed logs (ES API errors etc.) | |
| target_stream_name | string | Target Kinesis Data Stream name | |
| batch_size | string | Maximum number of records passed for a single Lambda invocation | |
| failed_log_s3_prefix | string | Path prefix for failed logs | |
| starting_position | string | Kinesis ShardIterator type (see: https://docs.aws.amazon.com/kinesis/latest/APIR | "TRIM_HORIZON" |
| log_timestamp_field | string | Key name for log timestamp | "time" |
| timezone | string | tz database timezone name (e.g. Asia/Tokyo) | "UTC" |
| handler | string | Lambda Function handler (entrypoint) | "main.handler" |
| log_type_field_whitelist | list(string) | Log type whitelist (if empty, all types will be processed) | [] |
| tags | map(string) | Tags for Lambda Function | {} |
| log_retention_in_days | string | Lambda Function log retention in days | 30 |
| timeout | string | Lambda Function timeout in seconds | 60 |
| runtime | string | Lambda Function runtime | "python3.7" |
| log_id_field | string | Key name for unique log ID | "log_id" |
| kinesis_max_retries | number | Times to retry PutRecords on errors (wait time between retires is 500ms) | 3 |
| tracing_mode | string | X-Ray tracing mode (see: https://docs.aws.amazon.com/lambda/latest/dg/API_Traci | "PassThrough" |
| enable_kinesis_mapping | string | Determines if the event source mapping will be enabled | true |
| log_type_field | string | Key name for log type | "log_type" |
| log_type_unknown_prefix | string | Log type prefix for logs without log type field | "unknown" |
| memory | string | Lambda Function memory in megabytes | 256 |
Terraform module to deploy landing zone subscriptions (and much more) in Azure
Terraform Azure Verified Pattern Module for Azure Landing Zone
Terraform Azure Verified Resource Module for Key Vault
Terraform module for building and deploying Next.js apps to AWS. Supports SSR (L