Youtor
Set up Dead Letter Queue in Amazon EventBridge to Improve Resiliency
>> YOUR LINK HERE: ___ http://youtube.com/watch?v=8-yC4XfuP0Y
Problem Statement: • ------------------------------------ • You are a Data engineer at LTDC. • You have been asked to improve the overall resiliency of the existing EventBridge setup and handle any event processing failures. • Solution: • --------------- • Sometimes an event isn't successfully delivered to the target specified in a rule. • This can happen when, for example, the target resource is unavailable, when EventBridge lacks permission to the target resource, or due to network conditions. • When an event isn't successfully delivered to a target , EventBridge retries sending the event. You set the length of time it tries, and number of retry attempts in the Retry policy settings for the target. • By default, EventBridge retries sending the event for 24 hours and up to 185 times. • If an event isn't delivered after all retry attempts are exhausted, the event is dropped and EventBridge doesn't continue to process it. • To avoid losing events after they fail to be delivered to a target, you can configure a dead-letter queue (DLQ) and send all failed events to it for processing later. • Implementation-- • Step 1: • ---------- • Create a simple Lambda • Step 2: • ----------- • Create a DLQ • Step 3: • ------------ • Use the default eventbus create a rule to run the lambda in every 1 min with DLQ Configuration • Step 4: • ----------- • Update the principle in DLQ Policy so that eventbridge can publish events-- • Service : events.amazonaws.com • Step 4: • -------------- • Remove Permission from Lambda try it • (Eventbridge will try 1 more time based on configuration then put the record in DLQ) • Prerequisite: • ---------------------- • Event Driven Architecture Pattern from On-premise event to Cloud • • Event Driven Architecture Pattern fro... • Fan out Architecture in AWS with SNS + SQS + Lambda + Python • • Fan out Architecture in AWS with SNS ... • Testing Retry Behavior and DLQs in AWS Lambda • • Testing Retry Behavior and DLQs in AW... • Content filtering in Amazon EventBridge event patterns • • Content filtering in Amazon EventBrid... • Integration of HTTP API with Event Bridge (Hands-on in-depth) • • Integration of HTTP API with Event Br... • You can explore more about Improving failure recovery for Amazon EventBridge from this link -- • https://aws.amazon.com/blogs/compute/... • Check this playlist for more AWS Projects or POC in Big Data domain: • • Demystifying Data Engineering with Cl...
#############################
