Enable Asynchronous Inter-Service Communication using Amazon SNS

Purpose & Background

Amazon Simple Notification Service (SNS) is a fully managed messaging service for both system-to-system and app-to-person (A2P) communication. It enables you to communicate between systems through publish/subscribe (pub/sub) patterns that enable messaging between decoupled microservice applications or to communicate directly to users via SMS, mobile push and email.

In a monolithic application, tables are stored in a huge database. However, when you strangle out services from the monolithic application, the corresponding table is also strangled out. How to make the data in these tables consistent will be a big challenge. In the current Unishop, the basket item stored in DynamoDB has a field called unicorn_id depending on another field with the same name in the RDS. Now, consider a scenario, a customer puts a unicorn X into his cart, the administrator delete this unicorn from the inventory. In this case, how would you notify the customer that this unicorn has been deleted and make the basket database (DynamoDB) consistent with the inventory database (RDS)? A common solution to this is to ensure eventual consistency based on asynchronous messaging.

In this part, you will learn how to enable asynchronous communication using Amazon SNS to achieve eventual consistency between the basket service and the inventory service.

Why choosing asynchronous messaging?

To create the scenario, we provide you a button to delete a unicorn. When you click a unicorn tab, you will see a “Delete unicorn” button there. That button allows you to delete the unicorn. Although only the administrators should be able to delete unicorns, we provide the button for simplicity.

Prerequisites

  • Environment Setup
  • Facade Configuration
  • Serverless Basket Service
  • Containerized Inventory Service

Communication Procedure

  1. When users delete a unicorn in the Unishop, the frontend will send a DELETE request to the inventory service.
  2. When inventory service receives this DELETE request, it will publish a message to a Amazon SNS topic.
  3. SNS will publish the message to one of its subscriber, i.e., a Lambda function in thebasket service. (In this lab, we will ask you to deploy a new lambda function to handle this message.)
  4. When the new lambda gets this message, it will mark corresponding unicorns in the UnicornBasket table as unavailable products.
  5. Later on, when users refresh the page, the frontend will send a GET request to the basket service to retrieve unicorns in the shopping cart. Since the response sent by the basket service will contain the available information, the frontend will be able to notify users some unicorns are out of stock.

Create Global Secondary Index in DynamoDB

Firstly, in order to update the availablity status of the unicorns in that basket, you would be required to search the table based on unicorn_id. For this purpose, we ask you to create an Index on the unicorn_id field.

  1. In the AWS Management Console, navigate to DynamoDB. On the side bar, click on “Tables” to view your current DynamoDB tables. Then, click on the table with name “UnicornBasket.”

    Navigating to DynamoDB Console

  2. Click on the “Indexes” tab and hit “Create Index”.

    Navigating to create index

  3. In the Create Index window, for Primary key enter unicorn_id, and for the Index name enter unicorn_id_index.

    Create index

  4. Hit “Create Index” and wait till your index status goes from “Creating” to “Active”.

    Status active

Deploy SNSHandler to Lambda

  1. Open the folder “SampleApp” on the desktop. Then open a solution file named “UnicornWorkshop.sln” in it.

    project sln

  2. Right click on the project “BasketLambda” select the option “Publish to AWS Lambda”.

    publish option

  3. Do the following changes and click “Next”. Then click “Upload” to publish the new lambda function.

    • “Function Name”: Enter BasketSNSLambda

    • “Method Name”: Enter SNSHandler

    configure new lambda

  4. In the AWS Management Console, navigate to Lambda.

    navigate lambda

  5. You will see “BasketSNSLambda” show up. Click that lambda function. If you don’t see that, please reload the page after several seconds.

    click new lambda

  6. Click the copy button in the top right corner to copy the ARN of this Lambda function. Paste it into your cheat sheet (You should paste it under “SNS Lambda ARN”).

    copy lambda arn

Create SNS Topic

Then, you need to set up a SNS topic.

  1. In the AWS Management Console, navigate to Simple Notification Service. Then, click the “Topics” tab on the left side.

    search for SNS

    click topics

  2. Click “Create topic” button.

    create topic button

  3. Provide a name for the topic such as “ItemDeleted”. Leave other settings as default and click “Create topic”.

    configure topic

  4. In the new page, copy the “ARN” (a unique identifier for AWS resources) of the topic to your cheat sheet (You should paste it under “SNS Topic ARN”). Keep the page OPEN since you need to create a subscription in the next step in the same page.

Create Subsription for the New Lambda Function

  1. In the topic page you just opened, click “Create subscription.”

    click create subscription

  2. Do the following changes. Then click “Create subscription” button.

    • “Topic ARN”: Paste the value of “SNS Topic ARN” from your cheat sheet

    • “Protocol”: Choose “AWS Lambda”

    • “Endpoint”: Paste the value of “SNS Lambda ARN” from your cheat sheet

    configure subscription

Update and Redeploy Inventory Service

  1. Expand the InventoryService project in the UnicornWorkshop.sln and open appsettings.json. In that file, update “SNSTopicArn” key value with the value of “SNS Topic ARN” in your cheat sheet.

    appsettings.json update topic arn

  2. Rebuild the project before you redeploy the inventory service.

    rebuild

  3. Redeployment will be largely the same with the deployment process in the part 1 of Containerized Inventory Service. In the solution explorer, right-click the “InventoryService” and select “Publish Container to AWS…” In the wizard, simply click “Next” through the entire interaction since the settings have been saved.

    Choose publish

    After the redeployment, you may notice that the tasks you deployed in previous steps and created by auto-scaling are not instantly stopped. This is because the previous version will be retained for some time before being stopped in the process of updating.

Test the Deletion

  1. Add a unicorn to your cart.

    add unicorn

  2. Click that unicorn, and click the “Delete unicorn” button.

    delete unicorn

  3. Click your cart again or refresh the page. You will see a notification called “Some unicorns in your cart are out of stock!” and corresponding unicorns get deleted from your cart.

    check result