In this chapter, we will cover the following topics
- SNS – Simple Notification Service
- SQS – Simple Queuing Service
- SWF – Simple Workflow Service
SNS – Simple Notification Service
Amazon SNS is a managed notification service
It works on a push mechanism, the publisher raises a request to send a message to the subscribers
Use Cases
- Installed applications on a smartphone, often send a notification for offers and other notifications
- SMS notification on any valid mobile number
- When EC2 machine over load or under load of the performance, it automatically sends a notification to the admin for the same
Push v/s Pull mechanism
- AWS Simple Notification Service (SNS) works on push technology.
- It is also called server push. It means, the message or the transaction is initiated by the publisher and SNS delivers the same to the subscribers
- It is opposite to the pull mechanism, which is also called client pull. In which, the client raises the request to fetch or pull data from the server
- SQS works on the pull mechanism while SNS works on a push mechanism
Way of Working
- First, you need to create an Amazon SNS topic
- An SNS topic acts as an access point in between the publisher and subscriber applications
- The publisher communicates asynchronously with the subscribers using SNS
- Subscribers can be an entity such as a Lambda function, SQS, HTTP or HTTPS endpoint, email, or a mobile device that subscribes to SNS topic for receiving notifications
- To receive notifications, subscribers must specify the protocol (that is, HTTP, HTTPS, Email, Email-JSON, Amazon SQS, Application, AWS Lambda, SMS)
- When a publisher has new information to notify to the subscribers, it publishes a message to the topic
- Finally, SNS delivers the message/notification to all subscribers.
Creating SNS topic
First, it is essential you create an SNS topic, then it is possible for a publisher to publish a message and for subscribers to subscribe to get a notification
SQS – Simple Queuing Service
SQS is a highly reliable, scalable, and distributed message queuing service provided by Amazon
It’s a hosted solution provided by Amazon so that you do not need to manage the service infrastructure
SQS stores the messages in transit as they travel between various applications and microservices
Use Case
- News website with image upload facility – Optimization
Way of Working
Features of SQS
Redundant infrastructure: ensures that a message is delivered at least once in a standard queue and it ensures that a message is delivered exactly once in a First In First Out (FIFO) queue.
Multiple producers and consumers: Multiple components of a distributed application can concurrently send and receive messages at the same time.
Queue-wise configurable settings: SQS provides options to configure each queue independently. You do not need to have the same configuration for all the queues.
Variable message size: SQS supports a maximum message size of 256 KB.
Queue access control: SQS allows you to control producers and consumers that can send and receive messages to or from the queue.
Delay queue: SQS enables you to set a delay time in a queue. Delay time ensures that a message inserted in the queue is postponed for the time configured as the delay time in the queue.
Types of Queues
Standard queue
- It’s available in all AWS regions.
- It supports an almost unlimited number of Transactions Per Second (TPS ) with each API action.
- It generally sends the data in the same order as it receives it, however, on certain occasions the order may change.
- Standard queue guarantees that a message will be delivered at least once, however, occasionally a message may be delivered more than once.
- It is used when the throughput is more important than the order in which the data is processed.
FIFO Queue
- It’s available only in the US East (N. Virginia), US East (Ohio), US West (Oregon), and EU (Ireland) regions.
- It supports limited throughput. It can support up to 300 messages per second without any batching.
- With a batch size of 10 messages, it can support up to 3,000 messages per second.
- It ensures that the data is sent in the FIFO order.
- FIFO queue guarantees that a message will be delivered exactly once. The message remains in the queue until the consumer confirms that the message is processed.
- It is used when the order in which the data is processed is more important than the throughput
Dead Letter Queue (DLQ)
- A DLQ is used by other queues for storing failed messages that are not successfully consumed by consumer processes.
- SQS uses a redrive policy to indicate the source queue and the scenario in which SQS transfers messages from the source queue to the DLQ.
- If a queue fails to process a message a predefined number of times, that message is moved to the DLQ.
- While creating a queue, you can enable a redrive policy and set the DLQ name as well as maximum receive count, after which if the message is still unprocessed, it can be moved to the DLQ.
Queue attributes
Default visibility Timeout – The length of time that a message is received from a queue will be invisible to other receiving components.
Message Retention Period – The amount of time that SQS retains a message if it does not get deleted.
Maximum Message Size – Maximum message size accepted by SQS.
Delivery Delay – The amount of time to delay the first delivery of all messages added to the queue.
Receive Message Wait Time – The maximum amount of time that a long polling receive call waits for a message to become available before returning an empty response.
Creating a queue
SWF – Simple Workflow Service
Amazon Simple Workflow Service (SWF) is a workflow management service that helps in building applications that can handle work through distributed components.
Using SWF, you can define a number of tasks that can be executed in a predefined sequence.
Use Cases
- Customer order processing workflow
- Business process workflow
- Analytics pipelines
Core concepts of SWF
Workflow:
It is a mechanism to execute a number of distributed application tasks in an asynchronous way.
While creating a workflow, you need to determine the tasks to be executed in the workflow. SWF recognizes these tasks as activities.
Workflow history:
SWF keeps the history or execution progress of any workflow in the workflow history.
Once the execution of a workflow starts, SWF keeps a detailed history of each and every step of the workflow.
Whenever the workflow execution state changes, such as when a new activity is scheduled in the workflow or an activity is completed, it is represented as an event in the workflow history.
- Actors:
- An actor is a program or an entity that performs different types of activities in a workflow.
- An actor can be any of the following:
- Workflow starter – is a program or an application that starts the execution of a workflow
- Decider – a program or an application that decides the coordination logic of a workflow.
- Activity worker – is a program or an application that receives tasks from SWF, executes the tasks, and returns the result back to SWF
Tasks – The work assignments that SWF provides to activity workers and deciders are called tasks.
AWF Domain – Domains in SWF are a mechanism to scope SWF resources such as workflows, activity types, and workflow execution.
All the resources are scoped to a domain.
Task List – Task lists are a mechanism to organize different tasks related to a workflow. Task lists can be thought of as dynamic queues.
Workflow execution closure – When a workflow execution is started, it changes an open state.
- Open workflow execution can be closed as one of the following:
- Completed
- Canceled
- Failed
- Timed out
Summary
In this chapter we have gone through an important concept of Pull and Push mechanism and which service uses which mechanism along with SNS, SQS and SWF.
You can always start your learning journey from the beginning with our AWS complete tutorial series from this link.
