AWS CDA Study List - SQS

SQS

Basics

• Messaging system for decoupling
• Does not support FIFO (first in, first out)
• Message are sent one or more times to ensure that messages are sent
• Supports short and long polling
• In short polling SQS Service gets subset of your messages or nothing
• In long polling SQS Service waits until at least on message is available in queue
• In simple terms short polling goes to check whether message exists and comes back either with data or not. Long polling waits as long as there is some data.
• VisibilityTimeout -parameter controls time time how long message is not visible to twice or from queue
• ChangeMessageVisibility -parameter can be used to prolong VisibilityTimeout period. Time is added to the to time how long message has been hidden
• ReceiveMessageWaitTimeSeconds -parameter controls whether long polling is on or not 
• DelaySeconds -parameter can be used to hid message from all clients from queue. This happens before VisibilityTimeout -parameter kicks in
• GetQueueAttributes API -action with "ApproximateNumberOfMessages" returns the number of messages waiting in the queue
• GetQueueAttributes API -action with "ApproximateNumberOfMessagesNotVisible" returns the number of messages in flight
• Dead Letter Queue is used for messages that can't be processed and need further investigation
• MessageRetentionPeriod -parameter determines how long SQS holds the message in the queue

Limits

• Minimum message size is 1KB
• Maximum message size is 256KB
• At max 120,000 messages can be inflight 
• Message can only contain XML, JSON and unformatted text
• Message can contain 10 metadata attributes
• Queue name can be up to 80 characters long
• Queue name is case-sensitive

Defaults

• VisibilityTimeout minimum is 0 seconds
• VisibilityTimeout default is 30 seconds
• VisibilityTimeout maximum time is 12 hours
• ReceiveMessageWaitTimeSeconds is 0
• MessageRetention minimum retention period is 1 minute
• MessageRetention default retention period is 4 days
• MessageRetention maximum retention period is 14 days
• Long Polling maximum timeout value is 20 seconds

Following topics are exam questions collected through Internet and should be evaluated as so. Answers are mine and have been checked with answers collected through the internet, but might still be wrong.

SQS message lifecycle

When a Simple Queue Service message triggers a task that takes 5 minutes to complete, which process below will result in successful processing of the message and remove it from the queue while minimizing the chances of duplicate processing?

A. Retrieve the message with an increased Visibility timeout, delete the message from the queue, process the message
B. Retrieve the message with increased DelaySeconds, process the message, delete the message
C. Retrieve the message with an increased Visibility timeout, process the message, delete the message from the queue
D. Retrieve the message with increased DelaySeconds, delete the message from the

Why?

Increased visibility timeout will reduce the possibility of duplicate processing. Delete message should always happen after proper processing of the message.
http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/AboutVT.html

SQS message FIFO or not?

Which of the following statements about SQS is true?

A. Messages will be delivered one or more times and messages will be delivered in First in, First out order

B. Messages will be delivered exactly once and message delivery order is indeterminate

C. Messages will be delivered exactly once and messages will be delivered in First in, First out order

D. Messages will be delivered one or more times and message delivery order is indeterminate

Why?

https://aws.amazon.com/sqs/faqs/

Q: Does Amazon SQS provide first-in-first-out (FIFO) access to messages?

Amazon SQS provides a loose-FIFO capability that attempts to preserve the order of messages. However, we have designed Amazon SQS to be massively scalable using a distributed architecture. Thus, we can't guarantee that you will always receive messages in the exact order you sent them (FIFO).

If your system requires the order of messages to be preserved, place sequencing information in each message so that messages can be ordered when they are received.

Q: Does Amazon SQS provide at-least-once delivery of messages?

Yes. Amazon SQS guarantees that each message is delivered at least once. Amazon SQS stores copies of your messages on multiple servers for redundancy and high availability. On rare occasions, one of the servers that stores a copy of a message might be unavailable when you receive or delete the message.

If this occurs, the copy of the message will not be deleted on that unavailable server, and you might get a copy of that message again when you receive messages (at-least-once delivery).

You must design your applications to be idempotent (that is, they must not be affected adversely when processing the same message more than once).

SQS Visibility timeout default

If a message is retrieved from a queue in Amazon SQS, how long is the message inaccessible to other users by default?

A. 30 seconds

B. 0 seconds

C. 1 hour

D. 1 day

E. Forever

Why? 

http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/AboutVT.html

Each queue starts with a default setting of 30 seconds for the visibility timeout. You can change that setting for the entire queue. Typically, you'll set the visibility timeout to the average time it takes to process and delete a message from the queue. When receiving messages, you can also set a special visibility timeout for the returned messages without changing the overall queue timeout.

Use "ChangeMessageVisibility" action to change it on the fly.

SQS Long polling

Company B provides an online image recognition service and utilizes SOS to decouple system components for scalability The SQS consumers poll the imaging queue as often as possible to keep end-to-end throughput as high as possible. However, Company B is realizing that polling in tight loops is burning CPU cycles and increasing costs with empty responses. How can Company B reduce the number of empty responses?

A. Set the imaging queue Visibility Timeout attribute to 20 seconds

B. Set the DelaySeconds parameter of a message to 20 seconds

C. Set the Imaging queue ReceiveMessageWaitTimeSeconds attribute to 20 seconds

D. Set the imaging queue MessageRetentionPeriod attribute to 20 seconds

Why? 

http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html

Enable long polling why changing ReceiveMessageWaitTimeSeconds > 0. 

One benefit of long polling with Amazon SQS is the reduction of the number of empty responses, when there are no messages available to return, in reply to a ReceiveMessage request sent to an Amazon SQS queue. Long polling allows the Amazon SQS service to wait until a message is available in the queue before sending a response. So unless the connection times out, the response to the ReceiveMessage request will contain at least one of the available messages (if any) and up to the maximum number requested in the ReceiveMessage call.

Reducing the number of empty responses and false empty responses also helps reduce your cost of using Amazon SQS.

There are three different API action calls you can use to enable long polling in Amazon SQS, ReceiveMessage, CreateQueue, and SetQueueAttributes. For ReceiveMessage, you configure the WaitTimeSeconds parameter, and for CreateQueue and SetQueueAttributes, you configure the ReceiveMessageWaitTimeSeconds attribute.

Your application provides data transformation services. Files containing data to be transformed are first uploaded to Amazon S3 and then transformed by a fleet of spot EC2 instances. Files submitted by your premium customers must be transformed with the highest priority. How should you implement such a system?

A. Use a DynamoDB table with an attribute defining the priority level. Transformation instances will scan the table for tasks, sorting the results by priority level.
B. Use Route 53 latency based-routing to send high priority tasks to the closest transformation instances.
C. Use two SQS queues, one for high priority messages, the other for default priority. Transformation instances first poll the high priority queue; if there is no message, they poll the default priority queue.
D. Use a single SQS queue. Each message contains the priority level. Transformation instances poll high-priority messages first.

Why? SQS is perfect fit for this and similar scenario of using two SQS queues has been question on CDA-exam.


A company has a workflow that sends video files from their on-premise system to AWS for transcoding. They use EC2 worker instances that pull transcoding jobs from SQS. Why is SQS an appropriate service for this scenario?

A. SQS guarantees the order of the messages.
B. SQS synchronously provides transcoding output.
C. SQS checks the health of the worker instances.
D. SQS helps to facilitate horizontal scaling of encoding tasks.

Why? You can rule out A, B and C. A and B are not features of SQS and as SQS is pull system, it doesn't check health of worker instances.