java SQS maxNumberOfMessages

Question

我使用Java客户机应用程序查询SQS队列中的消息。队列有12000条消息作为测试设置。我正在使用openJDK和aws java sdk最新版本（software.amazon.awssdk 2.10.62）pom。xml显示得更远

我看到的问题是，尽管设置了maxNumberOfMessages（10），但我只收到了3条。我知道这是一个最大值，但不能保证消息数量，但是返回的消息数量没有波动。总是3点

AWS Documentation: MaxNumberOfMessages The maximum number of messages to return. Amazon SQS never returns more messages than this value (however, fewer messages might be returned). Valid values: 1 to 10. Default: 1. Type: Integer Required: No

Consuming Messages Using Short Polling

When you consume messages from a queue using short polling, Amazon SQS samples a subset of its servers (based on a weighted random distribution) and returns messages from only those servers. Thus, a particular ReceiveMessage request might not return all of your messages. However, if you have fewer than 1,000 messages in your queue, a subsequent request will return your messages. If you keep consuming from your queues, Amazon SQS samples all of its servers, and you receive all of your messages.

因此，我们使用旧的aws sdk和新的aws sdk在java中测试了两个客户端，结果相同。始终只回3条消息

有趣的是，如果不在外部（在我强大的桌面上）运行应用程序，而是将其作为AWS Lambda运行，则会收到10条消息。这个lambda测试是由一位同事使用JavaScript完成的

所以问题仍然是，为什么我们每次请求只能收到3条消息，而在lambda中，似乎可以收到10条

假设每个请求有一个成本，则是基于亚马逊利润=）的加权随机分布。）

SQS试验方法：

public void SQStart()
{
    AwsBasicCredentials awsCreds = AwsBasicCredentials.create("accessKeyID", "secretKeyID");
    AwsCredentialsProvider creds = StaticCredentialsProvider.create(awsCreds);
    SqsClient sqs = SqsClient.builder().credentialsProvider(creds).region(Region.EU_WEST_1).build();
    GetQueueUrlRequest getQueueRequest = GetQueueUrlRequest.builder()
            .queueName(QUEUE_NAME)
            .build();
    String queueUrl = sqs.getQueueUrl(getQueueRequest).queueUrl();

    for (int x =1; x < 100; x++) {
        ReceiveMessageRequest receiveMessageRequest = ReceiveMessageRequest.builder()
                .queueUrl(queueUrl)
                .maxNumberOfMessages(10)
                .build();


        List<Message> messages = sqs.receiveMessage(receiveMessageRequest).messages();
        if (messages.size() > 3 ) {
            System.out.println("YEY More than 3 Messages: "+ messages.size());
        }
    }
}

波姆。XML:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>groupId</groupId>
    <artifactId>SQSTest</artifactId>
    <version>1.0-SNAPSHOT</version>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>8</source>
                    <target>8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>


    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>software.amazon.awssdk</groupId>
                <artifactId>bom</artifactId>
                <version>2.10.62</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>

            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>sqs</artifactId>
        </dependency>

        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.9</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpcore</artifactId>
            <version>4.4.10</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>commons-logging</groupId>
            <artifactId>commons-logging</artifactId>
            <version>1.2</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-s3</artifactId>
            <version>1.11.720</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.6.1</version>
        </dependency>
    </dependencies>
</project>

Answer 1

我认为this是一个类似的问题。正如雅各布所指出的那样，长期投票似乎是解决这个问题的办法

Answer 2

Given there is a cost per request is the weighted random distribution based on amazon profit =))

很明显，您的目标是降低成本，无论是通过向SQS发送更少的请求，还是通过强制SQS提供最大数量的可用消息

正如您在问题中所述，SQS没有义务提供最大数量的可用消息。不过，我想告诉你一件事，假设你还没有意识到

---

长时间投票

Amazon简单队列服务的Developer Guide表示：

The process of consuming messages from a queue depends on whether you use short or long polling. By default, Amazon SQS uses short polling, querying only a subset of its servers (based on a weighted random distribution) to determine whether any messages are available for a response. You can use long polling to reduce your costs while allowing your consumers to receive messages as soon as they arrive in the queue.

您发送给SQS的消息可能都存储在不同的服务器上。如文档所述，如果队列设置为使用短轮询，则只能查询服务器的子集。我猜你在调用receiveMessage时运气不好，每次只返回3

如果我们在同一个文档页面上查看长轮询的好处，就会发现：

Long polling offers the following benefits:

• Eliminate empty responses by allowing Amazon SQS to wait until a message is available in a queue before sending a response. Unless the connection times out, the response to the ReceiveMessage request contains at least one of the available messages, up to the maximum number of messages specified in the ReceiveMessage action.

• Eliminate false empty responses by querying all—rather than a subset of—Amazon SQS servers.

第二颗子弹在这里非常重要。即使没有看到空的响应，也可能有更多存储在服务器上的消息没有被查询。如果启用长轮询，则如果总共有3台以上的服务器，则返回的消息量有望增加

因此，我的建议是对您的队列启用长轮询。要做到这一点，请参见Setting Up Long Polling页面

---

正如DevilCode在下面的comment中提到的，他能够通过使用FIFO队列而不是标准队列，并通过对其启用长轮询来解决问题

Answer 3

长轮询：

        ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest(queueUrl)
              .withWaitTimeSeconds(10)     // long poll: wait 10 seconds, max is 20 seconds
              .withMaxNumberOfMessages(10);