Framework: SQS Queue could use long polling

[PanMan]

It would be nice to have if Laravel would support Long polling for SQS Queues. SQS does support this:
http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html While the queues also work without this, there currently is a balance between having a higher wait-time (and thus slower processing of items) or a lower wait time (and higher costs). Also the wait time makes the minimal wait time 1 second.
Using the Long Polling would make the costs for SQS significantly lower, and the response quicker. The underlaying AWS SDK does support long polling with its WaitTimeSeconds parameter
http://docs.aws.amazon.com/aws-sdk-php/guide/latest/service-sqs.html#receiving-messages
This could save 95% of the costs of having a queue poll every second.
I understand this is an optimisation and not a critical need, but it would be nice to have (as it would save costs and work quicker).

[PanMan]

It could be that using the WaitTimeSeconds parameter is currently possible, but I don't understand the laravel internals enough (yet) to use this.

[Edgpaez]

I might be stating the obvious but, behind the scenes Laravel uses Aws\Sqs\SqsClient you can find the receiveMessage call in the Illuminate\Queue\SqsQueue@pop method.

You can temporally replace the method call with this one:

$response = $this->sqs->receiveMessage(
    array('QueueUrl' => $queue, 'AttributeNames' => array('ApproximateReceiveCount'), 'WaitTimeSeconds' => 30)
);

Of course if it happens it works as expected, we'd then create a PR moving those values to the config file (I guess)

I hope this helps

[PanMan]

@Edgpaez Thanks for pointing out the right place. However, the plot thickens.. I put

echo "POPing: ".time()."\n";

on top of the pop function, expecting it to get called every second. However, it only gets called every 25 seconds or so.. But it Doesn't seem to wait 25 seconds, as it doesn't take 25 seconds for a new item to arrive.
Could it be that long polling is already working, via some default settings?

[Edgpaez]

I don't quite understand you. But I'm sure there's no other place where the long polling might be implemented other than in the Illuminate\Queue\SqsQueue@pop method.

I think you can use the Illuminate\Queue\Worker@process method to know when a job is getting fired.
Hope it helps!

[GrahamCampbell]

Probably better to go into a package.

[HSPDev]

A note for other Googlers.

Laravel doesn't provide a WaitTimeSeconds parameter at all.

I was about to extend the framework in our app, but this is actually a good thing.

You can open AWS Console, go to SQS, click on your queue, "Queue Actions" button and "Configure Queue"

In here you have the "Receive Message Wait Time" parameter. This defines ReceiveMessageWaitTimeSeconds, which is the default for the function, defined from the AWS side of things. Since Laravel uses the official PHP AWS SDK underneath, it will honor this default provided by SQS itself, and allow you to reconfigure standard Laravel SQS workers to have a WaitTimeSeconds = Long Polling.