Bref: Gracefully stop the PHP process before max execution time is reached

[t-geindre]

If our lambda reach the maximum execution time, it gets shut down. Since we use an external log system, we don't get any log in that case and we still have to check on cloudwatch what happened.

We could use set_time_limit() or ini_set() with the max_execution_time directive. But the issue with those functions is that you can't take control back when the limit is reached, the PHP process is immediatly stopped. The other issue is that systems calls are not taken in account (eg sleep(), file_get_contents(), etc.).

So I'm trying to find a solution to handle that case.

I had the following idea: we could configure a specific timeout (obviously shorter than the lambda max execution time) which, when it's reached, trigger a signal sending to the PHP process (eg SIGTERM).

This could let some time to the PHP process to gracefully shut down and do whatever is required during the time left (eg log what happened).

And this should obviously be handled in the parent process, the javascript one (and I'm not sure to be skilled enough in javascript to achive that).

This is just an idea, any other idea is very welcome.

[t-geindre]

I made a simple POC to validate my idea.

Here a simple PHP script which handle signals and runs forever:

<?php
declare(ticks = 1);

class SigtermException extends \Exception {}

pcntl_signal(SIGTERM, function () {
    throw new SigtermException('SIGTERM received');
});


set_exception_handler(function ($exception) {
    // Log exception as usual
    echo "Exception thrown: " , $exception->getMessage(), "\n";
    // If it's a Sigterm one, stop execution
    if ($exception instanceof SigtermException) {
        exit(1);
    }
});


for(;;){
    echo "Running\n";
    usleep(100000);
}

And here is the javascript handler which runs the PHP script:

const spawn = require('child_process').spawn;

var script = spawn('php', ['test.php']);
var timeoutHandler = setTimeout(() => script.kill('SIGTERM'), 2000);

script.on('close', (code) => {
    console.log('Process closed with code: ' + code);
    clearTimeout(timeoutHandler);
});

script.stdout.on('data', function(data) {
    console.log(data.toString());
});

script.stderr.on('data', function(data) {
    console.log('[STDERR] ' + data.toString());
});

And finally the execution output:

$ nodejs handler.js
Running

Running

[...]

Running

Exception thrown: SIGTERM received

Process closed with code: 1

So it seems to be working as expected and would allows us to to gracefully shut down the PHP process before the lambda shuts down.

[mnapoli]

Yes that is really interesting!

Just to be the devil's advocate: when not using AWS lambda (i.e. Apache/Nginx setup) there is a time limit too and there is the same problem right? So in a way my opinion is that since it's not a new constraint specific to lambdas, it's not critical to offer a solution for this.

However if it's easy and doesn't impact a lot the architecture of Bref then why not! And since we control the PHP binary we can make sure pcntl is available.

[t-geindre]

Just to be the devil's advocate: when not using AWS lambda (i.e. Apache/Nginx setup) there is a time limit too and there is the same problem right? So in a way my opinion is that since it's not a new constraint specific to lambdas, it's not critical to offer a solution for this.

We shouldn't think as if we were working on a classic setup. On natively supported languages, AWS provides a context object which provides a method to get the remaining execution time. That's exactly why I had this idea, in other languages we can check the time left and that's not possible at the moment with PHP/Bref.

Another solution could be to give access to that context object, this could be done through socket for example.

[t-geindre]

A simple solution is the following:

pcntl_async_signals(true);
pcntl_signal(SIGALRM, function () {
     throw new \RuntimeException('Maximum execution time reached');
});
pcntl_alarm(2); // a SIGALRM will be triggered after 2 seconds

This could work and wouldn't require any modification in Bref.