Aws-sdk-php: Consecutive API calls return curl error code 35 on CentOS 6.5

[bikegriffith]

After initializing the S3Client with just a key and secret, I can only make 1 API call before getting curl error code 35.

E.g.

$s3->putObject([
    'Bucket'=>'MyBucket',
    'Key'=>'MyFolder/',
    'Body'=>'',
    'ContentLength'=>0
]);

The 2nd time results in the following error:

PHP Fatal error:  Uncaught exception 'Guzzle\Http\Exception\CurlException' with message '[curl] 35:  [url] https://s3.amazonaws.com/MyBucket/MyFolder/' in /home/mike/src/awstest/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php:338
Stack trace:
#0 /home/mike/src/awstest/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php(279): Guzzle\Http\Curl\CurlMulti->isCurlException(Object(Guzzle\Http\Message\EntityEnclosingRequest), Object(Guzzle\Http\Curl\CurlHandle), Array)
#1 /home/mike/src/awstest/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php(244): Guzzle\Http\Curl\CurlMulti->processResponse(Object(Guzzle\Http\Message\EntityEnclosingRequest), Object(Guzzle\Http\Curl\CurlHandle), Array)
#2 /home/mike/src/awstest/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php(227): Guzzle\Http\Curl\CurlMulti->processMessages()
#3 /home/mike/src/awstest/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php(211): Guzzle\Http\Curl\CurlMulti->executeHandles()
#4 /home/mike/src/awstest/vendor/guzzle/guz in /home/mike/src/awstest/vendor/aws/aws-sdk-php/src/Aws/Common/Client/AbstractClient.php on line 286

I'm hitting a bucket in the US East Region.

I've tried passing several combinations of curl.options to the factory constructor, but no luck

$s3 = S3Client::factory([
    'key' => '****',
    'secret' => '****',
    'curl.options' => [
        'CURLOPT_SSLVERSION' => 3,
    ]
]);

Server package details:
CentOS 6.5
php 5.4.23-29
curl 7.19.7-37
openssl 1.0.1e-16

$ curl --version
curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

Edit: renamed and updated details based on latest findings.

[jeremeamia]

Hey, I just tried this code, and it seemed to work fine. You shouldn't need to specify ContentLength or add any curlopts.

$s3 = S3Client::factory(['key' => '****', 'secret' => '****']);

// Add wire logging to see raw request and response
$s3->addSubscriber(\Guzzle\Plugin\Log\LogPlugin::getDebugPlugin());

$s3->putObject([
    'Bucket' => 'my-bucket',
    'Key'    => 'subdir/',
    'Body'   => '',
]);

Also, why are you creating empty pseudo-folders in the first place? I can't think of any good reason why this would be needed.

[mtdowling]

Curl error 35 is described as follows:

CURLE_SSL_CONNECT_ERROR (35)

A problem occurred somewhere in the SSL/TLS handshake. You really want the error buffer and read the message there as it pinpoints the problem slightly more. Could be certificates (file formats, paths, permissions), passwords, and others.

This error could really be anything, so you'll need to turn on verbose cURL output in order to see what's happening. This can be done using the following snippet:

$s3 = S3Client::factory([
    'key' => '****',
    'secret' => '****',
    'curl.options' => ['CURLOPT_VERBOSE' => true]
]);

The PHP manual claims that you can only set CURLOPT_SSLVERSION to 2 or 3. It looks like setting this value to 1 is forcing TLSv1 (or CURL_SSLVERSION_TLSv1).

The SSL version (2 or 3) to use. By default PHP will try to determine this itself, although in some cases this must be set manually.

The cURL docs further describe CURLOPT_SSLVERSION:

• CURL_SSLVERSION_DEFAULT: The default action. This will attempt to figure out the remote SSL protocol version, i.e. either SSLv3 or TLSv1 (but not SSLv2, which became disabled by default with 7.18.1).
• CURL_SSLVERSION_TLSv1: Force TLSv1.x
• CURL_SSLVERSION_SSLv2: Force SSLv2
• CURL_SSLVERSION_SSLv3: Force SSLv3
• CURL_SSLVERSION_TLSv1_0: Force TLSv1.0 (Added in 7.34.0)
• CURL_SSLVERSION_TLSv1_1: Force TLSv1.1 (Added in 7.34.0)
• CURL_SSLVERSION_TLSv1_2: Force TLSv1.2 (Added in 7.34.0)

The automatic determination made when deciding which SSL version to use is probably pretty accurate, but it looks like you're in the minority that will need to manually specify this setting.

[bikegriffith]

I'm still futzing with curl options to see what works. The CURLOPT_SSLVERSION=>1 doesn't actually seem work (my bad), so I've removed it. And it's not just erroring on putObject, but rather for many other calls.

With curl debugging on, here's what I'm seeing:

First call works OK:

[3] > $s3->putObject(['Bucket'=>'DSITWConfig', 'Key'=>'TESTOH/foo', 'Body'=>'test']);
* About to connect() to s3.amazonaws.com port 443 (#0)
*   Trying 72.21.195.65... * Connected to s3.amazonaws.com (72.21.195.65) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Resources/cacert.pem
  CApath: none
* SSL connection using SSL_RSA_WITH_RC4_128_SHA
* Server certificate:
*       subject: CN=s3.amazonaws.com,O=Amazon.com Inc.,L=Seattle,ST=Washington,C=US
*       start date: Sep 09 00:00:00 2013 GMT
*       expire date: Sep 10 23:59:59 2014 GMT
*       common name: s3.amazonaws.com
*       issuer: CN=VeriSign Class 3 Secure Server CA - G3,OU=Terms of use at https://www.verisign.com/rpa (c)10,OU=VeriSign Trust Network,O="VeriSign, Inc.",C=US
> PUT /DSITWConfig/TESTOH/foo HTTP/1.1
Host: s3.amazonaws.com
User-Agent: aws-sdk-php2/2.4.12 Guzzle/3.7.4 curl/7.19.7 PHP/5.4.23
Content-MD5: CY9rzUYh03PK3k6DJie09g==
Date: Wed, 08 Jan 2014 17:14:52 +0000
Authorization: AWS ***(redacted)***
Content-Length: 4

< HTTP/1.1 200 OK
< x-amz-id-2: ***(redacted)***
< x-amz-request-id: BE71F0D8CE0243BC
< Date: Wed, 08 Jan 2014 17:15:04 GMT
< ETag: "098f6bcd4621d373cade4e832627b4f6"
< Content-Length: 0
< Server: AmazonS3
<
* Connection #0 to host s3.amazonaws.com left intact
# Request:
PUT /DSITWConfig/TESTOH/foo HTTP/1.1
Host: s3.amazonaws.com
User-Agent: aws-sdk-php2/2.4.12 Guzzle/3.7.4 curl/7.19.7 PHP/5.4.23
Content-MD5: CY9rzUYh03PK3k6DJie09g==
Date: Wed, 08 Jan 2014 17:14:52 +0000
Authorization: AWS ***(redacted)***
Content-Length: 4

test

# Response:
HTTP/1.1 200 OK
x-amz-id-2: ***(redacted)***
x-amz-request-id: BE71F0D8CE0243BC
Date: Wed, 08 Jan 2014 17:15:04 GMT
ETag: "098f6bcd4621d373cade4e832627b4f6"
Content-Length: 0
Server: AmazonS3



# Errors: 0
 â object(Guzzle\Service\Resource\Model)(

)
[4] >

Then the next call fails (even if it's the exact same call):

[6] *> $s3->listObjects(['Bucket' => 'DSITWConfig']);
* About to connect() to s3.amazonaws.com port 443 (#0)
*   Trying 72.21.195.65... * Connected to s3.amazonaws.com (72.21.195.65) port 443 (#0)
*   CAfile: /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Resources/cacert.pem
  CApath: none
* NSS error -8023
* Expire cleared
* Closing connection #0
* About to connect() to s3.amazonaws.com port 443 (#0)
*   Trying 72.21.195.65... * Connected to s3.amazonaws.com (72.21.195.65) port 443 (#0)
*   CAfile: /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Resources/cacert.pem
  CApath: none
* NSS error -8023
* Expire cleared
* Closing connection #0
* About to connect() to s3.amazonaws.com port 443 (#0)
*   Trying 72.21.195.65... * Connected to s3.amazonaws.com (72.21.195.65) port 443 (#0)
*   CAfile: /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Resources/cacert.pem
  CApath: none
* NSS error -8023
* Expire cleared
* Closing connection #0
* About to connect() to s3.amazonaws.com port 443 (#0)
*   Trying 72.21.195.65... * Connected to s3.amazonaws.com (72.21.195.65) port 443 (#0)
*   CAfile: /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Resources/cacert.pem
  CApath: none
* NSS error -8023
* Expire cleared
* Closing connection #0
PHP Fatal error:  Uncaught exception 'Guzzle\Http\Exception\CurlException' with message '[curl] 35:  [url] https://s3.amazonaws.com/DSITWConfig' in /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php:338
Stack trace:
#0 /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php(279): Guzzle\Http\Curl\CurlMulti->isCurlException(Object(Guzzle\Http\Message\Request), Object(Guzzle\Http\Curl\CurlHandle), Array)
#1 /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php(244): Guzzle\Http\Curl\CurlMulti->processResponse(Object(Guzzle\Http\Message\Request), Object(Guzzle\Http\Curl\CurlHandle), Array)
#2 /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php(227): Guzzle\Http\Curl\CurlMulti->processMessages()
#3 /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php(211): Guzzle\Http\Curl\CurlMulti->executeHandles()
#4 /home/mike/src/tests3/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php in /home/mike/src/tests3/vendor/aws/aws-sdk-php/src/Aws/Common/Client/AbstractClient.php on line 286

I attempted to reconnect before making a 2nd call and that doesn't help.

[bikegriffith]

Found an article suggesting that fix is to recompile CURL against NSS instead of openssl.
http://www.sebdangerfield.me.uk/2012/10/nss-error-8023-using-aws-sdk-for-php/

Is that really a solution? I haven't tried yet.

[mtdowling]

Found an article suggesting that fix is to recompile CURL against NSS instead of openssl.

I think you read that wrong. The article suggests that you compile cURL against OpenSSL instead of NSS.

I'm not sure if that is a valid solution, but it seems likely that it would work. This is a cURL and platform specific issue that I've never seen. You might also try just yum updating nss.

[bikegriffith]

@mtdowling Yes, I did read that wrong. I edited the original issue to include the output of curl --version, which shows I'm compiled against NSS. After more hacking around all afternoon (upgrading NSS, trying different versions of Guzzle, every curlopt in the book, etc.) I think I might have to go that route, which I'm really really not excited about :-/

[bikegriffith]

. o O ( wondering out loud if Guzzle's persistent connections have anything to do with this )

[bikegriffith]

Latest findings (after rebuilding curl and libcurl from the source RHEL rpms) is that this might just be an artifact of trying to run commands in Laravel's php artisan tinker shell. Stuffing the same 5 lines in a TestCase allows them to run successfully with no other hijinks.

This could be the fault of Boris.

[mtdowling]

Whoops! Didn't mean to close this issue.

Thanks for keeping us posted. Let us know if you get some concrete information that we can help others with.

[mtdowling]

I'm to go ahead and resolve this issue. Please let us know if you come up with anything that you think might be useful to document in our user guide.

[varunwy]

I am getting the same error and I have some idea on what's going on here.

Below are the steps to reproduce this problem while using PHP aws sdk.

1. make an aws api call
2. use pcntl to fork a child process
3. In the child process make another aws api call
   You will get error:
   Aws\Common\Exception\TransferException: [curl] 35: A PKCS #11 module returned CKR_DEVICE_ERROR, indicating that a problem has occurred with the token or slot.

This problem will happen only if you use curl with NSS. Curl with Openssl works fine.

According to this discussion (https://bugzilla.mozilla.org/show_bug.cgi?id=331096), the following
is illegal: A parent process uses PKCS11 module, then forks a child and the child tries to use PKCS11 module

The solution is to reinitialise all PKCS#11 modules in the child process. Any idea how do we do that?

[jeremeamia]

Hmmm... that is an interesting lead, but makes sense since @bikegriffith was only seeing the issue when using Boris, which does use pcntl.

What happens if, instead of using the same client to make the 2nd request, you instantiate a new client object to make that request?

[varunwy]

I am already creating a new client object in every child process. Same error.

[varunwy]

Any updates?

[allella]

If you happen to be connecting to a server utilizing CloudFlare then error 35 may be due to an out of date version of libcurl. Solution to error 35 on CentOS 6.x

[techouse]

I can confirm that this is a Tinker Shell issue. Running the same in PHP's interactive shell everything works fine.

[rgjodekerken]

I came across this thread while looking for a solution for this problem.
I just found the solution so I'm sharing it with you guys.

Starting tinker with the environment variable NSS_STRICT_NOFORK=DISABLED solves the issue.
Source: https://cohanrobinson.com/php-curl-libcurl-error-on-subsequent-requests/