Bref: PHP extensions for v0.3

[mnapoli]

Issue created from #129

I'll copy the last comment from @bubba-h57 below.

---

@sandrokeil really got me pondering this. I believe we really want to avoid the developer having to mess around with lambda layers. If we do a good job, they just configure the lambda task itself.

So we want a way for them to easily manage extensions from the /var/task directory.

We have built-in extensions _(like mbstring, opcache, curl, etc.)_ and we have external extensions _(igbinary, mongodp, redis, etc)_. Perhaps we want to move as many of the built-in's to external as possible, because I do not know how to disable a built-in extension.

As far as the external extensions, we provide as many as wel like in /opt/bref/lib/php/extensions/no-debug-non-zts-20180731/*.so and simply not enable them in /opt/bref/etc/php/config.d/*.ini. Then the developer can simply enable them via their own /var/task/php/config.d/00-php.ini

The size of the extensions themselves range from apcu.so at 100K to mongo.so at 1.4M. If we are happy setting the maximum unzipped target size of our layers to 100M, the current php-default-7.3.0.zip that I am building results in an unzipped layer of 63MB ... we have a lot of room to add a lot more extensions to our layer and let the developer enable them as they see fit.

I can imagine some alternatives, but all of the alternatives I imagine result in some horrendous maintenance nightmares down the street. The kind of dreams that keep me up at night. :-)

For example, we could compile/create a PHP layer, call it layer:1 with little to no extensions. Then compile all the plausible extensions we want to provide against that PHP binary and store those extensions in S3 somewhere. If a developer wants to run layer:1 with some of the extensions, they just configure bref to download and setup the correct ones in thier function, and they would then upload the extension.so in their function package.

But eventually, we get to layer:7 ... and we need to ensure that we have the layer:7 compatible extensions and not confuse them with the layer:1 extensions.

I like the idea of providing as many extensions as possible, in a nice fat layer, that 85% of our users would likely use. Then provide a stripped layer for the 15% that are really interested in micro managing the extensions themselves.

[mnapoli]

There are many possibilities here.

First let's focus on v0.3 and getting something useful out there for most people. As you suggest a fat layer that covers the most common use cases is best.

Then we can optimize.

If we are happy setting the maximum unzipped target size of our layers to 100M

That sounds like a great goal for a first version, especially if we have some room right now!

So to recap:

• let's include and enable the built-in extensions that users can find in most PHP installations
• let's include but not enable the most common and useful extensions on Lambda (e.g. MySQL, Postgre, Mongo, etc.)
• let's not include uncommon extensions in the base layer

Let's keep building extensions in separate layers for a future version of Bref, we need to have a reasonable goal for v0.3.

[bubba-h57]

Common Built-in Extensions

Looking at the Docker "Official Image" for php and comparing that with other PHP hosting sites, it appears to me that the most reasonable list of "common" extensions compiled into PHP _(Extensions that are enabled and cannot be disabled)_ is:

Proposed Compiled in Extensions (36)
Core ctype curl date dom exif fileinfo filter ftp gettext hash iconv	json libxml mbstring mysqlnd openssl pcntl pcre PDO pdo_sqlite Phar posix readline	Reflection session SimpleXML sodium SPL sqlite3 standard tokenizer xml xmlreader xmlwriter zlib

This is five more extensions than the Docker "Official Image" for php has and about eleven more extensions than Namecheap PHP hosting provides as built-ins.

We should also provide extensions that will be commonly used in AWS Lambda. These should be installed in our base bref package, but not enabled. The developer can choose to enable them or not in their lambda function.

Common AWS Lambda Common Extensions

• OPCache - Enabled, can be disabled.
• intl - Enabled, can be disabled.
• apcu - Disabled, can be enabled.
• ElastiCache php-memcached extension - Disabled, can be enabled.
• phpredis - Disabled, can be enabled.
• PostgreSQL PDO Driver - Disabled, can be enabled.
• MySQL PDO Driver - Disabled, can be enabled.
• Mongodb - Disabled, can be enabled.

My thoughts on the extensions that should be included in the bref v3 base package.

[mnapoli]

That sounds good!

The only one that comes to mind is intl, I think it is quite commonly used (at least I remember often having to install/enable it).

I'm also thinking opcache could be enabled by default, I see no reason not to have it at all.

And we could have apcu in the "installed but not enabled" extensions?

[sandrokeil]

Yes, intl is quite common these days. Enable opcache makes sense, because the code of the Lambda doesn't change and should not have any impact for development.

[bubba-h57]

Good calls! I updated the "Common Extension" list according to your comments.

[bubba-h57]

While working on that ^ commit, I realize that we cannot reasonably have extensions in the layer that are enabled, but can be disabled because it would require another layer to disable them. So instead, all the shared libraries are disabled, but can be enabled.

That commit also provides all the extensions that we have agreed upon in this issue.

[pmayet]

It is possible to enable zip ?

I think that additional documentation should be provided to explain how to add extensions and thus have its own layer.

[mnapoli]

Isn't that covered by zlib (which is installed)? I admit I'm not sure of the differences between the two.

[bubba-h57]

The zlib and the zip extensions are different, but the differences are subtle.

At a high level, think of zlib as handling gzip files and zip handling zip files.

It can get confusing because you will run across documentation that purports zlib can handle zip files. zlib can indeed handle zip files if, and only if, the zip file uses deflate compression.

However, while zlib has stuck to the deflate algorithm, zip files can support a much wider range of compression algorithms, even mixing compression in the same zip file, and it is not reliable to expect zlib to open a zip file.

Side Note: _Last Sept I had cause to implement my own zip library in pure python for streaming files from S3 out in a single zip file via lambda. Which meant I had to learn the zip standard. Before that, I would not have known myself._

[pmayet]

My need is to have the ZipArchive class provided by the zip extension for Laravel Excel.

That's the whole question, I have a specific need for an extension, do I generate my own layer or is it implemented in the base layer of Bref is part of the extensions disabled but can be enabled.

[bubba-h57]

At the moment, it is not in the layer at all and the developer would have to provide it themselves.

https://github.com/mnapoli/bref/blob/0.3/docs/environment/php.md

Is the best documentation on that at the moment, and it does not go into details. We probably need to rectify that. Open a new issue to get the process documented for v0.3 and I will get to it sometime this week.

[mnapoli]

@pmayet @bubba-h57 I've opened #173 to discuss that in a separate issue.

[mnapoli]

@bubba-h57 I think we can close this issue since all the extensions listed here are now installed and documented right?

[bubba-h57]

I concur.