Bref: PHP Layer(s) Runtime Pipeline for v3.0

[bubba-h57]

We are opening this issue for discussion pertaining to how bref builds and publishes runtimes for the supported layers. We are prepared to provide a pull request implementing all features discussed here.

• [x] Consolidate Build Process
• [x] Implement Build Pipeline
• [x] Use mult-stage docker builds where possible
• [x] Use the Offical AWS Docker Image for our build
• [x] Move the bref Layer(s) from /opt to opt/bref
• [x] Update the PHP Build to no longer use the _[deprecated]_ bundled libzip
• [x] Update the Layer(s) to OPENSSL 1.1.1a
• [x] Update the Layer(s) to CURL 7.63.0 w/OPENSSL 1.1.1a
• [x] Update the Layer(s) to LIBSSH 1.8.0
• [x] Update the Layer(s) to JPEG-TURBO 2.0.1
• [x] Update the Layer(s) to PNG16 1.6.36
• [x] Update the Layer(s) to WEBP 1.0.1
• [x] Update the Layer(s), add the Sodium Extension
• [x] Update the Layer(s), add IMAGEMAGICK 7.0.8-20
• [x] Update the Layer(s), add the IMAGICK Extension
• [x] Update the Layer(s), add the pdo-pgsl and pgsl Extensions
• [x] Compile PHP to natively load Configuration File /opt/bref/etc/php/php.ini
• [x] Compile PHP to natively scan /opt/bref/etc/php/config.d:/var/task/php/config.d for additional .ini files
• [x] Strip all shared libraries and executables to save as much disk space as possible

Consolidate Build Process

bref currently has two separate process, default and fpm, for compiling PHP. Because the processes are separate, we incur ongoing efforts to keep them synchronized. We will be less prone to future errors in synchronization if we consolidate the process so that we build everything we need once, with the precise same configuration.

Implement Build Pipeline

Refactoring the build process into a pipeline will help with ongoing maintenance by separating the build process into logical components that rely upon each other.

Multi-Stage Docker Builds

The amount of work proposed can potentially extend well beyond the layers allowed in an image. Multi-stage builds will solve that for us.

Official AWS Docker Image

Lambda tasks use the amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2 AMI, as documented under the AWS Lambda Runtimes. AWS has kindly provided us with it as a base docker image.

By starting with this minimal image, we can both ensure that we have nothing extraneous in the build system that will not be in the production system; as well as have confidence that all our libraries and executables are built as faithfully for the environment as possible.

Move the bref Layer(s) from /opt to opt/bref

AWS expects library dependencies to be placed in a folder supported by our runtime, bref. They do provide /opt/bin and /opt/lib for general use, and both are pre-populated in their respective environment variables (PATH and LD_LIBRARY_PATH).

We will compile and link our own binaries against our own libraries, and do it in such a way that it will definitely have a negative impact on default, older binaries, that might get confused when linking against the default LD_LIBRARY_PATH. It is better for our libraries to not go in there, where they can potentially be hijacked by other layers loaded after our own.

We propose a resulting filesystem layout like so:

bash-4.2# cd /opt/bref
bash-4.2# ls -al
total 40
drwxr-xr-x 1 root root 4096 Dec 29 19:05 .
drwxr-xr-x 1 root root 4096 Dec 29 18:21 ..
drwxr-xr-x 1 root root 4096 Dec 29 19:05 bin
drwxr-xr-x 1 root root 4096 Dec 29 19:01 etc
drwxr-xr-x 1 root root 4096 Dec 29 19:05 lib
drwxr-xr-x 1 root root 4096 Dec 29 19:05 lib64
drwxr-xr-x 4 root root 4096 Dec 29 18:53 php
drwxr-xr-x 1 root root 4096 Dec 29 19:05 sbin
drwxr-xr-x 1 root root 4096 Dec 29 19:05 share
drwxr-xr-x 5 root root 4096 Dec 29 18:40 ssl

If we really want to put our executables in the /opt/bin directory, we can simply create symlinks there from /opt/bref/bin/* and /opt/bref/sbin/*. Given that we own the bootstrap file, we can also simply add our own bin directories to the path and be done with it.

Strip all shared libraries and executables to save as much disk space as possible

Space is always a concern with AWS Lambda. We cannot upload any artifacts greater than 50MB when zipped and the combined size of the function (/var/task) code and any layers (/opt) used may not exceed 250MB.

A function can use up to 5 layers at a time. The total unzipped size of the function and all layers can't exceed the unzipped deployment package size limit of 250 MB. For more information, see AWS Lambda Limits.

It makes sense to carefully mind the sizes of our executables, libraries, and layers. Our goal is to keep our primary layer under 100MB when unzipped. Leaving developers with 150MB of space to use as they will in /var/task. We propose that if anyone is deploying Lambda Functions that use more than 150MB of space, they might want to rethink how and why they are using Lambda. :-)

PHP Configuration

By telling PHP about configuration settings at compile time, we can avoid having to specify things like that on the command line. We propose to natively load the /opt/bref/etc/php/php.ini file and then have PHP scan /opt/*bref*/etc/php/config.d and /var/task/php/config.d in that order for additional configuration settings.

This allows bref to easily set some sane defaults, but allows the developer to override that as they see fit in the /var/task/php/config.d. It also means that we do not have to concern ourselves with how the developer wants the system configured.

Let's say they have a custom extension, custom.so they intend to use. They can simply package it in their function code, and add the appropriate settings in the /var/task/php/config.d/custom.ini file. When we bootstrap, PHP will automagically pick that up for us.

Libraries and Extensions

We see a number of requests to add extensions to those supported by bref, and it seems like a great time to go ahead and compile those extensions in. There are also a number of libraries/extensions that are outdated and could stand to be updated for compliance reasons if no other.

For example, openssl being a few years out of date is a deal killer for many of our clients who have particular compliance requirements. They do not care that we might not even be using it. If it is installed, and the version is too old, it breaks compliance. So, updating it is a boon there.

We also do a lot of image and document processing. Having the latest image libraries (png, jpeg, webp) and access to a healthy ImageMagick via the imagick extensions is a great benefit to us, and we expect that to be useful by anyone else manipulating images, in php, via Lambda.

Current bref Extensions
Core ctype curl date dom fileinfo filter ftp gd hash iconv intl json	libxml mbstring mongodb mysqlnd openssl pcntl pcre PDO pdo_mysql pdo_sqlite Phar posix redis	Reflection session SimpleXML soap SPL sqlite3 standard tokenizer xml xmlreader xmlwriter zip zlib

 
 
 

Proposed Added Extensions
apcu bcmath calendar exif gmp	igbinary imagick memcached mysqli pdo_pgsql	pgsql readline sodium

Summary

We are prepared to provide a pull request against the v3.0 branch implementing all of this if it is welcome. We propose to simply add it all under /bref/runtime/experimental so that it conflicts with nothing else in place, and provide everyone an opportunity to pull it, play with it, and beat it up.

[mnapoli]

Hi, sorry for not answering sooner I had very poor internet access for the holidays. Back to work, that sounds awesome!

First let me say :+1: on everything by default.

A quick things to note:

• if you could aim for one pull request per change (or at least group very related changes together) that would be great: code/documentation review in an open source project is the bottleneck. Don't be afraid to stack branches on top of each other, I'll manage to review the PRs anyway.
• regarding the build pipeline and multi-stage Docker build that sounds great but I definitely don't want to overcomplicate things. Maybe what you suggest will be simpler and better, but at the moment I'm cautious. Let's give it a try? (ideally in a separate PR so that we can see if it helps?)

Move the bref Layer(s) from /opt to opt/bref

I'm wary about that too. For shared libraries what you suggest sounds really good. For the binaries I would aim for it to work by default, even if we override bootstrap (which has good use cases).

PHP Configuration

See the WIP documentation, it is already possible to create and load a extra php.ini.

PHP extensions

That sounds great too. Do you have any idea how we can provide a lot of extensions yet still keep the layer as light as possible? This is the main issue I see for now.

We propose to simply add it all under /bref/runtime/experimental

I think you can go ahead and propose the changes directly against the existing runtime in v0.3 since this is a new version that breaks compatibility completely with the previous version.

---

How about we discuss the topics that need discussion in separate issues?

[bubba-h57]

Sure, I have already done all the work though. Just waiting for a go ahead on the pull request. How about I just provide one PR with all of it in the experimental dir? That way we could go ahead and merge out without impacting anything in place, then we could break it down from there? Or, you could take a look at our fork directly and suggest what you would me to do?

https://github.com/stechstudio/bref the experimental branch.

[mnapoli]

Awesome! Open the PR then, we’ll discuss inline the diff.
Le 1 janv. 2019 à 22:39 +0100, Bubba [e-mail], a écrit :

Sure, I have already done all the work though. Just waiting for a go ahead on the pull request. How about I just provide one PR with all of it in the experimental dir? That way we could go ahead and merge out without impacting anything in place, then we could break it down from there? Or, you could take a look at our fork directly and suggest what you would me to do?
https://github.com/stechstudio/bref the experimental branch.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[sandrokeil]

Regarding to PHP extensions. Would it make sense to provide only some main PHP extensions like the official PHP Docker image and use this as a base layer? So user can add it's own extensions with dependencies if necessary. Because you have to update the layer if an extension was upgraded like mongodb driver. To much maintenance and it's complicated to keep in sync.

[bubba-h57]

@sandrokeil The way the layers work, the developer can update any of the extensions _(that are not compiled into PHP itself)_ at any time. Let's take the mongodb package and say that the version built into the layer is 1.5.3 but the developer really wants to use the 1.6.0alpha1 version.

After compiling the 1.6.0alpha1 PECL against PHP in the bref layer, they create their own layer to hold their extensions. Simply putting mongodb.so in the bref/lib/php/extensions/no-debug-non-zts-20180731 directory. The nature of layers is that when the developers layer is installed over the bref layer, the developers `mongodb.so' would literally overwrite the base one.

This allows us to support any number of extensions baked into the bref layer without the complication or maintenance of attempting to keep them all in sync with the latest versions of everything. It allows the developer the freedom to either upgrade or downgrade extensions at thier own pleasure.

It is just as easy for them to completly remove extensions from the bref layer. Simply override the appropriate /opt/bref/etc/php/config.d/extension.ini by replacing it with a blank file or something.

[mnapoli]

@bubba-h57 what do you think about the points I have raised here https://github.com/mnapoli/bref/issues/129#issuecomment-450750264 (except those related to how we deal with pull requests)?

[bubba-h57]

Move the bref Layer(s) from /opt to opt/bref
I believe the main point here was having binaries work by default, which I take to mean that we want to ensure the binaries end up in the /opt/bin directory, with is in the Lambda Task $PATH by default.

I reckon it is a fine thing to do. https://github.com/mnapoli/bref/blob/9ea33c127d7ff2055c5b1f18266e7ad7904ebb41/runtime/experimental/export.Dockerfile#L23-L25

So I went ahead and create the symlinks from all our binaries to it. Then when we zip
https://github.com/mnapoli/bref/blob/9ea33c127d7ff2055c5b1f18266e7ad7904ebb41/runtime/experimental/export.Dockerfile#L28
you can see that we store the symbolic links as the link instead of the referenced file via the -y flag.

PHP Configuration
Yes, the current build system defines
https://github.com/mnapoli/bref/blob/7c0167804902c02e9ec9d9f40d2b92e7e10a3f1a/runtime/default/build.sh#L52

and the developer can modify that with PHP_INI_SCAN_DIR. This follows a Configuration over Convention design paradigm. I am proposing a Convention over Configuration paradigm.

By setting:
https://github.com/mnapoli/bref/blob/9ea33c127d7ff2055c5b1f18266e7ad7904ebb41/runtime/experimental/makefiles/php.mk#L19-L20

PHP will load the /opt/bref/etc/php/php.ini, then it will load all /opt/bref/etc/php/config.d/*.ini files, followed by all /var/task/php/config.d/*.inifiles as configuration files.

By convention, the user can simply put their configuration settings in /var/task/php/config.d/00-php.ini to override anything in the previous two directories. However, if they still prefer to, they can configure the PHP_INI_SCAN_DIR as they see fit.

It just comes down the design paradigm we prefer. I vote for convention over configuration, but it is not a hill would die on. :-)

PHP extensions
@sandrokeil really got me pondering this. I believe we really want to avoid the developer having to mess around with lambda layers. If we do a good job, they just configure the lambda task itself.

So we want a way for them to easily manage extensions from the /var/task directory.

We have built-in extensions _(like mbstring, opcache, curl, etc.)_ and we have external extensions _(igbinary, mongodp, redis, etc)_. Perhaps we want to move as many of the built-in's to external as possible, because I do not know how to disable a built-in extension.

As far as the external extensions, we provide as many as wel like in /opt/bref/lib/php/extensions/no-debug-non-zts-20180731/*.so and simply not enable them in /opt/bref/etc/php/config.d/*.ini. Then the developer can simply enable them via their own /var/task/php/config.d/00-php.ini

The size of the extensions themselves range from apcu.so at 100K to mongo.so at 1.4M. If we are happy setting the maximum unzipped target size of our layers to 100M, the current php-default-7.3.0.zip that I am building results in an unzipped layer of 63MB ... we have a lot of room to add a lot more extensions to our layer and let the developer enable them as they see fit.

I can imagine some alternatives, but all of the alternatives I imagine result in some horrendous maintenance nightmares down the street. The kind of dreams that keep me up at night. :-)

For example, we could compile/create a PHP layer, call it layer:1 with little to no extensions. Then compile all the plausible extensions we want to provide against that PHP binary and store those extensions in S3 somewhere. If a developer wants to run layer:1 with some of the extensions, they just configure bref to download and setup the correct ones in thier function, and they would then upload the extension.so in their function package.

But eventually, we get to layer:7 ... and we need to ensure that we have the layer:7 compatible extensions and not confuse them with the layer:1 extensions.

I like the idea of providing as many extensions as possible, in a nice fat layer, that 85% of our users would likely use. Then provide a stripped layer for the 15% that are really interested in micro managing the extensions themselves.

Conclusion
I think that covered responding to all the points. Did I miss anything?

[sandrokeil]

For example, we could compile/create a PHP layer, call it layer:1 with little to no extensions.

I like this idea, because some extensions have system dependencies that I don't want to have. It's even possible to have a system package conflict depending on the extension. As I said earlier, I prefer the Docker image way. Minimal PHP image/layer and users add their own extensions. Think on some industry validation processes like PCI (Payment Card Industry Data Security Standard) where you have to know your system dependencies and versions. A minimal PHP layer would also reduce attack surface.

I like the idea of providing as many extensions as possible, in a nice fat layer, that 85% of our users would likely use.

This sounds also reasonable for prototyping or to try things out.

Hopefully we can achieve both.

[mnapoli]

Move the bref Layer(s) from /opt to opt/bref

So I went ahead and create the symlinks from all our binaries to it.

Awesome! Nothing else to add here then :)

PHP Configuration

I am proposing a Convention over Configuration paradigm.

Awesome I now see what you mean. I think this is a very good solution. The only thing left that is bothering me is that we are deciding the name of the directory for every users out there, and changing it later will be a BC break.

If all frameworks had the same config/ (or etc/ or whatever) directory for configuration then we could settle on this, but I don't think it's that simple.

I'm thinking the only "safe" way to handle this is to name the directory .bref/, that way it's much clearer. And that actually makes sense if we look at Travis, GitHub, Platform.sh, etc. The directory is the name of the vendor.

So what about --with-config-file-scan-dir=${TARGET}/etc/php/config.d:/var/task/.bref/php ? (I'd skip config.d to avoid having too many nested directories, the goal is to make it simple for users)

In the docs that would be:

PHP will load any *.ini file found in .bref/php/ in your project.

PHP extensions

I'll open a separate issue, this is an interesting problem ^^

[mnapoli]

PHP extensions -> #143

[bubba-h57]

I am going to close this issue.

The pull request has progressed well past this.