Deployer: Creating extension/addon/plugin/shared recipe for Deployer

[coreyworrell]

| Q | A
| ----------------- | ---
| Issue Type | Question
| Deployer Version | N/A
| Local Machine OS | N/A
| Remote Machine OS | N/A

Description

What is best practice for creating "shareable" recipes or extensions for Deployer? We would like to include some common tasks and other functionality between several of our applications.

It seems that the deploy.php file is loaded within the context of the PHAR (or vendor if installed via composer?). So if we installed a composer package with custom tasks what would the require path need to be?

Thank you.

[antonmedv]

Just include in directly require('vendor/.../recipe.php');

Also I have plant to refactor it in v7 to more robust api. And use config from composer.json to configure include_path.

[coreyworrell]

Thank you, I will try it!

[coreyworrell]

I cannot figure out a good way to do this when installing globally via Composer.

composer global require deployer/deployer our/package

require 'recipe/common.php';
require 'vendor/our/package/something.php'; // does not resolve because global install
require '../../our/package/recipe/something.php'; // does not resolve

set_include_path(getenv('HOME').'/.composer/vendor'.PATH_SEPARATOR.get_include_path());
require 'our/package/recipe/something.php'; // now works but seems like a hack?

[antonmedv]

Why do you install packet globally?

[coreyworrell]

Because Deployer is not a dependency of the app but rather a dependency of our deployment process. It is being installed in Docker as part of CI process.

[antonmedv]

Hmmm, now I think only way is to set correct include_path or specify full path in require.

[jasperf]

@coreyworrell very interesting. Been reading on integrating Deployer in our Laradock setup for quite some time now. Main issue is the need to enter the Docker container workspace to execute PHP code. That and to stay there and to have the proper symlinks generated. I am happy to help testing and brainstorming this. Also store Deployer globally though for loading vendor data I often store deployer locally as well.

[antonmedv]

@jasperf can you share your workflow with laradock and deployer.

Also i'm working on code for exec inside docker.

[jasperf]

I currently have been using the following setup

• lslaravel Laravel setup with deploy.php in the root as here https://github.com/Larastudio/lslaravel ,
• lsdock or Laradock setup as here https://github.com/Larastudio/lsdock and
• Docker Base for provisioning the Ubuntu 18.0.4 server with Docker, Docker Compose, sudo user, Swap File and UFW https://github.com/Larastudio/docker-base

Current _deploy.php_ cannot be used for Laradock but works on staging and production using a standard Laravel Forge based LEMP or LEP + MariaDB Server. This due to lack of PHP on Docker Host: Ubuntu Server (no need for PHP there as it runs in a container).

With my Docker setup I now simply work with Git repo versions and git updates on the server for the Laravel directory and Laradock directory. This as Deployer does not work properly due to wrong symlinks and the problem with not locating PHP as it is only accessible from a container.

I do however know we can run Docker commands from the host to get into the container and run commands that need php like docker-compose -f prod-docker-compose.yml exec workspace bash -l -c "php artisan list":

web@laradock-setup:~/lsdock$ docker-compose  -f prod-docker-compose.yml exec workspace  bash -l -c "php artisan list"
Laravel Framework 5.7.19

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
      --env[=ENV]       The environment the command should run under
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands:
  clear-compiled       Remove the compiled class file
  down                 Put the application into maintenance mode
  dump-server          Start the dump server to collect dump information.
  env                  Display the current framework environment
  help                 Displays help for a command
  inspire              Display an inspiring quote
  list                 Lists commands
  migrate              Run the database migrations
  optimize             Cache the framework bootstrap files
  preset               Swap the front-end scaffolding for the application
  serve                Serve the application on the PHP development server
  tinker               Interact with your application
  up                   Bring the application out of maintenance mode
 app
  app:name             Set the application namespace
 auth
  auth:clear-resets    Flush expired password reset tokens
 cache

  ------------------------------------- cut ---------------------

 route
  route:cache          Create a route cache file for faster route registration
  route:clear          Remove the route cache file
  route:list           List all registered routes
 schedule
  schedule:run         Run the scheduled commands
 session
  session:table        Create a migration for the session database table
 storage
  storage:link         Create a symbolic link from "public/storage" to "storage/app/public"
 vendor
  vendor:publish       Publish any publishable assets from vendor packages
 view
  view:cache           Compile all of the application's Blade templates
  view:clear           Clear all compiled view files

So if we can make a recipe or adjusted Laravel recipe that wraps command for PHP running container entry (which each user can adjust the path for depending on setup, we would be where we want to be. To adjust Deployer code for it might be a lot of work and or not necessary.

[jasperf]

@jasperf can you share your workflow with laradock and deployer.

Also i'm working on code for exec inside docker.

Any updates on docker-compose exec or docker exec to work with PHP from a container @antonmedv ?

[antonmedv]

@jasperf working on it :)

[jasperf]

@jasperf working on it :)

Great @antonmedv . Planning on doing a meetup in Bangkok on Docker setup with Ansible, Laradock and Laravel preferably using Deployer for deployment in like a week. Would be great if I we can get Deployer to work with Docker / Laradock by then ;-)

[antonmedv]

Can't promise anythink, but I'm going to put more time on deployer this week.

[jasperf]

@antonmedv Decided to move the meetup as I have not found a decent location yet AND... as I really would love to include deployer in the talk. Let me know what progress you made so far. Perhaps I can chip in with some ideas.

[antonmedv]

Now working on new API for executing inside containers on remote hosts. What ideas do you have?

[jasperf]

Well, the idea of a wrapper to wrap most if not all PHP dependent commands in a wrapper code is a good idea I think. So a wrapper where the data in quotes like php artisan list in docker-compose -f prod-docker-compose.yml exec workspace bash -l -c "php artisan list" is a way to do this.

Docker vs Docker-Compose

In my example I am using docker-compose and some may use docker only. Using docker does work as well however. And it does not depend on a docker-compose.yml. As long as you use the container name and not alias / network name. So let's use docker as command.

Variables

The container to aim at may have a different name. This means this should be a variable. Also the program to run php commands with in the container can vary a bit. So here you have two variables we need to go with the wrapper:

• container name
• program to run commands with in container: bash or ash often

Example

Example command using Docker with variable smt-docker_workspace_1 as container name and bash as program / shell to run in container:
docker exec smt-docker_workspace_1 bash -l -c "php artisan list"

[antonmedv]

I see. I'm working on both, docker and docker-compose integrations.

[jasperf]

I see. I'm working on both, docker and docker-compose integrations.

Well, awesome. That would even be more flexible and will allow a user like me to use docker-compose and refer to a yml file and so use the easier to use workspace as container indicator instead of the official container name. Looking forward to seeing a development branch on this!

[jasperf]

@antonmedv Will go ahead with meetup tomorrow. Does deployer work with Docker yet?

[antonmedv]

Now working curently.

[jasperf]

Great to hear that @antonmedv . Let me know if there is branch I can use for testing .

[vesper8]

@antonmedv is this now available on the master branch? really interested to get it working the same way @jasperf is describing

[kOld]

Hello guys, I found some tricks we can use with the current 6.* version and laradock, here is how:

host('SERVER-IP')
    ->set('deploy_path', '/path/to/deployer')
    ->set('bin/php', 'cd /path/to/laradock/ && docker-compose exec -T workspace php')
    ->set('bin/composer', 'cd /path/to/laradock/ && docker-compose exec -T workspace composer -d={{release_path}}');

I have used this configuration for the last months and works great besides one very frustrating thing, I need to restart the containers to get the new code, for that I use this task:

task('docker:restart', function () {
      run("cd /path/to/laradock/ && docker-compose down && docker-compose up -d nginx mariadb redis workspace laravel-horizon");
});

I think the reason why the docker volume isn't updating the code inside the containers is because we are git pulling directly at the machine level and not inside the workspace container.

Any ideas how to overcome this problem with the git pull and docker volumes?

I have tested the various sync flags associated with the docker volume without any success

[jasperf]

@kOld Very strange it does not load changes. That is not needed locally when changes to files are made which are stored on a volume. We do not even need to reload / restart. So it remains odd. I mean, I can pull changes locally and they are loaded, only need to do npm run production for assets in our workflow.

If we do need to reload stuff, wasn't adocker-compose up -d --build application-container-name better? See SO thread here. Restart does not load new image data. Definitely would not stop and start all containers anyways. But I also hate to stop a live application by stopping and starting the application container so it loads the new data.

As a side note I am also considering moving the Laravel application into a separate container. Something like Hackernoon uses.

app:
    build:
      context: .
      dockerfile: Dockerfile
    networks:
      - network
    links:
      - postgres
      - elasticsearch
#    logging:
#      driver: awslogs
#      options:
#        awslogs-region: "us-east-1"
#        awslogs-group: "laravel"
#        awslogs-create-group: "true"
    env_file:
        - docker-compose.env

but tweaked some. And then use docker deploy and a registry or docker-machine instead. May also be an option. Only now with current setup (standard Laradock setup app wise) the Laravel application is part of the web server container. And the app is loaded from the volume.

[kOld]

@jasperf found the problem related to the code update, it was associated with the php-fpm and system links (https://stackoverflow.com/questions/37318795/why-does-envoyer-need-to-restart-php-fpm-when-deploying)

Basically you need to instruct the fpm to reload after changing your current system link to the new release.

It was a bit tricky to get it work with laradock, but this seems to work fine:

set('php_fpm_service', 'php7.2-fpm');
set('php_fpm_command', 'echo "" | sudo -S /usr/sbin/service {{php_fpm_service}} reload'); // for setups like laravel forge
set('php_fpm_command', "kill -USR2 "$(ps -ef | grep '[p]hp-fpm: master' | awk '{print $2}')""); // for docker based setups with laradock or similar

task('fpm:reload', function () {
run('{{php_fpm_command}}');
})->desc('Reload the php-fpm service');

[jasperf]

Thanks for that @kOld . Will try this out as soon as possible. And I know many here will appreciate you sharing this. Mentioned your tip at https://imwz.io/laradock-production-setup/ as someone got stuck as well. Props kOld, of course.

[jasperf]

@antonmedv Any possibility of implementing the code to deploy to Docker containers based on this thread?

[antonmedv]

Yes, this is definitely possible. Maybe we can accumulate this thread into PR? This will be much simpler.

[jasperf]

@antonmedv well perhaps we can use the code give by @kOld

set('php_fpm_service', 'php7.2-fpm');
set('php_fpm_command', 'echo "" | sudo -S /usr/sbin/service {{php_fpm_service}} reload'); // for setups like laravel forge
set('php_fpm_command', "kill -USR2 "$(ps -ef | grep '[p]hp-fpm: master' | awk '{print $2}')""); // for docker based setups with laradock or similar

task('fpm:reload', function () {
run('{{php_fpm_command}}');
})->desc('Reload the php-fpm service');

And add it as a new recipe. One based on the laravel one and perhaps call it laradock. Will see if I can work on a Deployer clone with https://github.com/deployphp/deployer/blob/master/recipe/laradock.php Will need some time as I also do this besides other ventures mind you..

[jasperf]

Started working on it at https://github.com/Larastudio/deployer/blob/master/recipe/laradock.php (a fork of Deployer). Will need testing on several OS to see how this goes also so any help is appreciated. Did not do a pull request yet as I am just starting out. But leave a comment here so others can help out with code and or testing.

[antonmedv]

Now recipes for third-party live in https://github.com/deployphp/deployer/tree/master/contrib

[coreyworrell]

@antonmedv I'm not sure what those third party recipes have to do with this issue? It did get sidetracked with random Docker stuff, which it was never meant to be about.. The original issue was how to include a recipe from a globally installed Composer package.

[antonmedv]

Ahh, okay i see. This issue got little bit cumbersome. Did you find out solution on how to include it?