V8-archive: Deploy on Google AppEngine

[carolin-skiply]

Bug Report

I got the following error when I deploy directus on Google AppEngine Standard Env :
23/Nov/2018:15:11:44 +0000 [ERROR 0 /index.php] PHP message: PHP Fatal error: Uncaught Error: Call to undefined function Directus\get_api_project_from_request() in /srv/src/web.php:16 Stack trace: #0 /srv/public/index.php(3): require() #1 {main} thrown in /srv/src/web.php on line 16

Steps to Reproduce

1- I created an app.yaml file as follow :
`service: test-directus
runtime: php72

handlers:

• url: .*
  script: auto`

2- I created an composer.json file as follow:
'"autoload": {
"psr-4": { "": "vendor/" }
}
}'

3- I deploy my app on AppEngine:
'gcloud app deploy --project MY_PROJECT'

4- I go to the webpage and got the error above in my console (gcloud)

Expected Behavior

index page to display

Actual Behavior

HTTP ERROR 500

Technical Details

• Device: Desktop
• OS: ?
• Browser: Chrome
• Directus App: 7.0.0
• Install Method: cloned master branch

[vzool]

@carolin-skiply Which App Engine environment you working on, is it "Standard" or "Flexible" environment?

https://cloud.google.com/appengine/docs/php/

[carolin-skiply]

@vzool on Standard env as mentionned in my post.

[vzool]

Never noticed it there ^_^ , So which repo you cloned directus/api or directus/directus?

[carolin-skiply]

No problem.
I cloned : directus/directus

[vzool]

I think you need to use Flex environment instead of Standard one, so try to use these config for app.yaml instead of old one:

runtime: php
env: flex

runtime_config:
  document_root: public

## Uncomment for Cloud SQL
# beta_settings:
#   cloud_sql_instances: "INSTANCE_CONNECTION_NAME"

# This sample incurs costs to run on the App Engine flexible environment.
# The settings below are to reduce costs during testing and are not appropriate
# for production use. For more information, see:
# https://cloud.google.com/appengine/docs/flexible/python/configuring-your-app-with-app-yaml
manual_scaling:
  instances: 1
resources:
  cpu: 1
  memory_gb: 0.5
  disk_size_gb: 10

Then, do this command to deploy your app:

> gcloud app deploy --project YOUR_PROJECT_ID

Remember to revise these links for further instructions:

• https://cloud.google.com/appengine/docs/flexible/php/quickstart
• https://github.com/GoogleCloudPlatform/getting-started-php

Hope it may helps you ^_^

[carolin-skiply]

Thanks.
I just tried that but it failed.

1- I copy into my app.yaml the code you gave me and then deploy.

here is my app.yaml:
```runtime: php
env: flex

runtime_config:
document_root: public

Uncomment for Cloud SQL

beta_settings:`

cloud_sql_instances: "INSTANCE_CONNECTION_NAME"

This sample incurs costs to run on the App Engine flexible environment.

The settings below are to reduce costs during testing and are not appropriate

for production use. For more information, see:

https://cloud.google.com/appengine/docs/flexible/python/configuring-your-app-with-app-yaml

manual_scaling:

instances: 1
resources:
cpu: 1
memory_gb: 0.5
disk_size_gb: 10```

I got an unknown error :
error building image: error building stage: waiting for process to exit: exit status 1

[carolin-skiply]

And thanks for the link but I've gone through those and still not working.

I still think that I need a composer.json file to be able to build my project on AppEngine (Standard or Flex).
Do you have a shareable version ?

[carolin-skiply]

Ok. I deploy again within my standard env and a composer.json that I created from the installed.json found in vendor/compose.
But I got the same error in but report : PHP message: PHP Fatal error: Uncaught Error: Call to undefined function Directus\get_api_project_from_request() in /srv/src/web.php:16

[benhaynes]

Is this the file you're looking for?

https://github.com/directus/api/blob/master/composer.json

[wellingguzman]

@carolin-skiply I am not experienced with the Google AppEngine, but it seems that the composer autoloader it isn't working. Are you getting any logs or error trying to include the vendor/autoload.php?

That's all I can chip in at the moment, I will give this a look when I have access to a google AppEngine, but at the moment, someone with experience working with Google AppEngine and php+composer will be more suitable to help here.

[vzool]

@carolin-skiply Flex environment is not compalint about composer anymore and it does working on my side. So, first: clone a fresh copy of directus/directus

> git clone https://github.com/directus/directus gae-directus

Then, go to inside gae-directus and create app.yaml

> cd gae-directus
> touch app.yaml

And add the following to the app.yaml:

runtime: php
env: flex

runtime_config:
  document_root: public

## Uncomment for Cloud SQL
# beta_settings:
#   cloud_sql_instances: "INSTANCE_CONNECTION_NAME"

# This sample incurs costs to run on the App Engine flexible environment.
# The settings below are to reduce costs during testing and are not appropriate
# for production use. For more information, see:
# https://cloud.google.com/appengine/docs/flexible/python/configuring-your-app-with-app-yaml
manual_scaling:
  instances: 1
resources:
  cpu: 1
  memory_gb: 0.5
  disk_size_gb: 10

Finally, deploy your app:

> gcloud app deploy --project YOUR_PROJECT_ID

I did delopy an unconfigured copy at https://rae-directus.appspot.com/ but I think normal installation will not work there. So, I guess Manual config may work.

Thanks ^_^

[carolin-skiply]

Thanks all !

@vzool I did as you told me and it finally works! Thanks (I was on this for days).
So now that I finally got it working (thanks to you) : do you know how I can configure my directus instance : /admin does not seem to load (but no issue in logs :( )

I got this on /admin :(
{"error":{"code":14,"message":"This Directus API instance has not been configured. Install via the Directus App (eg: \/admin) or read more about configuration at: https:\/\/docs.directus.io\/install.html#configure"}}

[rijkvanzanten]

@carolin-skiply Did you deploy in a folder perhaps? Does apache's mod_rewrite work on app engine? Based on the repo the admin is in fact included in the code

[carolin-skiply]

@rijkvanzanten I don't think that it was deploy in a folder but I cannot see the source on AppEngine :(
It's possible to simulate a apache's mod_rewrite in necessary : https://cloud.google.com/appengine/docs/standard/php/config/mod_rewrite

[carolin-skiply]

Also I found this : https://cloud.google.com/appengine/docs/flexible/php/runtime#configuring_supervisord_in_the_php_runtime

I might need to set some additionnal config to be able to use Directus on AppEngine as user to www-data.
Is there any other conf to add ?

[carolin-skiply]

Ok little update here !
I configure manually my instance using the following :
```

return [
'app' => [
'env' => 'development',
'timezone' => 'America/New_York',
],

'settings' => [
    'logger' => [
        'path' => __DIR__ . '/logs/app.log',
    ],
],

'database' => [
    'type' => 'mysql',
    'host' => 'xxx.xxx.xxx.xxx',
    'port' => 3306,
    'name' => 'DATABASE',
    'username' => 'USER',
    'password' => 'PASSWORD',
    'engine' => 'InnoDB',
    'charset' => 'utf8mb4',
    // When using unix socket to connect to the database the host attribute should be removed
    // 'socket' => '/var/lib/mysql/mysql.sock',
],

'cache' => [
    'enabled' => false,
    'response_ttl' => 3600, // seconds
    // 'pool' => [
    //    'adapter' => 'apc'
    // ],
    // 'pool' => [
    //    'adapter' => 'apcu'
    // ],
    // 'pool' => [
    //    'adapter' => 'filesystem',
    //    'path' => 'cache', // relative to the api directory
    // ],
    // 'pool' => [
    //    'adapter'   => 'memcached',
    //    'host'      => 'localhost',
    //    'port'      => 11211
    // ],
    // 'pool' => [
    //    'adapter'   => 'redis',
    //    'host'      => 'localhost',
    //    'port'      => 6379
    // ],
],

'storage' => [
    'adapter' => 'local',
    // The storage root is the directus root directory.
    // All path are relative to the storage root when the path is not starting with a forward slash.
    // By default the uploads directory is located at the directus public root
    // An absolute path can be used as alternative.
    'root' => 'public/uploads/_/originals',
    // This is the url where all the media will be pointing to
    // here is where Directus will assume all assets will be accessed
    // Ex: (yourdomain)/uploads/_/originals
    'root_url' => '/uploads/_/originals',
    // Same as "root", but for the thumbnails
    'thumb_root' => 'public/uploads/_/thumbnails',
    //   'key'    => 's3-key',
    //   'secret' => 's3-secret',
    //   'region' => 's3-region',
    //   'version' => 's3-version',
    //   'bucket' => 's3-bucket'
],

'mail' => [
    'default' => [
        'transport' => 'sendmail',
        'from' => '[email protected]'
    ],
],

'cors' => [
    'enabled' => true,
    'origin' => ['*'],
    'methods' => [
        'GET',
        'POST',
        'PUT',
        'PATCH',
        'DELETE',
        'HEAD',
    ],
    'headers' => [],
    'exposed_headers' => [],
    'max_age' => null, // in seconds
    'credentials' => false,
],

'rate_limit' => [
    'enabled' => false,
    'limit' => 100, // number of request
    'interval' => 60, // seconds
    'adapter' => 'redis',
    'host' => '127.0.0.1',
    'port' => 6379,
    'timeout' => 10
],

'hooks' => [
    'actions' => [],
    'filters' => [],
],

'feedback' => [
    'token' => 'XXX',
    'login' => true
],

// These tables will not be loaded in the directus schema
'tableBlacklist' => [],

'auth' => [
    'secret_key' => 'XXX',
    'public_key' => 'XXX',
    'social_providers' => [
        // 'okta' => [
        //     'client_id' => '',
        //     'client_secret' => '',
        //     'base_url' => 'https://dev-000000.oktapreview.com/oauth2/default'
        // ],
        // 'github' => [
        //     'client_id' => '',
        //     'client_secret' => ''
        // ],
        // 'facebook' => [
        //     'client_id'          => '',
        //     'client_secret'      => '',
        //     'graph_api_version'  => 'v2.8',
        // ],
        // 'google' => [
        //     'client_id'       => '',
        //     'client_secret'   => '',
        //     'hosted_domain'   => '*',
        // ],
        // 'twitter' => [
        //     'identifier'   => '',
        //     'secret'       => ''
        // ]
    ]
],

];

And add/create an [additional-supervisord.conf](https://cloud.google.com/appengine/docs/flexible/php/runtime#configuring_supervisord_in_the_php_runtime) file to set user to www-data on AppEngine Flex Env :

[program:test-directus]
command = php %(ENV_APP_DIR)s/index.php
user = www-data
```

And now I got :
{"error":{"error":8,"message":"API Environment Configuration Not Found: admin"}}

Any idea ?

[vzool]

Ok little update here !
I configure manually my instance using ...

Very good 😃

Any idea ?

I think you need to configure the database, migrate the database then create admin user.
Can you access your instance by SSH? ^_^

https://cloud.google.com/sdk/gcloud/reference/app/instances/ssh

[carolin-skiply]

@vzool Hi Thanks.
I've already all the table in my database : I created it myself using this on my local instance which use the same database; and this to create the user.

[vzool]

@carolin-skiply You're very welcome, good to hear that :+1:
BTW, you don't need supervisord for directus/api or directus/directus, so you can disable it to gain more performance with the same budget. :kissing_heart:

[carolin-skiply]

@vzool great ! Thanks!
However it's working on my local server but I still got the error on my google AppEngine :(
{"error":{"error":8,"message":"API Environment Configuration Not Found: admin"}}

[vzool]

I thought you copied config/api_sample.php into config/api.php.
Have you done that before?

[carolin-skiply]

Actually I did this on my local instance and then I push it onto AppEngine

[vzool]

Actually I did this on my local instance and then I push it onto AppEngine

That's better than I suggested to copied by yourself directly because Directus will update many keys automatically, so after you did this have you see config/api.php file generated in your local files before deploy?

[carolin-skiply]

Yes. It generated an api.php file in config folder...

FYI, my database in a google CloudSQL instance.

[vzool]

According to this, api.php file should be api.admin.php or api._.php, which depended on your "Project Name URI". Try to rename it to api._.php or api.admin.php.

[carolin-skiply]

Alright I change my config/api.php by config/api.admin.php and now I'm back on this error :
{"error":{"code":14,"message":"This Directus API instance has not been configured. Install via the Directus App (eg: \/admin) or read more about configuration at: https:\/\/docs.directus.io\/install.html#configure"}}

but I think it's normal because here it says : I should have a default app/project with an config/api.php and the following app/project with api._.php

[vzool]

FYI, my database in a google CloudSQL instance.

So, you need to update app.yaml, update config/api.php then access your instance by SSH to do this.

BTW, at the end "Flex Environment" is a docker image and you can logged to your instance by SSH.

After that you can install your own MySQL database. ^_^

[carolin-skiply]

@vzool Ok. I'll upload a clean image of Directus and then I'll try to generate the config/api.php from my instance using SSH.

However I don't want to create a local database but I want to use my cloudSQL database.

[carolin-skiply]

@vzool So I'm stuck on bin/directus install:database

with this error : PHP Fatal error: Uncaught InvalidArgumentException: There was a problem connecting to the database: SQLSTATE[HY 000] [2002] Connection timed out in /app/vendor/robmorgan/phinx/src/Phinx/Db/Adapter/MysqlAdapter.php:115

It seems that I can't connect to my database CloudSQL :(

[carolin-skiply]

unix_socket sould look like this to connect to CloudSQL db from an AppEngine :
mysql:unix_socket=/cloudsql/INSTANCE_CONNECTION_NAME;dbname=DATABASE

I don't know how is built the unix_socket in Directus and how to adapt it to match CloudSQL requirement.

[vzool]

unix_socket sould look like this to connect to CloudSQL db :
mysql:unix_socket=/cloudsql/INSTANCE_CONNECTION_NAME;dbname=DATABASE

I don't know how is built the unix_socket in Directus and how to adapt it to match CloudSQL requirement.

You will find it in your api.config with the key socket, so uncommented it and do the proper changes.

[carolin-skiply]

I found it thanks. I made the following change:
'database' => [ 'type' => 'mysql', //'host' => 'localhost', //'port' => 3306, 'name' => 'XXx', 'username' => 'XXX', 'password' => 'XXX', 'engine' => 'InnoDB', 'charset' => 'utf8mb4', // When using unix socket to connect to the database the host attribute should be removed 'socket' => 'mysql:unix_socket=/cloudsql/MY-INSTANCE_CONNECTION_NAME ]

but it still won't work :(

[vzool]

You forget to add database name section in your socket, so add all the following to your socket:

mysql:unix_socket=/cloudsql/INSTANCE_CONNECTION_NAME;dbname=DATABASE

Then all of it will look like this:

// ..

'database' => [

    // ...

    'socket' => 'mysql:unix_socket=/cloudsql/INSTANCE_CONNECTION_NAME;dbname=DATABASE',
],

// ..

[carolin-skiply]

Actually, I tried both with the same result :(

this error : PHP Fatal error: Uncaught InvalidArgumentException: There was a problem connecting to the database: SQLSTATE[HY 000] [2002] Connection timed out in /app/vendor/robmorgan/phinx/src/Phinx/Db/Adapter/MysqlAdapter.php:115

[vzool]

Nope, this is will not working either.
Beacuse, it does need full one like this:

```
mysql:unix_socket=/cloudsql/INSTANCE_CONNECTION_NAME:YOUR_REGION:DATABASE;dbname=DATABASE
````

mysql:unix_socket=/cloudsql/INSTANCE_CONNECTION_NAME:YOUR_REGION:DATABASE;dbname=DATABASE

I grab this from working application on AppEngine.

[vzool]

What is your CloudSQL generation v1 or v2?

[carolin-skiply]

I double checked : I put this INSTANCE_CONNECTION_NAME:YOUR_REGION:DATABASE in my config/api.php.
I tried both with and without ;dbname=DATABASE

FYI : I already used it in python in another AppEngine and it was working, the only difference was that I put the user:password as well:
mysql+mysqldb://USER:PASS@/DBNAME?unix_socket=/cloudsql/INSTANCE_CONNECTION:YOUR_REGION:DATABASE

I also add in my app.yaml the following:
beta_settings: cloud_sql_instances: skiply-test:europe-west4:skiply-test-core-db-00001

[vzool]

What is your CloudSQL generation v1 or v2?

??

[carolin-skiply]

It's a MySQL 2nd generation instance.

I'm going to try this :
'socket' => '/cloudsql/INSTANCE_CONNECTION_NAME;dbname=DATABASE',

[vzool]

2nd generation always needs to select a region

[carolin-skiply]

I got it ! It seems to work !

I put this then :
'socket' => '/cloudsql/INSTANCE_CONNECTION_NAME:YOUR_REGION:DATABASE'

[carolin-skiply]

Ok I can connect to the database (I was able to create the tables in my database from my instance - connected by ssh) but can't access to admin :(

I'm back to : {"error":{"error":8,"message":"API Environment Configuration Not Found: admin"}}

[carolin-skiply]

this is working https://my-url/server/ping but:

• /_/collections returns this error : {"error":{"code":300,"message":"Reading items from \"directus_collections\" collection was denied","class":"Directus\\Permissions\\Exception\\ForbiddenCollectionReadException","file":"\/app\/src\/core\/Directus\/Permissions\/Acl.php","line":988}}
• /admin returns this error : {"error":{"error":8,"message":"API Environment Configuration Not Found: admin"}}
• /_/auth returns this error: {"error":{"code":1,"message":"Not Found"}}

[carolin-skiply]

Ok, it's working.
But weird thing, I have to enter this url for it to work: https://my-url/admin/index.html#/login

[carolin-skiply]

Anyway thanks all for your help !

[rijkvanzanten]

But weird thing, I have to enter this url for it to work: https://my-url/admin/index.html#/login

This sounds like it doesn't use index.html as a default index file for a folder like other web servers do?

[carolin-skiply]

Hi all,

Got an update : I got an issue connecting to my database when installing directus on an GAE of Prod.
I found out that I forgot to activate the 'Cloud SQL Admin API' which create the unix socket on GAE.

In case of any got the same issue.

[carolin-skiply]

@rijkvanzanten

@carolin-skiply Did you deploy in a folder perhaps? Does apache's mod_rewrite work on app engine? Based on the repo the admin is in fact included in the code

Nope there is no apache's mod_rewrite on AppEngine :(

[DasPartyEinhorn]

Hi all,
I try to run Directus on app engine, too.
I made it to come to the login screen under '../admin/index.html' but I'm not able to login, because the '/_/auth/' endpoint is returning a 500.
I guess the app.yaml redirects are not right set up.

they look like:

runtime: php72

handlers:
- url: /admin
  static_dir: public/admin

- url: /.*
  script: auto

does someone know how to set them up right?

@carolin-skiply, could you please share your app.yaml file?

[skiplyfrance]

Hi,

Here is my app.yaml

service: directus
runtime: php
env: flex

runtime_config:
  document_root: public

beta_settings:
  cloud_sql_instances: XXX

manual_scaling:
  instances: 1
resources:
  cpu: 1
  memory_gb: 0.5
  disk_size_gb: 10

[Varulv1997]

Hi all,

Got an update : I got an issue connecting to my database when installing directus on an GAE of Prod.
I found out that I forgot to activate the 'Cloud SQL Admin API' which create the unix socket on GAE.

In case of any got the same issue.

Hello May I know how did you get it to connect to your database ? I enabled the API as well but still not working can you also provide me your app.yaml and your config if possible ? Thanks.