Prefect: Flows and tasks are not saved to DB
Dear Team,
I would appreciate your help with this issue:
Description
Prefect server is running, I can access the DB through
http://localhost:3000/console/data/schema/public
Python Script 05_schedules.py from aircraft tutorial is running.
I cannot see any errors, but nothing is written to the DB and hence also nothing is displayed in the dashboard.
I see a connection error to: 172.19.0.4:4201
This IP Address does not exist.
Expected Behavior
See flow, task and schedule data in the DB and display them in the dashboard.
Reproduction
Install a clean VM with Ubuntu 19.10.
Set python3 to default
install python3-pip
install docker engine
install docker-compose
install prefect via pip
set prefect server to local
start prefect server
run python script
Environment
Python Script Log:
[2020-04-19 13:02:40,715] INFO - prefect.Flow: etl | Waiting for next scheduled run at 2020-04-19T13:03:36.018031+00:00
[2020-04-19 13:03:36,073] INFO - prefect.FlowRunner | Beginning Flow run for 'etl'
[2020-04-19 13:03:36,098] INFO - prefect.FlowRunner | Starting flow run.
[2020-04-19 13:03:36,257] INFO - prefect.TaskRunner | Task 'extract_reference_data': Starting task run...
fetching reference data...
[2020-04-19 13:03:37,797] INFO - prefect.TaskRunner | Task 'extract_reference_data': finished task run for task with final state: 'Success'
[2020-04-19 13:03:37,918] INFO - prefect.TaskRunner | Task 'radius': Starting task run...
[2020-04-19 13:03:37,976] INFO - prefect.TaskRunner | Task 'radius': finished task run for task with final state: 'Success'
[2020-04-19 13:03:38,134] INFO - prefect.TaskRunner | Task 'airport': Starting task run...
[2020-04-19 13:03:38,210] INFO - prefect.TaskRunner | Task 'airport': finished task run for task with final state: 'Success'
[2020-04-19 13:03:38,369] INFO - prefect.TaskRunner | Task 'load_reference_data': Starting task run...
saving reference data...
[2020-04-19 13:03:51,744] INFO - prefect.TaskRunner | Task 'load_reference_data': finished task run for task with final state: 'Success'
[2020-04-19 13:03:51,865] INFO - prefect.TaskRunner | Task 'extract_live_data': Starting task run...
fetching live aircraft data...
[2020-04-19 13:03:51,904] INFO - prefect.TaskRunner | Task 'extract_live_data': finished task run for task with final state: 'Success'
[2020-04-19 13:03:52,037] INFO - prefect.TaskRunner | Task 'transform': Starting task run...
cleaning & transform aircraft data...
[2020-04-19 13:03:52,075] INFO - prefect.TaskRunner | Task 'transform': finished task run for task with final state: 'Success'
[2020-04-19 13:03:52,183] INFO - prefect.TaskRunner | Task 'load_live_data': Starting task run...
saving live aircraft data...
[2020-04-19 13:03:53,114] INFO - prefect.TaskRunner | Task 'load_live_data': finished task run for task with final state: 'Success'
[2020-04-19 13:03:53,115] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded
[2020-04-19 13:03:53,137] INFO - prefect.Flow: etl | Waiting for next scheduled run at 2020-04-19T13:04:36.018031+00:00
Server start log:
/usr/local/lib/python3.7/dist-packages/prefect/cli/server.py:236: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
y = yaml.load(file)
Pulling postgres ... done
Pulling hasura ... done
Pulling graphql ... done
Pulling apollo ... done
Pulling scheduler ... done
Pulling ui ... done
Creating network "prefect-server" with the default driver
Creating tmp_postgres_1 ... done
Creating tmp_hasura_1 ... done
Creating tmp_graphql_1 ... done
Creating tmp_scheduler_1 ... done
Creating tmp_apollo_1 ... done
Creating tmp_ui_1 ... done
Attaching to tmp_postgres_1, tmp_hasura_1, tmp_graphql_1, tmp_scheduler_1, tmp_apollo_1, tmp_ui_1
apollo_1 |
apollo_1 | > @ serve /apollo
apollo_1 | > node dist/index.js
apollo_1 |
hasura_1 | {"type":"startup","timestamp":"2020-04-19T12:59:30.505+0000","level":"info","detail":{"kind":"server_configuration","info":{"live_query_options":{"batch_size":100,"refetch_delay":1},"transaction_isolation":"ISOLATION LEVEL READ COMMITTED","plan_cache_options":{"plan_cache_size":100},"enabled_log_types":["http-log","websocket-log","startup","webhook-log"],"server_host":"HostAny","enable_allowlist":false,"log_level":"info","auth_hook_mode":null,"use_prepared_statements":true,"unauth_role":null,"stringify_numeric_types":false,"enabled_apis":["metadata","graphql","config","pgdump"],"enable_telemetry":true,"enable_console":true,"auth_hook":null,"jwt_secret":null,"cors_config":{"allowed_origins":"","disabled":false,"ws_read_cookie":null},"console_assets_dir":null,"admin_secret_set":false,"port":3000}}}
hasura_1 | {"type":"startup","timestamp":"2020-04-19T12:59:30.505+0000","level":"info","detail":{"kind":"postgres_connection","info":{"retries":1,"database_url":"postgresql://prefect:...@postgres:5432/prefect_server"}}}
hasura_1 | {"internal":"could not connect to server: Connection refused\n\tIs the server running on host \"postgres\" (172.19.0.2) and accepting\n\tTCP/IP connections on port 5432?\n","path":"$","error":"connection error","code":"postgres-error"}
postgres_1 | The files belonging to this database system will be owned by user "postgres".
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting default max_connections ... 100
postgres_1 | selecting default shared_buffers ... 128MB
postgres_1 | selecting default timezone ... Etc/UTC
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | creating configuration files ... ok
postgres_1 | running bootstrap script ... ok
postgres_1 | performing post-bootstrap initialization ... ok
postgres_1 | syncing data to disk ... ok
postgres_1 |
postgres_1 | WARNING: enabling "trust" authentication for local connections
postgres_1 | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1 | --auth-local and --auth-host, the next time you run initdb.
ui_1 | 馃懢馃懢馃懢 UI running at localhost:8080 馃懢馃懢馃懢
postgres_1 |
postgres_1 | Success. You can now start the database server using:
postgres_1 |
postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1 |
postgres_1 | waiting for server to start....2020-04-19 12:59:30.041 UTC [45] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-04-19 12:59:30.098 UTC [46] LOG: database system was shut down at 2020-04-19 12:59:28 UTC
postgres_1 | 2020-04-19 12:59:30.104 UTC [45] LOG: database system is ready to accept connections
postgres_1 | done
postgres_1 | server started
postgres_1 | CREATE DATABASE
postgres_1 |
postgres_1 |
postgres_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/
postgres_1 |
postgres_1 | 2020-04-19 12:59:31.085 UTC [45] LOG: received fast shutdown request
postgres_1 | waiting for server to shut down....2020-04-19 12:59:31.091 UTC [45] LOG: aborting any active transactions
postgres_1 | 2020-04-19 12:59:31.098 UTC [45] LOG: background worker "logical replication launcher" (PID 52) exited with exit code 1
postgres_1 | 2020-04-19 12:59:31.098 UTC [47] LOG: shutting down
postgres_1 | 2020-04-19 12:59:31.118 UTC [45] LOG: database system is shut down
postgres_1 | done
postgres_1 | server stopped
postgres_1 |
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2020-04-19 12:59:31.266 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-04-19 12:59:31.267 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-04-19 12:59:31.270 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-04-19 12:59:31.324 UTC [63] LOG: database system was shut down at 2020-04-19 12:59:31 UTC
postgres_1 | 2020-04-19 12:59:31.338 UTC [1] LOG: database system is ready to accept connections
scheduler_1 | [2020-04-19 12:59:48,654] INFO - prefect-server.Scheduler | Scheduler will start after an initial delay of 83 seconds...
graphql_1 |
graphql_1 | Running Alembic migrations...
apollo_1 | 2020-04-19T12:59:49.626Z Error fetching GraphQL health: FetchError: request to http://graphql:4201/health failed, reason: connect ECONNREFUSED 172.19.0.4:4201
apollo_1 | 2020-04-19T12:59:49.633Z Error: Could not safely build a schema!
apollo_1 | at safelyBuildSchema (/apollo/dist/index.js:86:11)
apollo_1 | at process._tickCallback (internal/process/next_tick.js:68:7) Could not safely build a schema! Error: Could not safely build a schema!
apollo_1 | at safelyBuildSchema (/apollo/dist/index.js:86:11)
apollo_1 | at process._tickCallback (internal/process/next_tick.js:68:7)
apollo_1 | 2020-04-19T12:59:49.638Z
apollo_1 | Trying again in 3 seconds...
apollo_1 |
graphql_1 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
graphql_1 | INFO [alembic.runtime.migration] Will assume transactional DDL.
graphql_1 | INFO [alembic.runtime.migration] Running upgrade -> 723da8052cbd, create extensions
graphql_1 | INFO [alembic.runtime.migration] Running upgrade 723da8052cbd -> 2f249ccdcba7, set utc timezone
graphql_1 | INFO [alembic.runtime.migration] Running upgrade 2f249ccdcba7 -> 7ff37edbf446, Initial migration
graphql_1 | INFO [alembic.runtime.migration] Running upgrade 7ff37edbf446 -> 6c48e68b5e97, Add task run trigger
graphql_1 | INFO [alembic.runtime.migration] Running upgrade 6c48e68b5e97 -> aa594a91210c, Add run timing details trigger
graphql_1 | INFO [alembic.runtime.migration] Running upgrade aa594a91210c -> 8666a0ceb70e, Add trigger to update runs when states are inserted
graphql_1 | INFO [alembic.runtime.migration] Running upgrade 8666a0ceb70e -> 6b0389a57a0a, Add updated trigger to all tables
graphql_1 | INFO [alembic.runtime.migration] Running upgrade 6b0389a57a0a -> 7e65dadba625, fix log default timestamp
apollo_1 | 2020-04-19T12:59:52.649Z Error fetching GraphQL health: FetchError: request to http://graphql:4201/health failed, reason: connect ECONNREFUSED 172.19.0.4:4201
apollo_1 | 2020-04-19T12:59:52.649Z Error: Could not safely build a schema!
apollo_1 | at safelyBuildSchema (/apollo/dist/index.js:86:11)
apollo_1 | at process._tickCallback (internal/process/next_tick.js:68:7) Could not safely build a schema! Error: Could not safely build a schema!
apollo_1 | at safelyBuildSchema (/apollo/dist/index.js:86:11)
apollo_1 | at process._tickCallback (internal/process/next_tick.js:68:7)
apollo_1 | 2020-04-19T12:59:52.650Z
apollo_1 | Trying again in 3 seconds...
apollo_1 |
graphql_1 | Applied Hasura metadata from /prefect-server/services/hasura/migrations/metadata.yaml
graphql_1 |
graphql_1 | Database upgraded!
apollo_1 | 2020-04-19T12:59:55.654Z Error fetching GraphQL health: FetchError: request to http://graphql:4201/health failed, reason: connect ECONNREFUSED 172.19.0.4:4201
apollo_1 | 2020-04-19T12:59:55.655Z Error: Could not safely build a schema!
apollo_1 | at safelyBuildSchema (/apollo/dist/index.js:86:11)
apollo_1 | at process._tickCallback (internal/process/next_tick.js:68:7) Could not safely build a schema! Error: Could not safely build a schema!
apollo_1 | at safelyBuildSchema (/apollo/dist/index.js:86:11)
apollo_1 | at process._tickCallback (internal/process/next_tick.js:68:7)
apollo_1 | 2020-04-19T12:59:55.656Z
apollo_1 | Trying again in 3 seconds...
apollo_1 |
graphql_1 | INFO: Started server process [1]
graphql_1 | INFO: Waiting for application startup.
graphql_1 | INFO: Application startup complete.
graphql_1 | INFO: Uvicorn running on http://0.0.0.0:4201 (Press CTRL+C to quit)
graphql_1 | INFO: 172.19.0.6:33572 - "GET /health HTTP/1.1" 200 OK
apollo_1 | 2020-04-19T12:59:58.682Z Building schema...
graphql_1 | INFO: 172.19.0.6:33624 - "POST /graphql/ HTTP/1.1" 200 OK
apollo_1 | 2020-04-19T12:59:59.535Z Building schema complete!
apollo_1 | Server ready at http://0.0.0.0:4200/ 馃殌
Command ip a:
1: lo:
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3:
link/ether 08:00:27:36:ca:d7 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic noprefixroute enp0s3
valid_lft 84451sec preferred_lft 84451sec
inet6 fe80::503f:d9cc:1440:2711/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: docker0:
link/ether 02:42:4f:a1:70:07 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
34: br-792bbcdc4cca:
link/ether 02:42:f0:79:bb:62 brd ff:ff:ff:ff:ff:ff
inet 172.20.0.1/16 brd 172.20.255.255 scope global br-792bbcdc4cca
valid_lft forever preferred_lft forever
inet6 fe80::42:f0ff:fe79:bb62/64 scope link
valid_lft forever preferred_lft forever
36: vethaca424d@if35:
link/ether 42:1b:29:00:68:0c brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::401b:29ff:fe00:680c/64 scope link
valid_lft forever preferred_lft forever
40: veth7bc0c92@if39:
link/ether 32:16:05:92:8e:bd brd ff:ff:ff:ff:ff:ff link-netnsid 2
inet6 fe80::3016:5ff:fe92:8ebd/64 scope link
valid_lft forever preferred_lft forever
42: veth343cadb@if41:
link/ether 0a:0a:ea:e2:53:fc brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::80a:eaff:fee2:53fc/64 scope link
valid_lft forever preferred_lft forever
44: vetha089f24@if43:
link/ether 3a:f7:26:64:b9:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 4
inet6 fe80::38f7:26ff:fe64:b9a2/64 scope link
valid_lft forever preferred_lft forever
46: vethdd10a80@if45:
link/ether 3a:7a:a0:02:a8:76 brd ff:ff:ff:ff:ff:ff link-netnsid 3
inet6 fe80::387a:a0ff:fe02:a876/64 scope link
valid_lft forever preferred_lft forever
48: vethc683e06@if47:
link/ether ca:e9:60:50:eb:4f brd ff:ff:ff:ff:ff:ff link-netnsid 5
inet6 fe80::c8e9:60ff:fe50:eb4f/64 scope link
valid_lft forever preferred_lft forever
GZangl
All 3 comments
Hi @GZangl! It looks like you need to call flow.register() instead of flow.run() in the Prefect script :)
And to ensure you're hitting your local Prefect server deployment (instead of Prefect Cloud), make sure to call prefect backend server before running the Prefect script again!
znicholasbrown
on 19 Apr 2020
Hi @znicholasbrown ,
thanks a lot for your comment. Yes, that was the problem, I didn't know I need to register a flow. I quick start tutorial which covers these traps would be appreciated.
Nevertheless, Prefect is perfect, I love it from the first sight. Congratulations to everybody who is involved in the development. This is amazing :)
GZangl
on 19 Apr 2020
@GZangl thanks for the kind words! We鈥檒l keep working on making sure getting started is as smooth as possible.
jlowin
on 19 Apr 2020
Related issues
petermorrow
路
3Comments
fgblomqvist
路
4Comments
cicdw
路
3Comments
jameslamb
路
3Comments
WilliamBurdett
路
3Comments
Most helpful comment
Hi @znicholasbrown ,
thanks a lot for your comment. Yes, that was the problem, I didn't know I need to register a flow. I quick start tutorial which covers these traps would be appreciated.
Nevertheless, Prefect is perfect, I love it from the first sight. Congratulations to everybody who is involved in the development. This is amazing :)