Parse-server: Proposal to DROP SUPPORT of Postgres due to ongoing CI Instability

[Moumouls]

It seems that we have some CI fails
https://github.com/parse-community/parse-server/pull/6371
https://github.com/parse-community/parse-server/pull/6360
https://github.com/parse-community/parse-server/pull/6363

Sometimes Postgres error:

ERROR:  could not access file "$libdir/postgis-2.4": No such file or directory
The command "psql -c 'CREATE EXTENSION postgis;' -U postgres -d parse_server_postgres_adapter_test_database" failed and exited with 1 during .

Sometimes Push error:

PushController properly creates _PushStatus
  Message:
    Expected 'pending' to be 'running'.
  Stack:
    Error: Expected 'pending' to be 'running'.
        at <Jasmine>
        at reconfigureServer.then.then.then.then.then.catch.then (/home/travis/build/parse-community/parse-server/spec/PushController.spec.js:647:48)
        at process._tickCallback (internal/process/next_tick.js:68:7)
Pending:

Do you have any idea where the failures come from?

[davimacedo]

@Moumouls thanks for reporting. We need to figure it out. In the meantime, I managed to have your PRs tests passing.

[dplewis]

I tried to fix the postgres issue a while ago.

https://github.com/parse-community/parse-server/pull/6299

I could downgrade to 10 and see if that helps (originally was at 9.5 then jumped to 11). I don't know why its looking for postgis 2.4 (it should look for 2.5 and I don't know how to force it).

For the Push Status one I might be able to fix it indirectly.

[acinader]

I am just restarting builds to get them to pass. Not sustainable.

[davimacedo]

I've also done this for a long while now...

[acinader]

do you use postgres?

I really can't take the time to be able to test postgres.

I think we should take it out of testing and only add it back if someone who actually uses it in production can help.

[davimacedo]

I don't have any app using Postgres but I know there are some users out there using it. I am not sure about just removing it from the CI. I think we'd need to descontinue its support because if we remove it from the CI it will stop working sooner or later. Maybe we can think about moving it to a different repo with its own tests which would be pinned to use the current version of Parse till the moment someone join to maintain it. I'm not sure. I'm open to ideas because it is really consuming a lot of time to continue carrying it here and making sure that everything we do also passes on Postgres even though there are probably few users using it.

[cbaker6]

@acinader I use Postgres 11 and it was working well on parse-server 3.10. I might be able to help here, though I haven't done anything with CL before. If you point me to the files that's causing the issue, I can take a look.

On a sidetone, I do think there's a small issue with using it on 4.1.0 (I'm assuming it as it seems to have trouble adding a fields in some tables, I'm thinking this is whenever I'm adding a reference to another object in a table (I'm getting an error in parse_relation.c "errorMissingColumn", which I didn't get when running the same code in 3.10.0). I'm currently doing some tests and will post in "Issues" once I know more Update: Turns out this was an issue on my side and not the parse-server

[acinader]

Hi @cbaker6

I'd be glad to help out if I can. Having someone who is using postgres helping out would be awesome and is really a pre-requisite for us being able to continue to support postgres in a first class way.

You could also open a trivial pull request to set off a test. If it doesn't fail on the first try, I can just keep re-running it until it does break. It usually fails on the first try though.
So

1. we don't currently have an example of the failure that this issue cites (see @Moumouls issue above. he cites two errors). I see the first one frequently. Re-running has always fixed for me so far. Next time we get one, I won't re-rerun it so you can see what's going on.

2. for the new error between 3.10 and 4.1, if you could do a git bisect to find the commit that introduces the warning/error, that'd probably be a good place to start.

I can be of more help if we can slack each other in real time. Out channel is: parseopensource

[cbaker6]

If the fix works okay, can this be closed?

[acinader]

yup

[acinader]

boo. looks like #6490 did not fix the problem. see #6501's build

we can see that the fork contains the #6490 fix.

[cbaker6]

@acinader from the comments earlier, it seems this problem was pretty consistent. It looks like there was at least 5 pull requests after the commit that didn't have the issue.

Was it always this sporadic? As I would like to know how to replicate it to fix it

[acinader]

that's a good question that I, unfortunately, don't have the answer to.

My off the cuff is that we were seeing one or two jobs failing out the 5 jobs / build ~ 50% of the pr's

sometimes we'd see none and other times we'd see 4 or 5 job failures.

so it was purdy dang variable.

[dplewis]

To fix this issue it’s going to be a guess and check game. Commit check CI and repeat.

[cbaker6]

I'm pretty sure the issue is what I put in the comment for my attempted fix. According to this the Xenial image that the current CI uses comes with pre-installed versions of postgres 9.4 9.5 9.6 10. The "services: postgres" line I took out was previously enabling the old version of Postgres. Some things to look into /are:

1. Figure out how to remove postgres from the Xenial image or specify the correct version to test at all times (similar to what @dplewis mentioned before

2. Find a different image that serves the purpose of testing the parse-server, but doesn't have postgres installed

3. Set the test to run on postgres 9.4 until the problem is fixed. It looks like most of the previous tests ran in the past were against postgres 9 anyways. Here you will only be missing tests on postgres 10,11,12.

Note: I'm new to travis, but it looks like the way the current file is written, it's installing postgres for every test, even when testing mongo (I would guess this adding time on testing). I have seen some things online about specifying services and addons for specific test. This will be important even after the aforementioned problem is fixed because it will help identify issues in different versions of Postgres. For example, I'm sure Postgres 12 is buggy as I was having issues with it on my server, but testing against this version will help identify where the issues are. I can look into this later.

[dplewis]

From the build @acinader posted I discovered Postgres 10 is installed. I’m on mobile but View Raw logs and search Postgres. 10.7 is installing

I don’t know why but I’m sure we can uninstall 10

[srameshr]

@dplewis @cbaker6 @davimacedo
Please do not drop support for postgres! Our analytics and data dump platform runs on postgres with parse server ORM. Build issues should not have nothing to do with features

[cbaker6]

@srameshr I think #6505 addressed the issue that was mentioned in this thread. Though I believe it's worth leaving this open for a few more tests to make sure it's gone

[dplewis]

I believe this can be closed know. I haven't seen it this issue again after 20+ builds