Pgloader: Control stack exhausted

[machytkafitanalytics]

Hello,
I tried to google for this problem but did not find anything useful.
Therefore I would like to ask you for your help.

I built latest pgloader version "3.5.d4dc449" from sources on new cloud instance with Debian 9 and PostgreSQL 10.4 using SBCL 1.4.7.189-862e622fa

I built it the same way as on other instances but they still run 3.4.x - this older version works OK, but new 3.5.x repeatedly fails - here are last lines from run with -v -d options (with some anonymization)

2018-05-22T14:40:02.047000Z NOTICE Prepare PostgreSQL database.
2018-05-22T14:40:02.052000Z DEBUG CONNECTED TO #<PGLOADER.PGSQL:PGSQL-CONNECTION pgsql://myuser@localhost:5432/mydb {10064C9193}>
2018-05-22T14:40:02.052000Z DEBUG SET client_encoding TO 'utf8'
2018-05-22T14:40:02.052000Z DEBUG SET search_path TO schema1, schema2, public
2018-05-22T14:40:02.053000Z DEBUG SET application_name TO 'pgloader'
2018-05-22T14:40:02.054000Z DEBUG BEGIN
KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution
2018-05-22T14:40:02.326000Z INFO Stopping monitor

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

What am I doing wrong? Do I have to change some setting for build?

Thanks.

[dimitri]

That looks like a bug on my parts. Is the load finishing normally despite the error message? In particular, is your search_path set correctly and are comments migrated?

[machytkafitanalytics]

Hmmm, not sure - debug output shows CONNECTED TO postgresql but I do not see any new connection in the database and any change. Of course pgloader fails immediately after connection is supposedly established so it may be I simply cannot see it because reload of pg_stat_activity takes much longer than connection 'life time" and all changes are most likely rollbacked....

[Nihlus]

I've got the same issue. I'm converting SQLite, and it doesn't go bang until it's begun migrating into PG.

[dimitri]

Is there a way you could share a source file that allows me to reproduce the bug locally?

[dimitri]

@machytkafitanalytics do you have bits of information that would allow me to reproduce the issue here?

[machytkafitanalytics]

Hi, I tested last weeks several times the latest pgloader from sources, I tested with different number of workers and parallelism but I always got the same error. (I copy whole database from MySQL 5.7 to PG 10.4, ~25GB of data on Debian 9.4, GCE instance 6 CPUs, 28 GB RAM.)

For the last test I downloaded sources of release 3.5.1 and all works well.
So I now run on that instance pgloader version "3.5.1~devel" compiled with SBCL 1.4.8.109-7c529c24f and all looks fine. So from my point of view this issue can be closed.

[AsinRay]

ostgres@eth-calc:~$ pgloader mysql://boot:[email protected]/eth_spider_18 pgsql:///xx
2018-07-12T10:42:40.054000Z LOG Migrating from #<MYSQL-CONNECTION mysql://[email protected]:3306/eth_spider_18 {100605E943}>
2018-07-12T10:42:40.057000Z LOG Migrating into #<PGSQL-CONNECTION pgsql://postgres@UNIX:5432/xx {10062A7D33}>
KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

MySQL 5.7 to PG10.4 on Debian 9!!

[HorvathAkosPeter]

I've just downloaded the last master from the github, recompiled it...

...and I've got still the same error.

I need to consider this software not-working, dysfunctional, abandoned and find other solutions.

It is a big disappointment for me.

[dimitri]

@HorvathAkosPeter You could also try to contribute. It's open source, works for many people, just not for you. The bug is recent, you can also compile an older version where the bug doesn't exist. Well you seem to be quite new to this Open Source thing: you are welcome as a contributor. We don't have mere users in here. Your complaints are only meaningful to yourself, nobody cares. Unless you're contributing.

[dimitri]

By the way, this seems to be a duplicate of #810 where we are bound to make progress with reproducible test cases, if people are interested...

[im-n1]

This issue is still a thing. Just ran into that. Running 3.5.2.

[smagellan]

Same, got this problem importing dbf -> postgres.
pgloader --version
pgloader version "3.5.2"
compiled with SBCL 1.4.9.debian

[krzyk]

And same for converting mysql to postgres, pgloader 3.5.2:

pgloader version "3.5.2"
compiled with SBCL 1.4.9.debian

postgres 10, mysql 5.7

[dimitri]

Hi @krzyk ; can you please compile pgloader from current git's head and see if you still have the problem? I think I did a fix that should cover the bug, but it might not be true. If the problem persists, is it possible to give me access to an environment where I could reproduce the bug and then fix it?

[krzyk]

@dimitri HEAD prints the same error, but when I download 3.5.1 it works.
I don't have a way to give you access to the system :(
But I could try to do a git bisect to find the commit that broke this

[FelikZ]

Have the same with latest official docker image

[krzyk]

@dimitri First bad commit is this one: d4dc4499a82bb49e2a160876d16393c1b9075e2b

[dimitri]

Yes, and the stack trace is useless, and I can't reproduce. Feel free to hack away and fix it yourself, I'm in the blind.

[dimitri]

@krzyk can you try with the latest path, I think it should help avoiding the stack trace and having a proper error message instead.

[krzyk]

@dimitri Unfortunately it looks the same:

...
2018-11-21T19:06:40.072000+01:00 SQL MySQL: sending query: -- params: db-name
--         table-type-name
--         only-tables
--         only-tables
--         including
--         filter-list-to-where-clause incuding
--         excluding
--         filter-list-to-where-clause excluding
  SELECT table_name, index_name, index_type,
         sum(non_unique),
         cast(GROUP_CONCAT(column_name order by seq_in_index) as char)
    FROM information_schema.statistics
   WHERE table_schema = 'urban'



GROUP BY table_name, index_name, index_type;
2018-11-21T19:06:40.161000+01:00 NOTICE Prepare PostgreSQL database.
2018-11-21T19:06:40.163000+01:00 DEBUG CONNECTED TO #<PGLOADER.PGSQL:PGSQL-CONNECTION pgsql://urban@localhost:5432/urban {1005BB6C63}>
2018-11-21T19:06:40.163000+01:00 DEBUG SET client_encoding TO 'utf8'
2018-11-21T19:06:40.163000+01:00 DEBUG SET application_name TO 'pgloader'
KABOOM!
2018-11-21T19:06:40.165000+01:00 DEBUG BEGIN
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution
2018-11-21T19:06:40.344000+01:00 INFO Stopping monitor

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

[krzyk]

Attaching strace, maybe it will help:

write(5, "P\0\0\0*\0SET application_name TO 'p"..., 55) = 55                    
poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)                       
poll([{fd=5, events=POLLIN|POLLPRI}], 1, -1) = 1 ([{fd=5, revents=POLLIN}])     
read(5, "1\0\0\0\4t\0\0\0\6\0\0n\0\0\0\4", 4096) = 17                           
write(5, "B\0\0\0\f\0\0\0\0\0\0\0\0E\0\0\0\t\0\0\0\0\0S\0\0\0\4", 28) = 28      
poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)                       
poll([{fd=5, events=POLLIN|POLLPRI}], 1, -1) = 1 ([{fd=5, revents=POLLIN}])     
read(5, "2\0\0\0\4S\0\0\0\36application_name\0pgloa"..., 4096) = 51             
write(5, "P\0\0\0\30\0select version()\0\0\0D\0\0\0\6S\0"..., 37) = 37          
poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)                       
poll([{fd=5, events=POLLIN|POLLPRI}], 1, -1) = 1 ([{fd=5, revents=POLLIN}])     
read(5, "1\0\0\0\4t\0\0\0\6\0\0T\0\0\0 \0\1version\0\0\0\0\0\0"..., 4096) = 45  
write(5, "B\0\0\0\16\0\0\0\0\0\0\0\1\0\1E\0\0\0\t\0\0\0\0\0S\0\0\0\4", 30) = 30 
poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)                       
poll([{fd=5, events=POLLIN|POLLPRI}], 1, -1) = 1 ([{fd=5, revents=POLLIN}])     
read(5, "2\0\0\0\4D\0\0\0\206\0\1\0\0\0|PostgreSQL 10.6 "..., 4096) = 160       
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_ACCERR, si_addr=0x1000d91ad8} ---   
mprotect(0x1000d90000, 32768, PROT_READ|PROT_WRITE|PROT_EXEC) = 0               
rt_sigreturn({mask=[]})                 = 68736741055                           
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_ACCERR, si_addr=0x1001ea60c8} ---   
mprotect(0x1001ea0000, 32768, PROT_READ|PROT_WRITE|PROT_EXEC) = 0               
rt_sigreturn({mask=[]})                 = 68751614123                           
write(5, "P\0\0\0C\0BEGIN TRANSACTION ISOLATIO"..., 80) = 80                    
poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)                       
poll([{fd=5, events=POLLIN|POLLPRI}], 1, -1) = 1 ([{fd=5, revents=POLLIN}])     
read(5, "1\0\0\0\4t\0\0\0\6\0\0n\0\0\0\4", 4096) = 17                           
write(5, "B\0\0\0\f\0\0\0\0\0\0\0\0E\0\0\0\t\0\0\0\0\0S\0\0\0\4", 28) = 28      
poll([{fd=5, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)                       
poll([{fd=5, events=POLLIN|POLLPRI}], 1, -1) = 1 ([{fd=5, revents=POLLIN}])     
read(5, "2\0\0\0\4C\0\0\0\nBEGIN\0Z\0\0\0\5T", 4096) = 22                       
futex(0x10060eb6b0, FUTEX_WAKE_PRIVATE, 1) = 1                                  
futex(0x100567b858, FUTEX_WAKE_PRIVATE, 1) = 1                                  
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_ACCERR, si_addr=0x204841c0} ---     
mprotect(0x20484000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC) = 0                  
rt_sigreturn({mask=[]})                 = 168635008                             
write(2, "KABOOM!\n", 8KABOOM!
)                = 8                                                            

I assume KABOOM is not expected, so I cut the strace there

[qpfiffer]

Getting this now on v3.6.2 with SBCL1.5.2 on Ubuntu. MySQL 5.7.26-0ubuntu0.18.04.1 with Postgres 10.7

Edit: Seems to be working with 3.5.1. So if you're looking for a quickfix, compile from source with that tag.

[dimitri]

Hi @qpfiffer ; can you do pgloader --version and select version() in your PostgreSQL database so that I have more to work with? Thanks.

[qpfiffer]

Sure thing:

$ pgloader --version
pgloader version "3.5.1f35413"
compiled with SBCL 1.5.2

postgres=> SELECT VERSION();
                          version                           
------------------------------------------------------------
 PostgreSQL 10.7, compiled by Visual C++ build 1800, 64-bit

[dimitri]

Hi @qpfiffer ; please open a new issue with a full backtrace and debug information when using 3.6.1, because it does not look like it comes from the same issue. The following returns NIL when the bug that was fixed in this issue is reached:

CL-USER> (pgloader.pgsql::parse-postgresql-version-string "PostgreSQL 10.7, compiled by Visual C++ build 1800, 64-bit")
"10.7"
"10"
:PGDG

[yujiaao]

I switch to pgloader 3.5.1, it works fine, but pgloader 3.5.2 not.