Patroni: Error calculating the new values for wal_segment_size and wal_block_size

[gandalfmagic]

Hi,

when I apply the changes with the command:

$ patronictl -c /etc/patroni/patroni.yml edit-config

I get the following error:

2019-11-19 12:22:38,687 ERROR: Failed to reload config_file=/etc/patroni/patroni.yml
Traceback (most recent call last):
   File "/usr/local/lib/python3.7/dist-packages/patroni/__init__.py", line 79, in reload_config
     self.postgresql.reload_config(self.config['postgresql'], sighup)
   File "/usr/local/lib/python3.7/dist-packages/patroni/postgresql/__init__.py", line 194, in reload_config
     self.config.reload_config(config, sighup)
   File "/usr/local/lib/python3.7/dist-packages/patroni/postgresql/config.py", line 864, in reload_config
     self._handle_wal_buffers(old_values, changes)
   File "/usr/local/lib/python3.7/dist-packages/patroni/postgresql/config.py", line 834, in _handle_wal_buffers
     wal_segment_size = parse_int(wal_segment_size[1]) * parse_int(wal_segment_size[2], 'B') / wal_block_size
 TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'

In my old configuration, I already have wal_buffers = 32MB

Maybe the line:

https://github.com/zalando/patroni/blob/2f9a48fae4ac57658e8cd653db8dad57744a57cb/patroni/postgresql/config.py#L834

Should be:

        wal_segment_size = parse_int(wal_segment_size[1], wal_segment_size[2]) / wal_block_size

[CyberDem0n]

Hi @gandalfmagic,

May I ask you to execute:

SELECT name, setting, unit, vartype, context FROM pg_catalog.pg_settings
WHERE pg_catalog.lower(name) = ANY('{wal_block_size,wal_segment_size,shared_buffers,wal_buffers}');

and post results here?
What postgres version is it?

[gandalfmagic]

Here you go:

       name       | setting  | unit | vartype |  context   
------------------+----------+------+---------+------------
 shared_buffers   | 65536    | 8kB  | integer | postmaster
 wal_block_size   | 8192     |      | integer | internal
 wal_buffers      | 4096     | 8kB  | integer | postmaster
 wal_segment_size | 16777216 | B    | integer | internal

Postgres is version 11.6 on Debian 10

[CyberDem0n]

Yeah, on older versions it looks differently:

 wal_segment_size │ 2048    │ 8kB  │ integer │ internal

Will prepare a fix.

[dtseiler]

When did that change? From 11.5 to 11.6?

[gandalfmagic]

No, I also have a cluster with PG 11.5, but the output of the SELECT is similar:

psql (11.5 (Debian 11.5-3.pgdg100+1))
Type "help" for help.

postgres=# SELECT name, setting, unit, vartype, context FROM pg_catalog.pg_settings
postgres-# WHERE pg_catalog.lower(name) = ANY('{wal_block_size,wal_segment_size,shared_buffers,wal_buffers}');
       name       | setting  | unit | vartype |  context   
------------------+----------+------+---------+------------
 shared_buffers   | 368640   | 8kB  | integer | postmaster
 wal_block_size   | 8192     |      | integer | internal
 wal_buffers      | 4096     | 8kB  | integer | postmaster
 wal_segment_size | 16777216 | B    | integer | internal
(4 rows)

Maybe I didn't notice the problem before. I cannot verify on previous versions.

[dtseiler]

Looks like v10 reports in 8k chunks:

postgres=# SELECT name, setting, unit, vartype, context FROM pg_catalog.pg_settings
postgres-# WHERE pg_catalog.lower(name) = ANY('{wal_block_size,wal_segment_size,shared_buffers,wal_buffers}');
       name       | setting | unit | vartype |  context
------------------+---------+------+---------+------------
 shared_buffers   | 16384   | 8kB  | integer | postmaster
 wal_block_size   | 8192    |      | integer | internal
 wal_buffers      | 512     | 8kB  | integer | postmaster
 wal_segment_size | 2048    | 8kB  | integer | internal
(4 rows)

postgres=# select version();
                                                 version
----------------------------------------------------------------------------------------------------------
 PostgreSQL 10.11 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
(1 row)

[dtseiler]

Yeah looks like this change happened with pg11, confirmed with 11.0:

postgres=# SELECT name, setting, unit, vartype, context FROM pg_catalog.pg_settings
postgres-# WHERE pg_catalog.lower(name) = ANY('{wal_block_size,wal_segment_size,shared_buffers,wal_buffers}');
       name       | setting  | unit | vartype |  context
------------------+----------+------+---------+------------
 shared_buffers   | 4096     | 8kB  | integer | postmaster
 wal_block_size   | 8192     |      | integer | internal
 wal_buffers      | 128      | 8kB  | integer | postmaster
 wal_segment_size | 16777216 | B    | integer | internal
(4 rows)

postgres=# select version();
                                                 version
---------------------------------------------------------------------------------------------------------
 PostgreSQL 11.0 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28), 64-bit
(1 row)

[gandalfmagic]

OK, and I confirm that I didn't see the issue on my cluster with PG 11.5, because I still have Patroni v1.6.0 on it.

[dtseiler]

What version of Patroni do you see the error with?

[gandalfmagic]

# patronictl version
patronictl version 1.6.1