Openshift-ansible: OpenShift 3.10 openshift_master_audit_config is not getting encoded correctly into /etc/origin/master/master-config.yaml

[outcoldman]

Description

Provide a brief description of your issue here. For example:

On a multi master install, if the first master goes down we can no
longer scaleup the cluster with new nodes or masters.

Version

Please put the following version information in the code block
indicated below.

• Your ansible version per ansible --version

If you're operating from a git clone:

• The output of git describe

If you're running from playbooks installed via RPM

• The output of rpm -q openshift-ansible

Place the output between the code block below:

ansible 2.4.6.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/vagrant/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Feb 20 2018, 09:19:12) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]

openshift-ansible-3.10.21-1.git.0.6446011.el7.noarch

Steps To Reproduce

1. Put in your ansible hosts file

openshift_master_audit_config={'enabled': true, 'policyFile': '/var/lib/origin/openpaas-oscp-audit/openpaas-oscp-audit.log', 'logFormat': 'json', 'maximumFileRetentionDays': 10, 'maximumFileSizeMegabytes': 10, 'maximumRetainedFiles': 3, 'auditFilePath': '/var/log/audit-ocp.log'}

1. Run ansible-playbook -i /home/vagrant/.config/openshift/hosts /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml

Expected Results

Describe what you expected to happen.

Example command and output or error messages

Observed Results

Describe what is actually happening.

F0815 23:00:05.625354       1 start_api.go:68] could not load config file "/etc/origin/master/master-config.yaml" due to an error: error reading config: only encoded map or array can be decoded into a struct

Because in the /etc/origin/master/master-config.yaml you will get something similar to

auditConfig: {''enabled'': true, ''policyFile'': ''/var/lib/origin/openpaas-oscp-audit/openpaas-oscp-audit.log'', ''logFormat'': ''json'', ''maximumFileRetentionDays'': 10, ''maximumFileSizeMegabytes'': 10, ''maximumRetainedFiles'': 3, ''auditFilePath'': ''/var/log/audit-ocp.log''}

I have tried various different ways to encode openshift_master_audit_config, like with double quotes for keys - results in the same. Also with quoting keys and values - that actually produces correct output in YAML format, but with incorrect types, like

auditConfig:
  auditFilePath: "/var/lib/origin/openpaas-oscp-audit/openpaas-oscp-audit.log"
  enabled: "true"
  maximumFileRetentionDays: "10"
  maximumFileSizeMegabytes: "100"
  maximumRetainedFiles: "3"
  policyFile: "/etc/origin/master/audit-policy.yaml"
  logFormat: "json"

Where master will fail to read enabled as boolean.

Additional Information

Provide any additional information which may help us diagnose the
issue.

Red Hat Enterprise Linux Server release 7.5 (Maipo)

[outcoldman]

Run another test.

openshift_master_audit_config={'enabled': 'true', 'policyFile': '/etc/origin/master/audit-policy.yaml', 'logFormat': 'json', 'maximumFileRetentionDays': '10', 'maximumFileSizeMegabytes': '10', 'maximumRetainedFiles': '3', 'auditFilePath': '/var/lib/origin/openpaas-oscp-audit/openpaas-oscp-audit.log'}

results in

auditConfig:
  auditFilePath: /var/lib/origin/openpaas-oscp-audit/openpaas-oscp-audit.log
  enabled: 'true'
  logFormat: json
  maximumFileRetentionDays: '10'
  maximumFileSizeMegabytes: '10'
  maximumRetainedFiles: '3'
  policyFile: /etc/origin/master/audit-policy.yaml

And that in

F0816 00:33:11.471138       1 start_api.go:68] could not load config file "/etc/origin/master/master-config.yaml" due to an error: error reading config: [pos 807]: json: decode bool: got first char "

Changing the 'true' to true and numbers in quotes to numbers fixes the issue.

[outcoldman]

Also

openshift_master_audit_config={"enabled": true, "policyFile": "/etc/origin/master/audit-policy.yaml", "logFormat": "json", "maximumFileRetentionDays": 10, "maximumFileSizeMegabytes": 10, "maximumRetainedFiles": 3, "auditFilePath": "/var/lib/origin/openpaas-oscp-audit/openpaas-oscp-audit.log"}

Result in

auditConfig: '{"enabled": true, "policyFile": "/etc/origin/master/audit-policy.yaml",
  "logFormat": "json", "maximumFileRetentionDays": 10, "maximumFileSizeMegabytes":
  10, "maximumRetainedFiles": 3, "auditFilePath": "/var/lib/origin/openpaas-oscp-audit/openpaas-oscp-audit.log"}'

and

F0816 00:57:57.420867       1 start_api.go:68] could not load config file "/etc/origin/master/master-config.yaml" due to an error: error reading config: only encoded map or array can be decoded into a struct

[outcoldman]

@vrutkovs curious, is there are any docs or information how to track when fixes will end up in RedHat RPM?

[DanyC97]

@outcoldman is your question related to when the the OCP openshift-ansible rpm will be created ? or is it related to the CentOS rpm ?

for both cases, the rpm will be generated after the a new openshift-ansible release will be cut off which for 3.10 that happens twice a week.
Saying that i cannot speak for OCP but for CentOS i can since i'm dealing with. - in short once the tag is cut off i will manually trigger and it would take 1 day to appear in the mirror.centos.org

hth

[vrutkovs]

@outcoldman not sure if we have OCP workflow documented properly somewhere, but in short the repo would be tagged and an RPM created internally and gets tested by QE. Approx. every three weeks a new errata is created - the bugzilla ticket (usually linked in the PR) would change state "closed errata" and errata reference.

[mrafieee]

I have the same issue on release-3.11 branch