logging of tripleo-modify-image role during prepare is hard to read

Bug #1813684 reported by Steve Baker
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Steve Baker

Bug Description

The tripleo-modify-image role is invoked by a playbook with -vvv, which results in the below output when shell scripts produce a lot of output. This verbose stderr would be much more readable if the yaml stdout_callback was used with a lower level of verbosity (like -v)

Example output:

2019-01-24 01:29:15,616 p=28525 u=root | TASK [tripleo-modify-image : Run yum_update.sh] ********************************
2019-01-24 01:29:15,617 p=28525 u=root | task path: /usr/share/ansible/roles/tripleo-modify-image/tasks/yum_update_buildah.yml:36
2019-01-24 01:29:15,617 p=28525 u=root | Thursday 24 January 2019 01:29:15 +0000 (0:00:01.239) 0:00:11.682 ******
2019-01-24 01:29:15,641 p=28525 u=root | [DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using
`result|exists` use `result is exists`. This feature will be removed in version
 2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
2019-01-24 01:29:15,655 p=28525 u=root | Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py
2019-01-24 01:29:49,287 p=28525 u=root | changed: [localhost] => {
    "changed": true,
    "cmd": [
        "buildah",
        "run",
        "--volume",
        "/tmp/ansible.pg3Hcf:/tmp/yum_update.sh",
        "--volume",
        "/etc/yum.repos.d:/etc/yum.repos.d",
        "--volume",
        "/opt/gating_repo:/opt/gating_repo",
        "--user",
        "root",
        "--net",
        "host",
        "centos-binary-rsyslog-base-working-container",
        "/tmp/yum_update.sh",
        "gating-repo,delorean-current"
    ],
    "delta": "0:00:33.485473",
    "end": "2019-01-24 01:29:49.248262",
    "invocation": {
        "module_args": {
            "_raw_params": "buildah run\n --volume /tmp/ansible.pg3Hcf:/tmp/yum_update.sh\n --volume /etc/yum.repos.d:/etc/yum.repos.d\n --volume /opt/gating_repo:/opt/g
ating_repo\n --user root\n --net host\n centos-binary-rsyslog-base-working-container\n /tmp/yum_update.sh \"gating-repo,delorean-current\"",
            "_uses_shell": false,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "warn": true
        }
    },
    "rc": 0,
    "start": "2019-01-24 01:29:15.762789",
    "stderr": "+ packages_for_update=\n+ '[' -n gating-repo,delorean-current ']'\n+ command -v repoquery\n++ sort\n++ rpm -qa --qf '%{NAME}\\n'\n+ installed='acl\naudit-libs\naugeas-libs\nbasesystem\nbash\nbind-license\nbinutils\nbzip2-libs\nca-certificates\ncentos-release\ncentos-release-ceph-luminous\ncentos-release-opstools\ncentos-release-qemu-ev\ncentos-release-storage-common\ncentos-release-virt-common\nchkconfig\ncoreutils\ncpio\ncracklib\ncracklib-dicts\ncronie\ncronie-anacron\ncrontabs\ncrudini\ncryptsetup-libs\ncurl\ncyrus-sasl-lib\ndbus\ndbus-glib\ndbus-libs\ndbus-python\ndevice-mapper\ndevice-mapper-event\ndevice-mapper-event-libs\ndevice-mapper-libs\ndevice-mapper-persistent-data\ndiffutils\ndmidecode\ndracut\nelfutils-default-yama-scope\nelfutils-libelf\nelfutils-libs\nexpat\nfacter\nfile-libs\nfilesystem\nfindutils\ngawk\ngdbm\nglib2\nglibc\nglibc-common\ngmp\ngnupg2\ngobject-introspection\ngpg-pubkey\ngpg-pubkey\ngpg-pubkey\ngpg-pubkey\ngpgme\ngrep\ngzip\nhardlink\nhiera\nhostname\nhwdata\ninfo\niproute\niptables\niputils\niscsi-initiator-utils\niscsi-initiator-utils-iscsiuio\njson-c\nkeyutils-libs\nkmod\nkmod-libs\nkpartx\nkrb5-libs\nlibacl\nlibaio\nlibassuan\nlibattr\nlibblkid\nlibcap\nlibcap-ng\nlibcom_err\nlibcurl\nlibdb\nlibdb-utils\nlibestr\nlibfastjson\nlibffi\nlibgcc\nlibgcrypt\nlibgpg-error\nlibidn\nlibmnl\nlibmount\nlibnetfilter_conntrack\nlibnfnetlink\nlibpwquality\nlibselinux\nlibselinux-ruby\nlibselinux-utils\nlibsemanage\nlibsepol\nlibsmartcols\nlibssh2\nlibstdc++\nlibtasn1\nlibuser\nlibutempter\nlibuuid\nlibverto\nlibxml2\nlibxml2-python\nlibyaml\nlogrotate\nlsof\nlua\nlvm2\nlvm2-libs\nlz4\nncurses\nncurses-base\nncurses-libs\nnet-tools\nnspr\nnss\nnss-pem\nnss-softokn\nnss-softokn-freebl\nnss-sysinit\nnss-tools\nnss-util\nopenldap\nopenssl-libs\nopenstack-tripleo-common-container-base\np11-kit\np11-kit-trust\npam\npasswd\npciutils\npciutils-libs\npcre\npinentry\npkgconfig\npopt\nprocps-ng\npth\npuppet\npuppet-aodh\npuppet-apache\npuppet-archive\npuppet-auditd\npuppet-barbican\npuppet-cassandra\npuppet-ceilometer\npuppet-ceph\npuppet-certmonger\npuppet-cinder\npuppet-collectd\npuppet-concat\npuppet-congress\npuppet-contrail\npuppet-corosync\npuppet-datacat\npuppet-designate\npuppet-dns\npuppet-ec2api\npuppet-elasticsearch\npuppet-etcd\npuppet-fdio\npuppet-firewall\npuppet-fluentd\npuppet-git\npuppet-glance\npuppet-gnocchi\npuppet-haproxy\npuppet-heat\npuppet-horizon\npuppet-inifile\npuppet-ipaclient\npuppet-ironic\npuppet-java\npuppet-kafka\npuppet-keepalived\npuppet-keystone\npuppet-kibana3\npuppet-kmod\npuppet-manila\npuppet-memcached\npuppet-midonet\npuppet-mistral\npuppet-module-data\npuppet-mongodb\npuppet-mysql\npuppet-n1k-vsm\npuppet-neutron\npuppet-nova\npuppet-nssdb\npuppet-ntp\npuppet-octavia\npuppet-opendaylight\npuppet-openstack_extras\npuppet-openstacklib\npuppet-oslo\npuppet-ovn\npuppet-pacemaker\npuppet-panko\npuppet-placement\npuppet-powerdns\npuppet-qdr\npuppet-rabbitmq\npuppet-redis\npuppet
...
<snip>
...
    "stderr_lines": [
        "+ packages_for_update=",
        "+ '[' -n gating-repo,delorean-current ']'",
        "+ command -v repoquery",
        "++ sort",
        "++ rpm -qa --qf '%{NAME}\\n'",
        "+ installed='acl",
        "audit-libs",
        "augeas-libs",
        "basesystem",
        "bash",
        "bind-license",
        "binutils",
        "bzip2-libs",
        "ca-certificates",
        "centos-release",
        "centos-release-ceph-luminous",
        "centos-release-opstools",
        "centos-release-qemu-ev",
        "centos-release-storage-common",
        "centos-release-virt-common",
        "chkconfig",
        "coreutils",
        "cpio",
...
etc

Revision history for this message
Steve Baker (steve-stevebaker) wrote :

The yaml stdout_callback is available in ansible since 2.5, so I believe it is safe to make this change in stein

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-common (master)

Fix proposed to branch: master
Review: https://review.openstack.org/633633

Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-common (master)

Reviewed: https://review.openstack.org/633633
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=2a6ef6a4f9088436faf092dfee50eca4d37d19f9
Submitter: Zuul
Branch: master

commit 2a6ef6a4f9088436faf092dfee50eca4d37d19f9
Author: Steve Baker <email address hidden>
Date: Tue Jan 29 14:10:24 2019 +1300

    Use the yaml stdout_callback for better prepare logging

    Currently when the tripleo-modify-role runs a shell script which
    produces a lot of output, its logged once as a no-newline-blob, and
    again as one line per array item. Neither output is easy read, which makes
    debugging harder.

    This change switches to the yaml stdout_callback, and a verbosity
    level which only prints the script output once.

    Change-Id: I85fb8ca40e03f077147871fcd85536c0572d526e
    Closes-Bug: #1813684

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-common 10.4.0

This issue was fixed in the openstack/tripleo-common 10.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to ansible-role-tripleo-modify-image (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/645390

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to ansible-role-tripleo-modify-image (master)

Reviewed: https://review.openstack.org/645390
Committed: https://git.openstack.org/cgit/openstack/ansible-role-tripleo-modify-image/commit/?id=c0bcc3c438ef8095b51e18f43a5b7910af18caed
Submitter: Zuul
Branch: master

commit c0bcc3c438ef8095b51e18f43a5b7910af18caed
Author: Steve Baker <email address hidden>
Date: Fri Mar 22 11:30:56 2019 +1300

    Don't print commands during yum update

    The commands contain package lists, so having "set -x" makes the
    logged result very very verbose.

    Change-Id: I8ffda90eb0633b6b46e29c5af7ed2ff3571c14a2
    Related-Bug: #1813684

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.