Comment 1 for bug 1699990

    Please find below information about why some packages are not updated
after cluster update to 9.2MU .

The main reason - is that 'get_packages_for_upgrade.rb' puppet function doesn't return necessary list of packages (it returns a hash of packages).

To get needed packages for upgrade 'get_packages_for_upgrade.rb' script runs this shell command:
'apt-get --just-print -o Dir::etc::sourcelist='-' -o Dir::Etc::sourceparts='#{source_list}' dist-upgrade -qq'

, where 'source_list' - is a path to a directory, where repo file is located
 (it's hardcoded to '/etc/fuel/maintenance/apt/sources.list.d/').

In fact '/etc/fuel/maintenance/apt/sources.list.d/' directory contains a symbol link to a repository file located in standard place (/etc/apt/sources.list.d/),
for "mos9.2-updates.list" repo it contains the following:
mos9.2-updates.list -> /etc/apt/sources.list.d/mos9.2-updates.list

After run 'apt-get --just-print -o Dir::etc::sourcelist='-'
-o Dir::Etc::sourceparts='#{source_list}' dist-upgrade -qq' there is output like this:
....
..
Remv nova-compute-qemu [2:13.1.1-7~u14.04+mos20] [nova-compute:amd64 ]
Remv qemu [1:2.3+dfsg-5~u14.04+mos3]
Remv qemu-kvm [1:2.3+dfsg-5~u14.04+mos3]
Remv qemu-system [1:2.3+dfsg-5~u14.04+mos3]
Remv qemu-system-arm [1:2.3+dfsg-5~u14.04+mos3]
Remv qemu-system-ppc [1:2.3+dfsg-5~u14.04+mos3]
Remv qemu-system-mips [1:2.3+dfsg-5~u14.04+mos3]
Remv qemu-system-common [1:2.3+dfsg-5~u14.04+mos3] [qemu-system-x86:amd64 qemu-system-sparc:amd64 qemu-system-misc:amd64 ]
Remv qemu-system-misc [1:2.3+dfsg-5~u14.04+mos3] [qemu-system-x86:amd64 qemu-system-sparc:amd64 ]
Remv qemu-system-sparc [1:2.3+dfsg-5~u14.04+mos3] [qemu-system-x86:amd64 ]
Remv qemu-system-x86 [1:2.3+dfsg-5~u14.04+mos3]
Inst libxml2 [2.9.1+dfsg1-3ubuntu4.9] (2.9.3+dfsg1-1~u1404+mos2 mos9.0:mos9.0-updates [amd64])
Inst ceph [0.94.6-1~u14.04+mos1] (0.94.9-1~u14.04+mos2 mos9.0:mos9.0-updates [amd64]) []
Inst ceph-common [0.94.6-1~u14.04+mos1] (0.94.9-1~u14.04+mos2 mos9.0:mos9.0-updates [amd64]) []
Inst python-rbd [0.94.6-1~u14.04+mos1] (0.94.9-1~u14.04+mos2 mos9.0:mos9.0-updates [amd64]) []
Inst python-rados [0.94.6-1~u14.04+mos1] (0.94.9-1~u14.04+mos2 mos9.0:mos9.0-updates [amd64]) []
Inst librbd1 [0.94.6-1~u14.04+mos1] (0.94.9-1~u14.04+mos2 mos9.0:mos9.0-updates [amd64]) []
Inst librados2 [0.94.6-1~u14.04+mos1] (0.94.9-1~u14.04+mos2 mos9.0:mos9.0-updates [amd64]) []
Inst python-cephfs [0.94.6-1~u14.04+mos1] (0.94.9-1~u14.04+mos2 mos9.0:mos9.0-updates [amd64]) []
..
.......

'get_packages_for_upgrade.rb' script parses this output and select all lines that contain 'Inst' operation label.
Here is a line of code that performs it:
'package_list = package_list_common_out.split("\n").select { |line| line =~ /^Inst\b/ }'

Afer this parsing all packages with 'Remv' operation label are missed, they aren't returned to puppet manifest for package resource creation, package upgrade performing.
So it leads to a situation that is described in 'Bug Description': not all packages are upgraded after applying 9.2MU.

 Why there is 'Remv' operation label ?
    When we run:
'apt-get --just-print -o Dir::etc::sourcelist='-'
 -o Dir::Etc::sourceparts='/etc/fuel/maintenance/apt/sources.list.d/' dist-upgrade -qq'

, it tries to perform upgrade of all packages from specified repository and output list of the packages (as there is '--just-print parameter).
If the repository doesn't contain all needed packages (package dependencies) for package upgrade, this package is tagged like 'Remv' and will be removed by 'dist-upgrade'.

In our case we need to have not only 'mos9.2-updates' , but ubuntu upstream repository to satisfy all dependencies and to upgrade all packages (to get list of all needed packages after 'get_packages_for_upgrade.rb' script run).