packages CI failed due to Could not update packages

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

version

steps to reproduce

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

Dmitry, can you link to the failed job?

Also, this bug title is incorrect. Fuelmenu couldn't load an initial settings file (because we no longer install a default one). The error isn't actually fatal. Bootstrapping Fuel Master succeeded and it even built a bootstrap image. However, there were no bootstrapped nodes or DHCP requests. Something must have failed in determining that the Fuel Master was deployed.

I made https://bugs.launchpad.net/fuel/+bug/1567307 as a medium priority bug to fix the log message so that others are not reporting bugs about this non-issue.

I'm changing the bug title to "Fuel Master deployment failed" and marking as incomplete.

Yep,
https://packaging-ci.infra.mirantis.net/job/master-pkg-systest-ubuntu/502/

On Thu, Apr 7, 2016 at 12:24 PM, Matthew Mosesohn <email address hidden>
wrote:

> Dmitry, can you link to the failed job?
>
> Also, this bug title is incorrect. Fuelmenu couldn't load an initial
> settings file (because we no longer install a default one). The error
> isn't actually fatal. Bootstrapping Fuel Master succeeded and it even
> built a bootstrap image. However, there were no bootstrapped nodes or
> DHCP requests. Something must have failed in determining that the Fuel
> Master was deployed.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1566880
>
> Title:
> Fuel Master deployment failure
>
> Status in Fuel for OpenStack:
> Incomplete
>
> Bug description:
> Detailed bug description:
>
> master-pkg-systest ubuntu build #470 failed with error in fuelmenu.log
> "ERROR Unable to read YAML: /etc/fuel/astute.yaml"
>
> Expected results:
>
> build succeeded
>
> Actual result:
>
> build failed
>
> Reproducibility:
>
> issue happened in builds 470-474
>
> Also this issue was mentioned in code review
> https://review.openstack.org/#/c/98743/ (see comments by Artem
> Panchenko)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/fuel/+bug/1566880/+subscriptions
>

Sorry Matthew,

wrong link

build #503
https://packaging-ci.infra.mirantis.net/job/master-pkg-systest-ubuntu/503/

On Thu, Apr 7, 2016 at 2:01 PM, Dmitry Stepanenko <email address hidden>
wrote:

> Yep,
> https://packaging-ci.infra.mirantis.net/job/master-pkg-systest-ubuntu/502/
>
> On Thu, Apr 7, 2016 at 12:24 PM, Matthew Mosesohn <email address hidden>
> wrote:
>
>> Dmitry, can you link to the failed job?
>>
>> Also, this bug title is incorrect. Fuelmenu couldn't load an initial
>> settings file (because we no longer install a default one). The error
>> isn't actually fatal. Bootstrapping Fuel Master succeeded and it even
>> built a bootstrap image. However, there were no bootstrapped nodes or
>> DHCP requests. Something must have failed in determining that the Fuel
>> Master was deployed.
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/1566880
>>
>> Title:
>> Fuel Master deployment failure
>>
>> Status in Fuel for OpenStack:
>> Incomplete
>>
>> Bug description:
>> Detailed bug description:
>>
>> master-pkg-systest ubuntu build #470 failed with error in fuelmenu.log
>> "ERROR Unable to read YAML: /etc/fuel/astute.yaml"
>>
>> Expected results:
>>
>> build succeeded
>>
>> Actual result:
>>
>> build failed
>>
>> Reproducibility:
>>
>> issue happened in builds 470-474
>>
>> Also this issue was mentioned in code review
>> https://review.openstack.org/#/c/98743/ (see comments by Artem
>> Panchenko)
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/fuel/+bug/1566880/+subscriptions
>>
>
>

The error is:
2016-04-06 13:13:59,019 - ERROR decorators.py:250 -- Could not update packages
and then diagnostic snapshot generation failed.

This job failed because it's using master branch (with 10.0 packages). It looks like we need to fix some branch configuration for this job:

Error: Package: fuel-10.0.0-1.mos6330.noarch (temporary)
           Requires: fuel-provisioning-scripts >= 10.0.0
           Installed: fuel-provisioning-scripts-9.0.0-1.mos8602.noarch (@mos)
               fuel-provisioning-scripts = 9.0.0-1.mos8602
           Available: fuel-provisioning-scripts-8.0.0-1.mos8183.noarch (temporary)
               fuel-provisioning-scripts = 8.0.0-1.mos8183
           Available: fuel-provisioning-scripts-8.0.0-1.mos8253.noarch (temporary)
               fuel-provisioning-scripts = 8.0.0-1.mos8253

Folks, I think this bug should be cleaned a little. So IMO: there are three issues which were linked (try to link) to this single bug:

1. Not critical as Matt said "ERROR Unable to read YAML: /etc/fuel/astute.yaml" and he created the related bug https://bugs.launchpad.net/fuel/+bug/1567307

2. issue happened in builds 470-474:
2016-04-04 01:17:13,761 - INFO environment.py:688 -- Result of "yum clean expire-cache; yum update -y" command on master node: {'exit_code': 1, 'stderr': ['\n', '\n', ' One of the configured repositories failed (Unknown),\n', " and yum doesn't have enough cached data to continue. At this point the only\n", ' safe thing yum can do is fail. There are a few ways to work "fix" this:\n', '\n', ' 1. Contact the upstream for the repository and get them to fix the problem.\n', '\n', ' 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n', ' upstream. This is most often useful if you are using a newer\n', ' distribution release than is supported by the repository (and the\n', ' packages for the previous distribution release still work).\n', '\n', " 3. Disable the repository, so yum won't use it by default. Yum will then\n", ' just ignore the repository until you permanently enable it again or use\n', ' --enablerepo for temporary usage:\n', '\n', ' yum-config-manager --disable <repoid>\n', '\n', ' 4. Configure the failing repository to be skipped, if it is unavailable.\n', ' Note that yum will try to contact the repo. when it runs most commands,\n', ' so will have to try and fail each time (and thus. yum will be be much\n', ' slower). If it is a very temporary problem though, this is often a nice\n', ' compromise:\n', '\n', ' yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true\n', '\n', 'Cannot find a valid baseurl for repo: mos9.0-security/7\n'], 'stdout': ['Loaded plugins: fastestmirror, priorities\n', 'Cleaning repos: 9.0_auxiliary mos mos9.0-security mos9.0-updates nailgun\n', ' : temporary-0\n', '7 metadata files removed\n', 'Loaded plugins: fastestmirror, priorities\n', 'Could not retrieve mirrorlist http://mirror.fuel-infra.org/mos-repos/centos/mos9.0-centos7/mos-mirrors-security.txt error was\n', '14: HTTP Error 404 - Not Found\n', 'Could not retrieve mirrorlist http://mirror.fuel-infra.org/mos-repos/centos/mos9.0-centos7/mos-mirrors-updates.txt error was\n', '14: HTTP Error 404 - Not Found\n', 'Loading mirror speeds from cached hostfile\n']}

IMO it's related to https://bugs.launchpad.net/fuel/+bug/1563870 and it was fixed, so no such issues should happen again
but I propose to update bug number on builds 470-474

3) issue happened in build 503, 510. As Matt said:
Error: Package: fuel-10.0.0-1.mos6330.noarch (temporary)
           Requires: fuel-provisioning-scripts >= 10.0.0
           Installed: fuel-provisioning-scripts-9.0.0-1.mos8602.noarch (@mos)
               fuel-provisioning-scripts = 9.0.0-1.mos8602
           Available: fuel-provisioning-scripts-8.0.0-1.mos8183.noarch (temporary)
               fuel-provisioning-scripts = 8.0.0-1.mos8183
           Availab...

Fuel packages list to remove:

fencing-agent
fuel
fuel-agent
fuel-bootstrap-cli
fuel-ha-utils
fuel-library
fuel-migrate
fuel-mirror
fuel-misc
fuel-nailgun
fuel-notify
fuel-openstack-metadata
fuel-ostf
fuel-provisioning-scripts
fuel-rabbit-fence
fuel-release
fuel-setup
fuel-ui
fuel-umm
fuel-utils
fuelmenu
ironic-fa-bootstrap-configs
nailgun-agent
nailgun-mcagents
network-checker
network-checker-debuginfo
python-fuelclient
python-packetary
rubygem-astute
shotgun

ok, it seems we need to delete only some packages in order not to break install tests, so please ignore previous comment

To delete in both mirror:
http://mirror.seed-cz1.fuel-infra.org/mos-repos/centos/mos-master-centos7/os/x86_64/Packages/
http://mirror.seed-cz1.fuel-infra.org/mos-repos/centos/mos9.0-centos7/os/x86_64/Packages/

fencing-agent-8.0.0-1.mos8183.noarch.rpm 29-Dec-2015 11:58 6424
fuel-nailgun-8.0.0-1.mos8183.noarch.rpm 29-Dec-2015 11:58 5856896
fuel-openstack-metadata-8.0.0-1.mos8183.noarch.rpm 29-Dec-2015 11:59 16164
fuel-provisioning-scripts-8.0.0-1.mos8183.noarc..> 29-Dec-2015 11:58 4740
fuel-upgrade-7.0.0-1.mos139.noarch.rpm 29-Dec-2015 11:59 162364

Set to medium cause this issue happened only on timer events and not on patchsets, also should be fixed by https://review.fuel-infra.org/#/c/19431/ since we need to update fuel packages in repos

Fixed by updating fuel packages in mirror