nova-compute fails to auth to libvirtd when `hostname` != `hostname -f`

Bug #1989791 reported by Radosław Piliszek
38
This bug affects 7 people
Affects Status Importance Assigned to Milestone
kolla-ansible
In Progress
Medium
Maksim Malchuk
Antelope
Confirmed
Medium
Unassigned
Bobcat
Confirmed
Medium
Unassigned
Caracal
In Progress
Medium
Maksim Malchuk
Yoga
Confirmed
Medium
Unassigned
Zed
Confirmed
Medium
Unassigned

Bug Description

$ cat /etc/hosts (or from the DNS)
1.2.3.4 hostname.domainname hostname

$ cat /etc/hostname
hostname

$ hostname
hostname

$ hostname -s
hostname

$ hostname -f
hostname.domainname

nova-compute will log:
2022-09-15 18:45:48.608 7 ERROR nova.virt.libvirt.host [-] Connection to libvirt failed: authentication failed: authentication failed: libvirt.libvirtError: authentication failed: authentication failed

(fun fact: the error message says "authentication failed" 4 times - 4 times! - must be a serious violation ;-) )

libvirtd will log:
2022-09-15 16:45:48.606+0000: 3900339: error : virNetSASLSessionServerStep:594 : authentication failed: Failed to start SASL negotiation: -20 (SASL(-13): user not found: unable to canonify user and get auxprops)
2022-09-15 16:45:48.606+0000: 3900339: error : remoteDispatchAuthSaslStep:3985 : authentication failed: authentication failed

The workaround is to modify /etc/hosts (or set for the first time if the original source is DNS) such that the FQDN becomes the alias:

$ cat /etc/hosts
1.2.3.4 hostname hostname.domainname

and then

$ hostname -f
hostname

and then restart nova_libvirtd (probably no need to restart nova_compute but it keeps restarting in a loop so hard to tell ;-) ).

BUT BEWARE!
If this hypervisor existed before, it will remain under the old FQDN name in Placement and previously running instances will not count against the usage!
TODO: report that bug separately

Current workaround to the workaround: shelve and unshelve the instances (though note this will obviously stop and restart them in the process).

description: updated
Revision history for this message
Radosław Piliszek (yoctozepto) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (master)
Changed in kolla-ansible:
status: Triaged → In Progress
Revision history for this message
Przemysław Kuczyński (przemekkuczynski) wrote :

workaround is also run bootstrap so it will create hosts entry

main.yml
customize_etc_hosts: true

Revision history for this message
zhangminglong (zhangminglong) wrote :

I have added "1.2.3.4 hostname hostname.domainname" but the problem still exists

Revision history for this message
Maksim Malchuk (mmalchuk) wrote :

zhangminglong You should rerun deploy or apply the command from the patch provided to solve the problem.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.