Instances are not accessible via networks

Bug #1540470 reported by Kyrylo Romanenko
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Invalid
High
Kyrylo Romanenko

Bug Description

Can not ping or access via ssh instances by floating IP from controller node.
Also can not ping ironic instance by floating IP from controller and by baremetal IP from conductor node.

Steps for Nova instance:
1. Boot TestVM by usual way, link it to admin internal network.
2. Add security rules:
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
3. Assign floating IP to instance.
4. Try to ping instance from Controller:
ping 172.16.51.78
PING 172.16.51.78 (172.16.51.78) 56(84) bytes of data.
From 172.16.51.78 icmp_seq=1 Destination Host Unreachable
The same with ssh:
ssh: connect to host 172.16.51.78 port 22: No route to host

Settings:
Cluster: 1 Controller, 1 Compute, 1 Ironic.
Network verification passed successfully.
Network: Neutron with VLAN segmentation

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "478"
  build_id: "478"
  fuel-nailgun_sha: "ae949905142507f2cb446071783731468f34a572"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "481ed135de2cb5060cac3795428625befdd1d814"
  fuel-nailgun-agent_sha: "b2bb466fd5bd92da614cdbd819d6999c510ebfb1"
  astute_sha: "b81577a5b7857c4be8748492bae1dec2fa89b446"
  fuel-library_sha: "420c6fa5f8cb51f3322d95113f783967bde9836e"
  fuel-ostf_sha: "ab5fd151fc6c1aa0b35bc2023631b1f4836ecd61"
  fuel-mirror_sha: "b62f3cce5321fd570c6589bc2684eab994c3f3f2"
  fuelmenu_sha: "fac143f4dfa75785758e72afbdc029693e94ff2b"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "9f0ba4577915ce1e77f5dc9c639a5ef66ca45896"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "6c6b088a3d52dd0eaf43d59f3a3a149c93a07e7e"

Diagnostic snapshot link: https://drive.google.com/a/mirantis.com/file/d/0B6E70aHvCcRQcHluSC1saHlLQm8/view?usp=sharing

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Kyrylo, connectivity to instances via floating IPs is tested during BVT (https://product-ci.infra.mirantis.net/job/8.0.ubuntu.bvt_2/) 4 times a day for each ISO. If it was broken, we would have known by now.

It's not tested for baremetal instances, though. But in this particular case, the diagnostic snapshot won't be a great help for Neutron team.

That's why I'd really appreciate if you could reproduce the problem for us and give access to your environment to mos-neutron team to perform the RCA.

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

In the mean time, Oleg Bondarev from neutron team is going to analyze logs in the diagnostic snapshot.

Changed in mos:
assignee: MOS Neutron (mos-neutron) → Kyrylo Romanenko (kromanenko)
status: New → Incomplete
Revision history for this message
Oleg Bondarev (obondarev) wrote :

Indeed logs from the snapshot don't tell much also because they are INFO level (not DEBUG).

However I can see that l3 agent was down for some time due to possible problems with RabbitMQ. Please ensure rabbit is healthy (pcs resource) and neutron agents are alive (neutron agent-list) while booting instances and assigning floating ips.

Anyway an env with the problem will help better in such cases.

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Tentatively moving this to 8.0-updates, we'll continue to investigate this, but currently we believe it's a problem with environment.

Changed in mos:
milestone: 8.0 → 8.0-updates
tags: added: move-to-mu
Revision history for this message
Kyrylo Romanenko (kromanenko) wrote :

Really it looks a problem with lab. Now i can not reproduce this failure on similar environment.

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Thanks, Kyrylo! I'm closing this as Invalid, please re-open it again, if needed.

Changed in mos:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.