Neutron DHCP Agent KeyError during NeutronNetworks.create_and_delete_subnets test execution

Bug #1544069 reported by Sergey Arkhipov
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Status tracked in 10.0.x
Performance QA
Won't Fix
MOS Maintenance
Won't Fix
MOS Neutron

Bug Description

During execution of 'NeutronNetworks.create_and_delete_subnets' Rally test scenario, I observe following stacktraces on node-13 (was observed ~2016-02-10T12:33:34). MOS 8.0 RC1 (build 529) is used.

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/neutron/agent/dhcp/", line 115, in call_driver
    getattr(driver, action)(**action_kwargs)
  File /usr/lib/python2.7/dist-packages/neutron/agent/linux/, line 206, in enable
  File /usr/lib/python2.7/dist-packages/neutron/agent/linux/, line 414, in spawn_process
  File /usr/lib/python2.7/dist-packages/neutron/agent/linux/, line 423, in _spawn_or_reload_process
  File /usr/lib/python2.7/dist-packages/neutron/agent/linux/, line 450, in _output_config_files
  File /usr/lib/python2.7/dist-packages/neutron/agent/linux/, line 746, in _output_opts_file
    options, subnet_index_map = self._generate_opts_per_subnet()
  File /usr/lib/python2.7/dist-packages/neutron/agent/linux/, line 796, in _generate_opts_per_subnet
    subnet_dhcp_ip = subnet_to_interface_ip[]
KeyError: u'0b2f1504-9a27-4919-ba6d-6065c8b23432'

Test passed without any problems.

  10 baremetal nodes
    * 2 LMA nodes
    * 5 computes (KVM)
    * 3 controllers
      - Neutron with VLAN segmentation, no DVR and no L3 HA
      - Cinder LVM over iSCSI for volumes (Ceph is not used at all)
      - Nova quotas are not enabled

Test scenario:
  "NeutronNetworks.create_and_delete_subnets": [
      "runner": {
        "type": "constant",
        "concurrency": 10,
        "times": 18
      "args": {
        "network_create_args": {},
        "subnet_cidr_start": "",
        "subnets_per_network": 5,
        "subnet_create_args": {}
      "sla": {
        "failure_rate": {
          "max": 0
      "context": {
        "users": {
          "project_domain": "default",
          "users_per_tenant": 2,
          "tenants": 2,
          "resource_management_workers": 30,
          "user_domain": "default"
        "quotas": {
          "neutron": {
            "subnet": -1,
            "network": -1

    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "529"
  build_id: "529"
  fuel-nailgun_sha: "baec8643ca624e52b37873f2dbd511c135d236d9"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "658be72c4b42d3e1436b86ac4567ab914bfb451b"
  fuel-nailgun-agent_sha: "b2bb466fd5bd92da614cdbd819d6999c510ebfb1"
  astute_sha: "b81577a5b7857c4be8748492bae1dec2fa89b446"
  fuel-library_sha: "e2d79330d5d708796330fac67722c21f85569b87"
  fuel-ostf_sha: "3bc76a63a9e7d195ff34eadc29552f4235fa6c52"
  fuel-mirror_sha: "fb45b80d7bee5899d931f926e5c9512e2b442749"
  fuelmenu_sha: "e071216cb214e34b4d861478033425ee6a54a3be"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "a43cf96cd9532f10794dce736350bf5bed350e9d"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "a365f05b903368225da3fea9aa42afc1d50dc9b4"

Please find diagnostic snapshot:

Log of rally execution:

tags: added: scale
Ilya Kutukov (ikutukov)
Changed in fuel:
status: New → Confirmed
assignee: nobody → MOS Maintenance (mos-maintenance)
tags: added: area-mos
Revision history for this message
Vitaly Sedelnik (vsedelnik) wrote :

Won't Fix for 8.0-updates because of Medium importance. Nominated for 9.0

Revision history for this message
Bug Checker Bot (bug-checker) wrote : Autochecker

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

actual result

expected result

steps to reproduce

For more detailed information on the contents of each of the listed sections see

tags: added: need-info
Revision history for this message
Alexander Ignatov (aignatov) wrote :

Test passed without any problems, decreasing to Low importance.

Changed in mos:
milestone: none → 10.0
status: New → Confirmed
importance: Undecided → Low
assignee: nobody → MOS Neutron (mos-neutron)
no longer affects: fuel
no longer affects: fuel/8.0.x
no longer affects: fuel/mitaka
Changed in mos:
assignee: MOS Maintenance (mos-maintenance) → MOS Neutron (mos-neutron)
tags: added: move-to-10.0
Revision history for this message
Alexander Ignatov (aignatov) wrote :

Dear Performance QA team, could you please recheck this Rally test once again in 9.0 GA build and please provide logs/snapshot if this issue occurs there.

Changed in mos:
milestone: 10.0 → 9.1
status: Confirmed → Incomplete
assignee: MOS Neutron (mos-neutron) → Performance QA (performance-qa)
Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

Moving to Invalid after a month in Incomplete.

Changed in mos:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.