ext-ports accumulate after mojo tests: IpAddressGenerationFailureClient: No more IP addresses available on network
Bug #1656306 reported by
Ryan Beisner
This bug affects 1 person
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| OpenStack Charm Test Infra |
Fix Released
|
High
|
Ryan Beisner | ||
Bug Description
Following many iterations of mojo spec full deploy jobs, OSCI slaves are clogged with residual ext-port ports and IP address assignments. This leads to depletion of the the DHCP pools.
neutronclient.
Neutron server returns request_ids: ['req-6bb32e95-
ubuntu@
549
Related branches
lp://qastaging/~1chb1n/ubuntu-openstack-ci/use-port-cleanup-tool
- Andrew McLeod (community): Approve
-
Diff: 836 lines (+196/-227)19 files modified.bzrignore (+1/-0)
common/osci_utils.py (+4/-4)
job-parts/build_deploy_test.sh (+1/-1)
job-parts/build_deployer_quickcheck.sh (+1/-1)
job-parts/build_git_deploy_test.sh (+1/-1)
job-parts/build_juju_deploy_single_charm.sh (+2/-2)
job-parts/build_mojo_runner.sh (+9/-5)
job-parts/build_osi.sh (+1/-1)
job-parts/deprecated/Z_DEPRECATE_build_mojo_runner_baremetal.sh (+0/-69)
job-parts/deprecated/part_juju_cleanup.sh (+0/-22)
job-parts/juju1_bighammer_destroy.sh (+32/-27)
job-parts/juju1_deploy_ok.sh (+45/-35)
job-parts/osci_job_common.sh (+33/-16)
job-parts/osci_mojo_common.sh (+21/-35)
job-parts/trigger_by_mojo_group.sh (+6/-0)
populate/mappings.yaml (+9/-7)
tools/juju_status_health_check.py (+1/-1)
tools/requirements.txt (+10/-0)
tools/tox.ini (+19/-0)
Revision history for this message
| Ryan Beisner (1chb1n) wrote | #1 |
| summary: |
- neutronclient.common.exceptions.IpAddressGenerationFailureClient: No - more IP addresses available on network + ext-ports accumulate after mojo tests: IpAddressGenerationFailureClient: + No more IP addresses available on network |
| Changed in charm-test-infra: | |
| status: | New → Confirmed |
| assignee: | nobody → Ryan Beisner (1chb1n) |
| importance: | Undecided → High |
Revision history for this message
| Ryan Beisner (1chb1n) wrote | #2 |
| Changed in charm-test-infra: | |
| status: | Confirmed → In Progress |
Revision history for this message
| Ryan Beisner (1chb1n) wrote | #3 |
| Changed in charm-test-infra: | |
| status: | In Progress → Fix Committed |
Revision history for this message
| Ryan Beisner (1chb1n) wrote | #4 |
| Changed in charm-test-infra: | |
| milestone: | none → 17.02 |
| Changed in charm-test-infra: | |
| status: | Fix Committed → Fix Released |
Revision history for this message
| Ryan Beisner (1chb1n) wrote | #5 |
To post a comment you must log in.
Finally circling back to this issue. We've have a number of jenkins slaves offline due to this.
⟫ neutron port-list | grep "juju-osci-
818
I have a WIP to add ext-port cleanup on all jobs (pre and post), to ensure these are always reaped.