Connection pool is full, discarding connection

Bug #1479183 reported by zhu zhu
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
oslo.vmware
Fix Released
Medium
zhu zhu

Bug Description

Running concurrently I have seen the deployment throughput fluctuate meaningfully.
The issue is that I often see the deployment request become slower, often managed through compute service but not spawn on VMware. When this happen I see from nova-compute log returning warning message like the ones below

2015-07-24 14:30:36.110 3921 WARNING urllib3.connectionpool [-] Connection pool is full, discarding connection: 192.168.80.7
2015-07-24 14:30:42.639 3921 INFO nova.scheduler.client.report [req-87e96a07-2d21-48db-ac42-4c22b80c23e4 54391aa0a12c4c94bf332b22dc9e00c9 7c85a93836de421cb1df5614a8a01cc9 - - -] Compute_service record updated for ('vcenter-PROV_PVT2', domain-c118(PROV_PVT2))
2015-07-24 14:30:52.555 3921 WARNING urllib3.connectionpool [-] Connection pool is full, discarding connection: 192.168.80.7
2015-07-24 14:30:59.938 3921 WARNING urllib3.connectionpool [-] Connection pool is full, discarding connection: 192.168.80.7
2015-07-24 14:31:00.052 3921 WARNING urllib3.connectionpool [-] Connection pool is full, discarding connection: 192.168.80.7
2015-07-24 14:31:03.106 3921 INFO nova.scheduler.client.report [req-87e96a07-2d21-48db-ac42-4c22b80c23e4 54391aa0a12c4c94bf332b22dc9e00c9 7c85a93836de421cb1df5614a8a01cc9 - - -] Compute_service record updated for ('vcenter-PROV_PVT2', domain-c118(PROV_PVT2))
2015-07-24 14:31:06.875 3921 WARNING urllib3.connectionpool [-] Connection pool is full, discarding connection: 192.168.80.7

192.168.80.7 is the vcenter IP.

Looks like the Vim object will use python requests module to communicate to vcenter to do the soap calls. The requests module will holde the httpcollectionpool which default is only have pool size 10. Which in the multiple concurrent cases, the pool is obviously smaller. So oslo.vmware need to expose the pool size to upper layer applications.

Revision history for this message
zhu zhu (zhuzhubj) wrote :
Revision history for this message
zhu zhu (zhuzhubj) wrote :
Changed in oslo.vmware:
importance: Undecided → Medium
status: New → In Progress
assignee: nobody → zhu zhu (zhuzhubj)
Changed in oslo.vmware:
status: In Progress → Fix Committed
Changed in oslo.vmware:
milestone: none → 1.21.0
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.