networking-midonet

tempest jobs failing with midonet-cluster complaining about keystone

Bug #1684338 reported by YAMAMOTO Takashi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
networking-midonet
Fix Released
Critical
YAMAMOTO Takashi
networking-midonet 5.0.0

Bug Description

eg. http://logs.openstack.org/11/458011/1/check/gate-tempest-dsvm-networking-midonet-ml2-ubuntu-xenial/86d989d/logs/midonet-cluster.txt.gz

2017.04.19 10:50:50.132 ERROR [rest-api-55] auth Login authorization error occurred for user null
java.net.ConnectException: Connection refused (Connection refused)
 at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_121]
 at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_121]
 at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_121]
 at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_121]
 at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_121]
 at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_121]
 at java.net.Socket.connect(Socket.java:538) ~[na:1.8.0_121]
 at sun.net.NetworkClient.doConnect(NetworkClient.java:180) ~[na:1.8.0_121]
 at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) ~[na:1.8.0_121]
 at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) ~[na:1.8.0_121]
 at sun.net.www.http.HttpClient.<init>(HttpClient.java:211) ~[na:1.8.0_121]
 at sun.net.www.http.HttpClient.New(HttpClient.java:308) ~[na:1.8.0_121]
 at sun.net.www.http.HttpClient.New(HttpClient.java:326) ~[na:1.8.0_121]
 at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1202) ~[na:1.8.0_121]
 at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138) ~[na:1.8.0_121]
 at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032) ~[na:1.8.0_121]
 at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:966) ~[na:1.8.0_121]
 at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1316) ~[na:1.8.0_121]
 at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1291) ~[na:1.8.0_121]
 at com.sun.jersey.client.urlconnection.URLConnectionClientHandler$1$1.getOutputStream(URLConnectionClientHandler.java:238) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.api.client.CommittingOutputStream.commitStream(CommittingOutputStream.java:117) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.api.client.CommittingOutputStream.write(CommittingOutputStream.java:89) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.api.client.filter.LoggingFilter$LoggingOutputStream.write(LoggingFilter.java:110) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:1848) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.fasterxml.jackson.core.json.UTF8JsonGenerator.flush(UTF8JsonGenerator.java:1041) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:854) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:650) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.api.client.RequestWriter.writeRequestEntity(RequestWriter.java:300) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:217) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 ... 39 common frames omitted
Wrapped by: com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused (Connection refused)
 at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.api.client.filter.LoggingFilter.handle(LoggingFilter.java:217) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at org.midonet.cluster.auth.keystone.KeystoneClient$$anonfun$org$midonet$cluster$auth$keystone$KeystoneClient$$post$1.apply(KeystoneClient.scala:412) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at org.midonet.cluster.auth.keystone.KeystoneClient.tryRequest(KeystoneClient.scala:422) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 ... 32 common frames omitted
Wrapped by: org.midonet.cluster.auth.keystone.KeystoneConnectionException: Connection to Keystone failed
 at org.midonet.cluster.auth.keystone.KeystoneClient.tryRequest(KeystoneClient.scala:430) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at org.midonet.cluster.auth.keystone.KeystoneClient.org$midonet$cluster$auth$keystone$KeystoneClient$$post(KeystoneClient.scala:405) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at org.midonet.cluster.auth.keystone.KeystoneClient$$anonfun$authenticate$1.applyOrElse(KeystoneClient.scala:154) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at org.midonet.cluster.auth.keystone.KeystoneClient$$anonfun$authenticate$1.applyOrElse(KeystoneClient.scala:132) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at org.midonet.cluster.auth.keystone.KeystoneClient.org$midonet$cluster$auth$keystone$KeystoneClient$$withVersion(KeystoneClient.scala:370) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at org.midonet.cluster.auth.keystone.KeystoneClient.authenticate(KeystoneClient.scala:132) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at org.midonet.cluster.auth.keystone.KeystoneService.authenticate(KeystoneService.scala:58) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at org.midonet.cluster.rest_api.auth.LoginFilter.doFilter(LoginFilter.java:115) ~[midonet-cluster.jar:5.6-SNAPSHOT]
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.midonet.cluster.services.rest_api.CorsFilter.doFilter(CorsFilter.scala:96) [midonet-cluster.jar:5.6-SNAPSHOT]
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) [midonet-cluster.jar:5.6-SNAPSHOT]
 at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:120) [midonet-cluster.jar:5.6-SNAPSHOT]
 at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1667) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1114) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1048) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.server.Server.handle(Server.java:517) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:302) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:245) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) [midonet-cluster.jar:5.6-SNAPSHOT]
 at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) [midonet-cluster.jar:5.6-SNAPSHOT]
 at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]

Tags: gate-failure
YAMAMOTO Takashi (yamamoto)
tags: added: gate-failure
YAMAMOTO Takashi (yamamoto)
Changed in neutron:
assignee: nobody → YAMAMOTO Takashi (yamamoto)
status: New → In Progress
Revision history for this message
Ihar Hrachyshka (ihar-hrachyshka) wrote :

It's not clear why it's a Neutron issue and not Midonet, so I changed the component to networking-midonet for now. Feel free to move back or add neutron to the list of affected projects if you have more information that points to neutron.

affects: neutron → networking-midonet
Revision history for this message
YAMAMOTO Takashi (yamamoto) wrote :

Ihar,
i guess it was just a simple mistake when i filed this bug.
sorry for noise.

Revision history for this message
YAMAMOTO Takashi (yamamoto) wrote :

fixed by https://review.openstack.org/#/c/457173/
("devstack: Adapt to the recent uwsgi switch" part)

Changed in networking-midonet:
importance: Undecided → Critical
milestone: none → 5.0.0
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-midonet 5.0.0.0b2

This issue was fixed in the openstack/networking-midonet 5.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.