Test result failed to display

Relevant lines from log file:

<snip>
ERROR:django.request:Internal Server Error: /lava-server/dashboard/streams/anonymous/stylesen/bundles/64adf2a942a9b579ef07e1c61ca2c7521253e14e/
Traceback (most recent call last):
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/srv/lava/.<email address hidden>/dashboard_app/views/__init__.py", line 270, in bundle_detail
    "bundle_stream": bundle_stream
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/views/generic/list_detail.py", line 150, in object_detail
    response = HttpResponse(t.render(c), mimetype=mimetype)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 140, in render
    return self._render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 134, in _render
    return self.nodelist.render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 823, in render
    bit = self.render_node(node, context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 837, in render_node
    return node.render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/loader_tags.py", line 123, in render
    return compiled_parent._render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 134, in _render
    return self.nodelist.render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 823, in render
    bit = self.render_node(node, context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 837, in render_node
    return node.render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/loader_tags.py", line 123, in render
    return compiled_parent._render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 134, in _render
    return self.nodelist.render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 823, in render
    bit = self.render_node(node, context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 837, in render_node
    return node.render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/loader_tags.py", line 123, in render
    return compiled_parent._render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 134, in _render
    return self.nodelist.render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 823, in render
    bit = self.render_node(node, context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/base.py", line 837, in render_node
    return node.render(context)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/template/loader_tags.py", line 123, in render
    return compiled_parent._render(context)
 ...

Traceback (most recent call last):
  File "/srv/lava/.<email address hidden>/dashboard_app/models.py", line 495, in deserialize
    self._do_deserialize(prefer_evolution)
  File "/srv/lava/.<email address hidden>/dashboard_app/models.py", line 516, in _do_deserialize
    helper.deserialize(self, prefer_evolution)
  File "/srv/lava/.<email address hidden>/dashboard_app/helpers.py", line 787, in deserialize
    fmt, doc = DocumentIO.load(s_bundle.content)
  File "/srv/lava/.cache/eggs/linaro_dashboard_bundle-1.8-py2.7.egg/linaro_dashboard_bundle/io.py", line 119, in load
    fmt = cls.check(doc)
  File "/srv/lava/.cache/eggs/linaro_dashboard_bundle-1.8-py2.7.egg/linaro_dashboard_bundle/io.py", line 223, in check
    raise DocumentFormatError(fmt)
DocumentFormatError: Unrecognized or missing document format

So, this was not fixed with the upgrade.

I discovered that fastmodels01, which is the node that runs the device involved in this case (rtsm_ve-armv8_01), was set to run the code from production instead of the code that was "blessed" for deployment (i.e. lp:lava-manifest). So we had a results bundle produced by the trunk code, and the old dashboard on VLO trying to deserialize that and crashing.

Now, after the upgrade, it does not crash with an Error 500 anymore, but it says "Deserialization error", and shows some stack trace. Maybe it has this error cached from when fastmodels01 was running more recent code than control?

May be running the same job again (of course with a different bundle stream
URL) will help?

--
http://www.stylesen.org/
Sent from my Nexus4
On Apr 16, 2013 11:50 PM, "Antonio Terceiro" <email address hidden>
wrote:

> So, this was not fixed with the upgrade.
>
> I discovered that fastmodels01, which is the node that runs the device
> involved in this case (rtsm_ve-armv8_01), was set to run the code from
> production instead of the code that was "blessed" for deployment (i.e.
> lp:lava-manifest). So we had a results bundle produced by the trunk
> code, and the old dashboard on VLO trying to deserialize that and
> crashing.
>
> Now, after the upgrade, it does not crash with an Error 500 anymore, but
> it says "Deserialization error", and shows some stack trace. Maybe it
> has this error cached from when fastmodels01 was running more recent
> code than control?
>
> --
> You received this bug notification because you are a member of Linaro
> Validation Team, which is subscribed to LAVA Dashboard.
> https://bugs.launchpad.net/bugs/1169471
>
> Title:
> Test result failed to display
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/lava-dashboard/+bug/1169471/+subscriptions
>

On Tue, Apr 16, 2013 at 07:01:39PM -0000, Senthil Kumaran S wrote:
> May be running the same job again (of course with a different bundle stream
> URL) will help?

Tyler re-scheduled it:
http://validation.linaro.org/lava-server/scheduler/job/51895

Let's what happens there

--
Antonio Terceiro
Software Engineer - Linaro
http://www.linaro.org

Antonio Terceiro <email address hidden> writes:

> Now, after the upgrade, it does not crash with an Error 500 anymore, but
> it says "Deserialization error", and shows some stack trace. Maybe it
> has this error cached from when fastmodels01 was running more recent
> code than control?

You can request a second attempt to deserialize a bundle with
lava-dashboard-tool fwiw.

On Tue, Apr 16, 2013 at 08:19:18PM -0000, Michael Hudson-Doyle wrote:
> Antonio Terceiro <email address hidden> writes:
>
> > Now, after the upgrade, it does not crash with an Error 500 anymore, but
> > it says "Deserialization error", and shows some stack trace. Maybe it
> > has this error cached from when fastmodels01 was running more recent
> > code than control?
>
> You can request a second attempt to deserialize a bundle with
> lava-dashboard-tool fwiw.

Brilliant - that fixed it \o/

https://validation.linaro.org/lava-server/dashboard/streams/private/team/linaro/android-daily/bundles/7e3c826df7c6f493c1107aaa33d27fc8c7a9dd82/

FTR the command line was

$ lava-dashboard-tool deserialize --dashboard-url https://<email address hidden>/lava-server/RPC2/ 7e3c826df7c6f493c1107aaa33d27fc8c7a9dd82

 status fixreleased

--
Antonio Terceiro
Software Engineer - Linaro
http://www.linaro.org

On Apr 17, 2013 3:10 AM, "Antonio Terceiro" <email address hidden>
wrote:
>
> On Tue, Apr 16, 2013 at 08:19:18PM -0000, Michael Hudson-Doyle wrote:
> > Antonio Terceiro <email address hidden> writes:
> >
> > > Now, after the upgrade, it does not crash with an Error 500 anymore,
but
> > > it says "Deserialization error", and shows some stack trace. Maybe it
> > > has this error cached from when fastmodels01 was running more recent
> > > code than control?
> >
> > You can request a second attempt to deserialize a bundle with
> > lava-dashboard-tool fwiw.
>
> Brilliant - that fixed it \o/

That's awesome!

Daily test result on Linux Linaro dashboard is still unavailable:

http://validation.linaro.org/lava-server/dashboard/image-reports/

On Fri, Apr 19, 2013 at 07:56:16AM -0000, Botao Sun wrote:
> Daily test result on Linux Linaro dashboard is still unavailable:
>
> http://validation.linaro.org/lava-server/dashboard/image-reports/

Senthil, can you please investigate?

Botao, it would be nice if you could list explicitly which bundles do
not show properly since you already want looking anyway.

--
Antonio Terceiro
Software Engineer - Linaro
http://www.linaro.org

On 19 April 2013 19:49, Antonio Terceiro <email address hidden> wrote:
> On Fri, Apr 19, 2013 at 07:56:16AM -0000, Botao Sun wrote:
>> Daily test result on Linux Linaro dashboard is still unavailable:
>>
>> http://validation.linaro.org/lava-server/dashboard/image-reports/
>
> Senthil, can you please investigate?
>
> Botao, it would be nice if you could list explicitly which bundles do
> not show properly since you already want looking anyway.

I clarified with Botao/Senthil on irc. It's a regression in the script
to submit job to LAVA (I fixed it earlier today).
The dashboard is safe :) results should be back to normal.

@Fathi, Now the daily test result has been shown up on Samsung Arndale, ARM Versatile Express A9, TI Panda 4430 and 4460. This bug can be considered as "Fix Released" now.