PlainBox (Toolkit)

results refer to non-existent temp files and there seems to be no cache of results

Bug #1297344 reported by Jeff Lane  on 2014-03-25

This bug report is a duplicate of: Bug #1312731: most clients use single-session API. Edit Remove

This bug report was converted into a question: question #247501: results refer to non-existent temp files and there seems to be no cache of results.

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	PlainBox (Toolkit)	Confirmed	Undecided	Unassigned

Bug Description

What SHOULD happen:
Testing session data (command output, log files, etc) should be preserved after the test run. I understand that saving "ALL" sessions is a bit more complex, but for now, we're not even saving a single run.

What ACTUALLY happens:
Session data is not saved after a completed run, even if there are failures that would need to be investigated, making that data invaluable.

Plainbox versions:
ii canonical-certification-server 0.17.9+bzr2823~ubuntu14.04.1 all Canonical Certification Server.
ii plainbox-insecure-policy 0.17.9+bzr2823~ubuntu14.04.1 all Insecure PolicyKit action file for PlainBox
ii plainbox-provider-certification-server 0.17.9+bzr2823~ubuntu14.04.1 all Server Certification
ii plainbox-provider-checkbox 0.17.9+bzr2823~ubuntu14.04.1 amd64 CheckBox jobs
ii plainbox-provider-resource-generic 0.17.9+bzr2823~ubuntu14.04.1 amd64 CheckBox generic resource jobs provider
ii python3-plainbox 0.17.9+bzr2823~ubuntu14.04.1 all Simple replacement for CheckBox

I ran the latest dev version of canonical-certification-server and at the end, I noticed that the results file had tracebacks in it.

These tracebacks referred to output in /tmp that no longer exist.

<comment>Traceback (most recent call last):
  File "/tmp/nest-ryoz4v59.11eb509ebecd40680a2c542695af56491bf9cb92530d47ab123e5e2f308b943e/network", line 648, in <module>
    sys.exit(main())
  File "/tmp/nest-ryoz4v59.11eb509ebecd40680a2c542695af56491bf9cb92530d47ab123e5e2f308b943e/network", line 644, in main
    return args.func(args)
  File "/tmp/nest-ryoz4v59.11eb509ebecd40680a2c542695af56491bf9cb92530d47ab123e5e2f308b943e/network", line 450, in interface_test
    if "example.com" in test_target:
TypeError: argument of type 'NoneType' is not iterable
</comment>

However, now that the test run is complete, those files no longer exist in /tmp

I next looked in .local/share/plainbox but there is only results files there:

root@CCS-SUT:~# ls .local/share/plainbox/ -a
. .. results.html submission.xml

I then checked the cache dir:

root@CCS-SUT:~# ls -la .cache/plainbox/*
total 8
drwxr-xr-x 2 root root 4096 Mar 25 10:45 .
drwxr-xr-x 3 root root 4096 Mar 25 10:30 ..
lrwxrwxrwx 1 root root 52 Mar 25 10:44 last-session -> /root/.cache/plainbox/sessions/pbox-bkd5fdar.session

And the only thing there is a symlink to a non-existent session file.

So there is no way for me to go back and actually look at the full output from the network tests.

See original description

Revision history for this message

Zygmunt Krynicki (zyga) wrote on 2014-03-25: Re: [Bug 1297344] [NEW] results refer to non-existent temp files and there seems to be no cache of results

Thanks for reporting this bug.

The temporary directory is a container for symlinks to executables from
various providers. In this case the network script from of of the provider
crashed.

The symlink in ~/cache/plainbox/sessions is broken because some other
session had removed the previous one. PlainBox has the capacity to store
all the sessions but we're not using that in checkbox-ng at this time.

Thanks
ZK

Revision history for this message

Zygmunt Krynicki (zyga) wrote on 2014-03-28:

I have a feeling this is not a bug but a support question. Is there something actionable here?

Changed in checkbox:
status:	New → Incomplete

Revision history for this message

Zygmunt Krynicki (zyga) wrote on 2014-04-23:

This is a support question.

affects:	checkbox → plainbox
Changed in plainbox:
status:	Incomplete → Invalid

Revision history for this message

Jeff Lane  (bladernr) wrote on 2014-04-23:

Sorry, I suck at bug followup... from the question this was converted to:

Not so... this is on the current bits for checkbox-ng and canonical-certification-server, on a brand new 14.04 install and the first time c-c-s has been run:

ubuntu@supermicro:~$ ll .cache/plainbox/sessions/
total 8
drwxrwxr-x 2 ubuntu ubuntu 4096 Apr 23 14:31 ./
drwxrwxr-x 3 ubuntu ubuntu 4096 Apr 23 14:29 ../
lrwxrwxrwx 1 ubuntu ubuntu 59 Apr 23 14:29 last-session -> /home/ubuntu/.cache/plainbox/sessions/pbox-wuvyk69_.session
ubuntu@supermicro:~$ ll .local/share/plainbox/
total 868
drwxrwxr-x 2 ubuntu ubuntu 4096 Apr 23 14:31 ./
drwxrwxr-x 3 ubuntu ubuntu 4096 Apr 23 14:31 ../
-rw-rw-r-- 1 ubuntu ubuntu 399181 Apr 23 14:31 results.html
-rw-rw-r-- 1 ubuntu ubuntu 476919 Apr 23 14:31 submission.xml

I ran c-c-s and after the run checked the cache dir, and found that there is no session data saved. So no test output or stderr, none of the files that were generally written to $CHECKBOX_DATA by various info or other tests, nada. This is my original bug. When testing fails, no session data is being saved at all, so it is impossible for me to debug testing issues in the field.

Note that last-session is a broken symlink to a non-existant session directory in .cache/plainbox.

Additionally, there is no logging at all as there used to be with checkbox, so no canonical-certification-server.log (like checkbox-certification.log that had all the runtime output from checkbox itself.

ubuntu@supermicro:~$ COLUMNS=125 dpkg -l |grep plainbox
ii plainbox-insecure-policy 0.6~dev+bzr2937+p all policykit policy required to use plainbox (insecure ver
ii plainbox-provider-certifi 0.1~dev+bzr2937+p all Server Certification
ii plainbox-provider-checkbo 0.5~dev+bzr2937+p amd64 CheckBox provider for PlainBox
ii plainbox-provider-resourc 0.4~dev+bzr2937+p amd64 CheckBox generic resource jobs provider
ii python3-plainbox 0.6~dev+bzr2937+p all toolkit for software and hardware testing (python3 modu
ubuntu@supermicro:~$ COLUMNS=125 dpkg -l |grep checkbox
ii checkbox-ng 0.4~dev+bzr2937+p all PlainBox based test runner
ii python3-checkbox-ng 0.4~dev+bzr2937+p all PlainBox based test runner (Python 3 library)
ii python3-checkbox-support 0.3~dev+bzr2937+p all collection of Python modules used by PlainBox providers

Sorry, I suck at bug followup... from the question this was converted to:

Not so... this is on the current bits for checkbox-ng and canonical-certification-server, on a brand new 14.04 install and the first time c-c-s has been run:

Note that last-session is a broken symlink to a non-existant session directory in .cache/plainbox.

Changed in plainbox:
status:	Invalid → Confirmed
description:	updated
affects:	plainbox → checkbox-ng

Revision history for this message

Jeff Lane  (bladernr) wrote on 2014-04-23:

I also moved this (for now) from plainbox to checkbox-ng as that is what we're actually running, however, IIRC this is the same regardless of which backend is being used (unless plainbox has improved on this since I last used it)

Zygmunt Krynicki (zyga) on 2014-04-29

affects:

checkbox-ng → plainbox

Revision history for this message

Jeff Lane  (bladernr) wrote on 2014-04-29:

Just to be clear: Steps to reproduce:

Install canonical-certification-server
run tests
exit canonical-certification-server
look in .share and .local for ANY test output, test generated logs, or plainbox session logs, or any logging at all.