Testflinger

traceback when checkbox submits results to C3 from testflinger agent

Bug #1957187 reported by Jeff Lane 
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Next Generation Checkbox (CLI)
Fix Released
Undecided
Maciej Kisielewski
Next Generation Checkbox (CLI) 1.17.0
Testflinger
New
Undecided
Paul Larson

Bug Description

This just happened on a test run. Oddly enough, the SecureID seemed to work because checkbox was able so successfully submit the results (the C3 link works correctly). Filed against testflinger because at first I thought that, for some reason the SID isn't in the environment there, but since checkbox was able to submit the results using the SID, I'm not sure exactly why it is now throwing a traceback about the SID being invalid.

file:///home/ubuntu/.local/share/checkbox-ng/submission_2022-01-12T02.12.02.061016.html
file:///home/ubuntu/.local/share/checkbox-ng/submission_2022-01-12T02.12.02.061016.junit.xml
file:///home/ubuntu/.local/share/checkbox-ng/submission_2022-01-12T02.12.02.061016.tar.xz
https://certification.canonical.com/submissions/status/117638
WARNING:checkbox-ng.launcher.stages:Invalid secure_id
Retrying in 30s
WARNING:checkbox-ng.launcher.stages:Invalid secure_id
Retrying in 60s
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/checkbox_ng/launcher/stages.py", line 485, in _export_results
    self._create_transport(params['transport'])
  File "/usr/lib/python3/dist-packages/checkbox_ng/launcher/stages.py", line 437, in _create_transport
    self.transports[transport] = cls(url, options)
  File "/usr/lib/python3/dist-packages/checkbox_ng/certification.py", line 62, in __init__
    self._validate_secure_id(self._secure_id)
  File "/usr/lib/python3/dist-packages/checkbox_ng/certification.py", line 124, in _validate_secure_id
    raise InvalidSecureIDError(
plainbox.impl.transport.InvalidSecureIDError: 'secure_id must be 15-character (or more) alphanumeric string'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/checkbox-cli", line 11, in <module>
    load_entry_point('checkbox-ng==1.16.0.dev0', 'console_scripts', 'checkbox-cli')()
  File "/usr/lib/python3/dist-packages/checkbox_ng/launcher/checkbox_cli.py", line 139, in main
    subcmd.invoked(ctx)
  File "/usr/lib/python3/dist-packages/checkbox_ng/launcher/subcommands.py", line 225, in invoked
    self._export_results()
  File "/usr/lib/python3/dist-packages/checkbox_ng/launcher/stages.py", line 515, in _export_results
    self.sa.config.transports['c3'].pop('secure_id')
KeyError: 'secure_id'

Related branches

~kissiel/checkbox-ng:fix-1957187
Sylvain Pineau (community): Approve
Diff: 16 lines (+5/-0)
1 file modified
checkbox_ng/launcher/stages.py (+5/-0)
Revision history for this message
Jeff Lane  (bladernr) wrote :

Added checkbox-ng too because I'm not sure right now where the problem is, since checkbox gets the SID from the environment in TF...

I should add that this is using checkbox remote.

I've attached a copy of the YAML that is being generated. This YAML has worked without this issue, the traceback is new behaviour.

Revision history for this message
Paul Larson (pwlars) wrote :

Can you get a copy of the launcher that is getting generated? It looks like it doesn't have something for the secure_id in the checkbox launcher, but it should be pulling it from the environment on doubletusk device agent.

Changed in testflinger:
assignee: nobody → Paul Larson (pwlars)
Revision history for this message
Jeff Lane  (bladernr) wrote (last edit ):

276 [launcher]
277 app_id = com.canonical.certification:certification-server
278 launcher_version = 1
279 stock_reports = submission_files, certification
280 local_submission = yes
281
282 [test plan]
283 unit = com.canonical.certification::egx-regression-tests
284 forced = yes
285
286 [test selection]
287 forced = yes
288
289 [ui]
290 output = hide-resource-and-attachment
291 type = silent
292
293 [config]
294 config_filename = canonical-certification.conf
295
296 [transport:certification]
297 type = submission-service
298 secure_id = $HEXR_DEVICE_SECURE_ID
299
300 [report:upload to submission service]
301 transport = certification
302 exporter = tar

This is the launcher that's generated, and has been used for a long time now... like I said, this just started happening today. Even last week this did not occur.

I am running a couple retests too, just to be sure this wasn't some weird one-off glitch.

Revision history for this message
Paul Larson (pwlars) wrote :

So I ran a quick sanity test on doubletusk to just do this:
  test_cmds: |
    echo "HEXR_DEVICE_SECURE_ID is $HEXR_DEVICE_SECURE_ID"

Everything worked fine and it properly filled that env var from the environment. However it's definitely not filling it in for that launcher script. So it looks like everything is ok from the testflinger side, but something is preventing that value from getting filled in.

In the output from the test run, did you get any error when cheetah ran to try to fill in that value?

Revision history for this message
Maciej Kisielewski (kissiel) wrote :

I think this is a bug in checkbox logic. I'll let myself work on it.

Changed in checkbox-ng:
assignee: nobody → Maciej Kisielewski (kissiel)
status: New → In Progress
Revision history for this message
Jeff Lane  (bladernr) wrote : Re: [Bug 1957187] Re: traceback when checkbox submits results to C3 from testflinger agent

I also wonder if this is just an oddity I stumbled on. I've kicked
off a couple additional runs to see if it reoccurs but of the 40 or so
I did before this, not one threw a traceback. So... I dunno... I need
to confer with my BOFH excuse generator to find a plausible
explanation.

Sylvain Pineau (sylvain-pineau)
Changed in checkbox-ng:
milestone: none → 1.17.0
Sylvain Pineau (sylvain-pineau)
Changed in checkbox-ng:
status: In Progress → Fix Committed
Devices Certification Bot (ce-certification-qa)
Changed in checkbox-ng:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.