"testflinger-cli artifacts" should check checksum after artifacts be downloaded

Yes, there's a possible window where your job can be finished executing all the commands, but the artifacts tarball is not yet available on the server. This is because the agent automatically takes care of that part for you. In fact, it's also smart enough to retry it if the upload fails for any reason. At the time, I was thinking it would be better to return control to the user as soon as the test run is complete, and let the agent take care of all this in the background. But if you have something monitoring the test run and trying to collect the results (like jenkins, etc), you want to get those artifacts before finishing. The way I deal with this in our jenkins jobs is to do something like this:

testflinger-cli artifacts ${{JOB_ID}} || (sleep 30 && testflinger-cli artifacts ${{JOB_ID}})

I'm not sure about making poll hang indefinitely until the artifacts are confirmed to be uploaded - it's actually possible that the agent has moved on to processing other test jobs if that happens, but maybe we can try once and if it fails, warn the user that they may need to wait for their artifacts to be available for download. I'm not sure if that really solves your problem though, because I think you may still need to handle the possibility that your artifacts are not there yet. But it might be ok for 99% of the cases, because in reality, the upload rarely if ever fails on the first try.
what do you think?

Hi Paul,
" testflinger-cli artifacts ${{JOB_ID}} || (sleep 30 && testflinger-cli artifactstestflinger-cli artifacts ${{JOB_ID}} ${{JOB_ID}})"

Not help this issue, because that's already current way we did as I wrote in description.

The issue here is testflinger-cli artifacts ${{JOB_ID}} will success, but get a truncated artifacts.tgz.

So, I guess the real fix should be:
testflinger-cli artifacts ${{JOB_ID}} to check the checksum of artifacts.tgz after finish downloading.

So this might be fixed for your purposes, but it doesn't really check the checksum yet. I'd still like to add that so I'm going to leave it open but there were some changes a while back on the testflinger-agent side, which should mean that you don't exit from testflinger poll, until that artifact is uploaded completely. Because if anything goes wrong during the artifact upload, it will fall out of that and put it into a results queue to retry. It's not until later when the results json gets uploaded that the job will be marked "complete", and at that point polling will stop listening, so by then, the artifact *must* be uploaded, which means you shouldn't get a truncated artifact unless some other network problem interferes.