I wrote this patch: https://github.com/Linaro/squadplugins/pull/10
In my local testing it fixes the problem and a few other issues I wansn't aware before.
milosz
On Thu, 13 Jun 2019 at 12:58, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
OK, found the culprit. It's the VTS test name:
arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit VTS decided to use "/" at the end of the test name which SQUAD isn't very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions.
As a side note, I found that the plugin code doesn't always recognize ABI properly. I'll fix that in the plugin.
milosz
On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Thank Axel, debugging now. I don't see anything that stands out in any obvious way.
milosz
On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Milosz,
I privately shared with you the archive containing what you asked for.
On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this.
milosz
On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error :
Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2)
Do you have any idea ?
Regards, Axel
On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis < axel.lebourhis@linaro.org> wrote:
Great, thanks!
On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
> I forgot to release it. Will do it today. > > milosz > > On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > > > > Hi Milosz, > > > > I saw your fix has been merged, is it available now with pip ? > > > > Regards, > > Axel > > > > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski < > milosz.wasilewski@linaro.org> wrote: > >> > >> https://github.com/Linaro/squadplugins/pull/9 > >> Antonio, could you take a look? > >> > >> milosz > >> > >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> > > >> > Hi Milosz, > >> > > >> > Great thank you. > >> > > >> > Axel > >> > > >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski < > milosz.wasilewski@linaro.org> wrote: > >> >> > >> >> I think it's interactive test section that makes the plugin > unhappy. > >> >> I'll fix that and upload new version. > >> >> > >> >> milosz > >> >> > >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> >> > > >> >> > Hi Milosz, > >> >> > > >> >> > Yes sure, you will find it attached. This definition is the > one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. > >> >> > I hid the sensible information with "****". > >> >> > > >> >> > Regards, > >> >> > Axel > >> >> > > >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski < > milosz.wasilewski@linaro.org> wrote: > >> >> >> > >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> >> >> > > >> >> >> > Hi all, > >> >> >> > > >> >> >> > I get back to this topic as I have a new issue now. > >> >> >> > Squad worker return a "Plugin postprocessing error" when > the tradefed plugin is triggered. > >> >> >> > The logs : > >> >> >> > > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: > [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: > 1717 > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: > [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: > 'definitions' > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: > Traceback (most recent call last): > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File > "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in > __postprocess_testjob__ > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: > plugin.postprocess_testjob(test_job) > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File > "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in > postprocess_testjob > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for > test_definition in test_action['test']['definitions']: > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: > KeyError: 'definitions' > >> >> >> > > >> >> >> > Am I missing something ? > >> >> >> > >> >> >> Could you show your test job definition? CTS plugin will > first check > >> >> >> whether it's supposed to work on the test job by inspecting > the > >> >> >> definition. Apparently there is some mismatch between what > is expected > >> >> >> and what is actually there. You can hide/obfuscate all > URLs, they're > >> >> >> not that important. > >> >> >> > >> >> >> milosz > >> >> >> > >> >> >> > > >> >> >> > Best regards, > >> >> >> > Axel > >> >> >> > > >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> >> >> >> > >> >> >> >> Hi Milosz, > >> >> >> >> > >> >> >> >> Thank you for your email. It is now working fine. > >> >> >> >> I setup my own local HTTP server and updated my tradefed > test definition (based on yours) to match my setup. > >> >> >> >> The main thing that was missing in my test definition > was the "lava-test-reference". > >> >> >> >> > >> >> >> >> Thank you for you help. > >> >> >> >> > >> >> >> >> Regards, > >> >> >> >> Axel > >> >> >> >> > >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski < > milosz.wasilewski@linaro.org> wrote: > >> >> >> >>> > >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> >> >> >>> > > >> >> >> >>> > Hi all, > >> >> >> >>> > > >> >> >> >>> > I saw you have a repository with tradefed and ltp > plugins for SQUAD. > >> >> >> >>> > If I understand correctly, those plugins retrieve > CTS/VTS/LTP logs to > >> >> >> >>> > attach the error logs to failed test cases. > >> >> >> >>> > > >> >> >> >>> > My questions are : > >> >> >> >>> > How can I setup those plugins to work with my SQUAD > instance ? > >> >> >> >>> > >> >> >> >>> you need to install the package from pip > >> >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to > your squad > >> >> >> >>> instance. That should do the trick. Each project has to > have the > >> >> >> >>> plugin enabled manually. > >> >> >> >>> > >> >> >> >>> > Do I need to use a specific tradefed test definition > to interface it with > >> >> >> >>> > the plugin ? > >> >> >> >>> > >> >> >> >>> yes, tradefed plugin assumes that all logs are stored > outside of LAVA. > >> >> >> >>> Tradefed creates a zip file that is later uploaded to > 'artifactorial' > >> >> >> >>> (https://archive.validation.linaro.org/). You can use > some other > >> >> >> >>> service but it needs to support upload request in the > following form: > >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F > >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" > >> >> >> >>> ATTACHMENT is the zip file that tradefed created and > >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user > uploading the > >> >> >> >>> content. > >> >> >> >>> This step is done automatically when you use tradefed > [1] or > >> >> >> >>> noninteractive-tradefed [2] from test-definitions > >> >> >> >>> > >> >> >> >>> When running in squad post processing step plugin > downloads the zip > >> >> >> >>> file and extracts logs from it. Failed tests are > updated with logs. > >> >> >> >>> > >> >> >> >>> > Does the job submission works the same way ? using > the submitjob API ? > >> >> >> >>> > >> >> >> >>> Test job submission works the same way. Plugins work as > a post > >> >> >> >>> processing step after LAVA jobs are completed. > >> >> >> >>> > >> >> >> >>> [1] > https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... > >> >> >> >>> [2] > https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... > >> >> >> >>> > >> >> >> >>> milosz > >> >> >> >>> > >> >> >> >>> > > >> >> >> >>> > Thank you in advance, > >> >> >> >>> > Axel > >> >> >> >>> > _______________________________________________ > >> >> >> >>> > Squad-dev mailing list > >> >> >> >>> > Squad-dev@lists.linaro.org > >> >> >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev >