Infrastructure error: 1 retries out of 2 failed for download-retry
lava-dispatcher, installed at version: 2023.01
start: 0 validate
Start time: 2023-11-14 19:40:21.288522+00:00 (UTC)
Validating that http://download.automotivelinux.org/AGL/upload/ci/29407/1/qemuarm/zImage exists
Validating that http://download.automotivelinux.org/AGL/upload/ci/29407/1/qemuarm/agl-demo-platform-qemuarm.ext4.xz exists
cmd: ['docker', 'pull', 'dl9pf/qemu']
Calling: 'nice' 'docker' 'pull' 'dl9pf/qemu'
>> Using default tag: latest
>> latest: Pulling from dl9pf/qemu
>> Digest: sha256:8255fbbb44d1eb1eefd088c92d5e74b21402733509cc1fdab305ee7baf1b3eb4
>> Status: Image is up to date for dl9pf/qemu:latest
>> docker.io/dl9pf/qemu:latest
Returned 0 in 2 seconds
cmd: ['docker', 'run', '--rm', '--init', 'dl9pf/qemu', 'qemu-system-arm', '--version']
Calling: 'nice' 'docker' 'run' '--rm' '--init' 'dl9pf/qemu' 'qemu-system-arm' '--version'
>> QEMU emulator version 8.0.0 (Debian 1:8.0+dfsg-4)
>> Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers
Returned 0 in 4 seconds
validate duration: 12.99
case: validate
case_id: 3941329
definition: lava
result: pass
start: 1 deployimages (timeout 01:30:00) [common]
start: 1.1 lava-overlay (timeout 01:30:00) [common]
[common] Preparing overlay tarball in /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_
makedir: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin
makedir: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/tests
makedir: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/results
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-add-keys
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-add-sources
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-background-process-start
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-background-process-stop
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-common-functions
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-echo-ipv4
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-install-packages
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-installed-packages
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-os-build
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-probe-channel
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-probe-ip
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-target-ip
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-target-mac
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-target-storage
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-test-case
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-test-event
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-test-feedback
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-test-raise
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-test-reference
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-test-runner
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-test-set
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-test-shell
Updating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-install-packages (oe)
Updating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/bin/lava-installed-packages (oe)
Creating /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/environment
LAVA metadata
- LAVA_JOB_ID=33482
- LAVA_DISPATCHER_IP=172.31.0.2
start: 1.1.1 lava-vland-overlay (timeout 01:30:00) [common]
skipped lava-vland-overlay
end: 1.1.1 lava-vland-overlay (duration 00:00:00) [common]
start: 1.1.2 lava-multinode-overlay (timeout 01:30:00) [common]
skipped lava-multinode-overlay
end: 1.1.2 lava-multinode-overlay (duration 00:00:00) [common]
start: 1.1.3 test-definition (timeout 01:30:00) [common]
Loading test definitions
start: 1.1.3.1 inline-repo-action (timeout 01:30:00) [common]
Using /lava-33482 at stage 0
uuid=33482_1.1.3.1 testdef=None
end: 1.1.3.1 inline-repo-action (duration 00:00:00) [common]
start: 1.1.3.2 test-overlay (timeout 01:30:00) [common]
end: 1.1.3.2 test-overlay (duration 00:00:00) [common]
case: test-overlay
case_id: 3941350
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.2
namespace: common
result: pass
start: 1.1.3.3 test-install-overlay (timeout 01:30:00) [common]
end: 1.1.3.3 test-install-overlay (duration 00:00:00) [common]
case: test-install-overlay
case_id: 3941351
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.3
namespace: common
result: pass
start: 1.1.3.4 test-runscript-overlay (timeout 01:30:00) [common]
runner path: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/0/tests/0_fixup-weston-watchdog test_uuid 33482_1.1.3.1
end: 1.1.3.4 test-runscript-overlay (duration 00:00:00) [common]
case: test-runscript-overlay
case_id: 3941352
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.4
namespace: common
result: pass
start: 1.1.3.5 git-repo-action (timeout 01:30:00) [common]
Using /lava-33482 at stage 1
Fetching tests from https://git.automotivelinux.org/src/qa-testdefinitions
Running '/usr/bin/git clone --depth=1 https://git.automotivelinux.org/src/qa-testdefinitions /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/1_ci-busybox'
Removing '.git' directory in /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/1_ci-busybox
Tests stored (tmp) in /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/1_ci-busybox/test-suites/short-smoke/busybox.yaml
uuid=33482_1.1.3.5 testdef={'testdef_pattern': {'pattern': '^(?P<test_case_id>[a-zA-Z0-9_-]+):\\s(?P<result>\\w+)', 'fixupdict': {'PASS': 'pass', 'FAIL': 'fail'}}}
end: 1.1.3.5 git-repo-action (duration 00:00:00) [common]
case: git-repo-action
case_id: 3941349
definition: lava
duration: 0.41
extra: ...
level: 1.1.3.5
namespace: common
result: pass
start: 1.1.3.6 test-overlay (timeout 01:30:00) [common]
end: 1.1.3.6 test-overlay (duration 00:00:00) [common]
case: test-overlay
case_id: 3941350
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.6
namespace: common
result: pass
start: 1.1.3.7 test-install-overlay (timeout 01:30:00) [common]
end: 1.1.3.7 test-install-overlay (duration 00:00:00) [common]
case: test-install-overlay
case_id: 3941351
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.7
namespace: common
result: pass
start: 1.1.3.8 test-runscript-overlay (timeout 01:30:00) [common]
runner path: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/1_ci-busybox test_uuid 33482_1.1.3.5
end: 1.1.3.8 test-runscript-overlay (duration 00:00:00) [common]
case: test-runscript-overlay
case_id: 3941352
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.8
namespace: common
result: pass
start: 1.1.3.9 git-repo-action (timeout 01:30:00) [common]
Using /lava-33482 at stage 1
Fetching tests from https://git.automotivelinux.org/src/qa-testdefinitions
Running '/usr/bin/git clone --depth=1 https://git.automotivelinux.org/src/qa-testdefinitions /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/2_ci-smoke-tests-basic'
Removing '.git' directory in /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/2_ci-smoke-tests-basic
Tests stored (tmp) in /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/2_ci-smoke-tests-basic/test-suites/short-smoke/smoke-tests-basic.yaml
uuid=33482_1.1.3.9 testdef=None
end: 1.1.3.9 git-repo-action (duration 00:00:00) [common]
case: git-repo-action
case_id: 3941349
definition: lava
duration: 0.43
extra: ...
level: 1.1.3.9
namespace: common
result: pass
start: 1.1.3.10 test-overlay (timeout 01:29:59) [common]
end: 1.1.3.10 test-overlay (duration 00:00:00) [common]
case: test-overlay
case_id: 3941350
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.10
namespace: common
result: pass
start: 1.1.3.11 test-install-overlay (timeout 01:29:59) [common]
end: 1.1.3.11 test-install-overlay (duration 00:00:00) [common]
case: test-install-overlay
case_id: 3941351
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.11
namespace: common
result: pass
start: 1.1.3.12 test-runscript-overlay (timeout 01:29:59) [common]
runner path: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/2_ci-smoke-tests-basic test_uuid 33482_1.1.3.9
end: 1.1.3.12 test-runscript-overlay (duration 00:00:00) [common]
case: test-runscript-overlay
case_id: 3941352
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.12
namespace: common
result: pass
start: 1.1.3.13 git-repo-action (timeout 01:29:59) [common]
Using /lava-33482 at stage 1
Fetching tests from https://git.automotivelinux.org/src/qa-testdefinitions
Running '/usr/bin/git clone --depth=1 https://git.automotivelinux.org/src/qa-testdefinitions /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/3_ci-service-check'
Removing '.git' directory in /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/3_ci-service-check
Tests stored (tmp) in /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/3_ci-service-check/test-suites/short-smoke/service-check.yaml
uuid=33482_1.1.3.13 testdef=None
end: 1.1.3.13 git-repo-action (duration 00:00:00) [common]
case: git-repo-action
case_id: 3941349
definition: lava
duration: 0.44
extra: ...
level: 1.1.3.13
namespace: common
result: pass
start: 1.1.3.14 test-overlay (timeout 01:29:59) [common]
end: 1.1.3.14 test-overlay (duration 00:00:00) [common]
case: test-overlay
case_id: 3941350
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.14
namespace: common
result: pass
start: 1.1.3.15 test-install-overlay (timeout 01:29:59) [common]
end: 1.1.3.15 test-install-overlay (duration 00:00:00) [common]
case: test-install-overlay
case_id: 3941351
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.15
namespace: common
result: pass
start: 1.1.3.16 test-runscript-overlay (timeout 01:29:59) [common]
runner path: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1/tests/3_ci-service-check test_uuid 33482_1.1.3.13
BUILD_TAGS=''
DEVICE_TAGS=''
end: 1.1.3.16 test-runscript-overlay (duration 00:00:00) [common]
case: test-runscript-overlay
case_id: 3941352
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.16
namespace: common
result: pass
start: 1.1.3.17 git-repo-action (timeout 01:29:59) [common]
Using /lava-33482 at stage 2
Fetching tests from https://git.automotivelinux.org/src/qa-testdefinitions
Running '/usr/bin/git clone --depth=1 https://git.automotivelinux.org/src/qa-testdefinitions /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/2/tests/4_screenshooter'
Tests stored (tmp) in /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/2/tests/4_screenshooter/test-suites/short-smoke/screenshooter.yaml
uuid=33482_1.1.3.17 testdef=None
end: 1.1.3.17 git-repo-action (duration 00:00:00) [common]
case: git-repo-action
case_id: 3941349
definition: lava
duration: 0.41
extra: ...
level: 1.1.3.17
namespace: common
result: pass
start: 1.1.3.18 test-overlay (timeout 01:29:58) [common]
end: 1.1.3.18 test-overlay (duration 00:00:00) [common]
case: test-overlay
case_id: 3941350
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.18
namespace: common
result: pass
start: 1.1.3.19 test-install-overlay (timeout 01:29:58) [common]
end: 1.1.3.19 test-install-overlay (duration 00:00:00) [common]
case: test-install-overlay
case_id: 3941351
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.19
namespace: common
result: pass
start: 1.1.3.20 test-runscript-overlay (timeout 01:29:58) [common]
runner path: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/2/tests/4_screenshooter test_uuid 33482_1.1.3.17
end: 1.1.3.20 test-runscript-overlay (duration 00:00:00) [common]
case: test-runscript-overlay
case_id: 3941352
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.20
namespace: common
result: pass
start: 1.1.3.21 git-repo-action (timeout 01:29:58) [common]
Using /lava-33482 at stage 3
Fetching tests from https://git.automotivelinux.org/src/qa-testdefinitions
Running '/usr/bin/git clone --depth=1 https://git.automotivelinux.org/src/qa-testdefinitions /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/3/tests/5_ci-dumpjournal'
Removing '.git' directory in /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/3/tests/5_ci-dumpjournal
Tests stored (tmp) in /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/3/tests/5_ci-dumpjournal/test-suites/short-smoke/dumpjournal.yaml
uuid=33482_1.1.3.21 testdef={'testdef_pattern': {'pattern': '^(?P<test_case_id>[a-zA-Z0-9_-]+):\\s(?P<result>\\w+)', 'fixupdict': {'PASS': 'pass', 'FAIL': 'fail'}}}
end: 1.1.3.21 git-repo-action (duration 00:00:00) [common]
case: git-repo-action
case_id: 3941349
definition: lava
duration: 0.41
extra: ...
level: 1.1.3.21
namespace: common
result: pass
start: 1.1.3.22 test-overlay (timeout 01:29:58) [common]
end: 1.1.3.22 test-overlay (duration 00:00:00) [common]
case: test-overlay
case_id: 3941350
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.22
namespace: common
result: pass
start: 1.1.3.23 test-install-overlay (timeout 01:29:58) [common]
end: 1.1.3.23 test-install-overlay (duration 00:00:00) [common]
case: test-install-overlay
case_id: 3941351
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.23
namespace: common
result: pass
start: 1.1.3.24 test-runscript-overlay (timeout 01:29:58) [common]
runner path: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/3/tests/5_ci-dumpjournal test_uuid 33482_1.1.3.21
end: 1.1.3.24 test-runscript-overlay (duration 00:00:00) [common]
case: test-runscript-overlay
case_id: 3941352
definition: lava
duration: 0.00
extra: ...
level: 1.1.3.24
namespace: common
result: pass
Creating lava-test-runner.conf files
Using lava-test-runner path: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/0 for stage 0
- 0_fixup-weston-watchdog
Using lava-test-runner path: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/1 for stage 1
- 1_ci-busybox
- 2_ci-smoke-tests-basic
- 3_ci-service-check
Using lava-test-runner path: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/2 for stage 2
- 4_screenshooter
Using lava-test-runner path: /var/lib/lava/dispatcher/tmp/33482/lava-overlay-svrcxjq_/lava-33482/3 for stage 3
- 5_ci-dumpjournal
end: 1.1.3 test-definition (duration 00:00:02) [common]
start: 1.1.4 compress-overlay (timeout 01:29:58) [common]
end: 1.1.4 compress-overlay (duration 00:00:00) [common]
start: 1.1.5 persistent-nfs-overlay (timeout 01:29:58) [common]
end: 1.1.5 persistent-nfs-overlay (duration 00:00:00) [common]
end: 1.1 lava-overlay (duration 00:00:02) [common]
start: 1.2 apply-overlay-guest (timeout 01:29:58) [common]
Overlay: /var/lib/lava/dispatcher/tmp/33482/compress-overlay-q1c75ylg/overlay-1.1.4.tar.gz
end: 1.2 apply-overlay-guest (duration 00:00:04) [common]
case: apply-overlay-guest
case_id: 3941353
definition: lava
duration: 3.91
extra: ...
level: 1.2
namespace: common
result: pass
start: 1.3 deploy-device-env (timeout 01:29:54) [common]
end: 1.3 deploy-device-env (duration 00:00:00) [common]
start: 1.4 download-retry (timeout 01:29:54) [common]
start: 1.4.1 http-download (timeout 01:29:54) [common]
downloading http://download.automotivelinux.org/AGL/upload/ci/29407/1/qemuarm/zImage
saving as /var/lib/lava/dispatcher/tmp/33482/deployimages-iv__hxc2/kernel/zImage
total size: 11694592 (11MB)
No compression specified
progress 0% (0MB)
progress 5% (0MB)
progress 10% (1MB)
progress 15% (1MB)
progress 20% (2MB)
progress 25% (2MB)
progress 30% (3MB)
progress 35% (3MB)
progress 40% (4MB)
progress 45% (5MB)
progress 50% (5MB)
progress 55% (6MB)
progress 60% (6MB)
progress 65% (7MB)
progress 70% (7MB)
progress 75% (8MB)
progress 80% (8MB)
progress 85% (9MB)
progress 90% (10MB)
progress 95% (10MB)
progress 100% (11MB)
11MB downloaded in 193.13s (0.06MB/s)
end: 1.4.1 http-download (duration 00:03:13) [common]
case: http-download
case_id: 3941465
definition: lava
duration: 193.13
extra: ...
level: 1.4.1
namespace: common
result: pass
end: 1.4 download-retry (duration 00:03:13) [common]
start: 1.5 download-retry (timeout 01:26:41) [common]
start: 1.5.1 http-download (timeout 01:26:41) [common]
downloading http://download.automotivelinux.org/AGL/upload/ci/29407/1/qemuarm/agl-demo-platform-qemuarm.ext4.xz
saving as /var/lib/lava/dispatcher/tmp/33482/deployimages-iv__hxc2/rootvd/agl-demo-platform-qemuarm.ext4
total size: 477658220 (455MB)
Using unxz to decompress xz
progress 0% (0MB)
progress 5% (22MB)
progress 10% (45MB)
progress 15% (68MB)
progress 20% (91MB)
progress 25% (113MB)
progress 30% (136MB)
progress 35% (159MB)
progress 40% (182MB)
progress 45% (205MB)
progress 50% (227MB)
progress 55% (250MB)
progress 60% (273MB)
http-download timed out after 5201 seconds
end: 1.5.1 http-download (duration 01:26:41) [common]
case: http-download
case_id: 3941465
definition: lava
duration: 5201.00
extra: ...
level: 1.5.1
namespace: common
result: fail
download-retry failed: 1 of 2 attempts. 'http-download timed out after 5201 seconds'
Cleaning up download directory: /var/lib/lava/dispatcher/tmp/33482/deployimages-iv__hxc2/rootvd
1 retries out of 2 failed for download-retry
end: 1.5 download-retry (duration 01:26:41) [common]
case: download-retry
case_id: 3941466
definition: lava
duration: 5201.00
extra: ...
level: 1.5
namespace: common
result: fail
1 retries out of 2 failed for download-retry
end: 1 deployimages (duration 01:30:00) [common]
case: deployimages
case_id: 3941467
definition: lava
duration: 5400.49
extra: ...
level: 1
namespace: common
result: fail
Cleaning after the job
Cleaning up download directory: /var/lib/lava/dispatcher/tmp/33482/deployimages-iv__hxc2/kernel
start: 7.1 power-off (timeout 00:00:10) [common]
end: 7.1 power-off (duration 00:00:00) [common]
start: 7.2 read-feedback (timeout 00:30:00) [common]
No connection for namespace common
end: 7.2 read-feedback (duration 00:00:00) [common]
Root tmp directory removed at /var/lib/lava/dispatcher/tmp/33482
InfrastructureError: The Infrastructure is not working correctly. Please report this error to LAVA admins.
case: job
case_id: 3941468
definition: lava
error_msg: 1 retries out of 2 failed for download-retry
error_type: Infrastructure
result: fail
Please read the triage guidelines for help on debugging failures in the test job, test definitions or in individual test cases.
This section helps test writers to check for actions with a duration which is much shorter than the requested timeout. Reducing these timeouts will allow failures to be identified more quickly.
The graph only shows actions that are longer than 1 second. The full list is anyway available in the table.