summaryrefslogtreecommitdiff
path: root/docs/devel/testing.rst
AgeCommit message (Collapse)Author
2021-11-08tests/avocado: Rename avocado_qemu.Test -> QemuSystemTestPhilippe Mathieu-Daudé
To run user-mode emulation tests, we introduced the avocado_qemu.QemuUserTest which inherits from avocado_qemu.QemuBaseTest. System-mode emulation tests are based on the avocado_qemu.Test class, which also inherits avocado_qemu.QemuBaseTest. To avoid confusion, rename it as avocado_qemu.QemuSystemTest. Suggested-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20211105143416.148332-7-f4bug@amsat.org>
2021-11-08tests/acceptance: rename tests acceptance to tests avocadoWillian Rampazzo
In the discussion about renaming the `tests/acceptance` [1], the conclusion was that the folders inside `tests` are related to the framework running the tests and not directly related to the type of the tests. This changes the folder to `tests/avocado` and adjusts the MAKEFILE, the CI related files and the documentation. [1] https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg06553.html Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20211105155354.154864-3-willianr@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-10-01docs: reorganize testing.rstPaolo Bonzini
Clean up the heading levels to use === --- ~~~ ^^^ '''. Reorganize the outline for the Avocado part, and always include headings for the class names. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-01docs: move gcov section at the end of testing.rstPaolo Bonzini
gcov testing applies to all tests, not just make check. Move it out of the make check section. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-09-27tests/Makefile: add AVOCADO_TESTS option to make check-acceptanceWillian Rampazzo
Add the possibility of running all the tests from a single file, or multiple files, running a single test within a file or multiple tests within multiple files using `make check-acceptance` and the AVOCADO_TESTS environment variable. Suggested-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20210923161141.232208-4-willianr@redhat.com>
2021-09-27docs/devel/testing: add instruction to run a single acceptance testWillian Rampazzo
Add instructions to the Acceptance tests section about running a single test file or a test within the test file. Signed-off-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20210923161141.232208-3-willianr@redhat.com>
2021-09-27tests/Makefile: allow control over tags during check-acceptanceWillian Rampazzo
Although it is possible to run a specific test using the avocado command-line, a user may want to use a specific tag while running the ``make check-acceptance`` during the development or debugging. This allows using the AVOCADO_TAGS environment variable where the user takes total control of which tests should run based on the tags defined. This also makes the check-acceptance command flexible to restrict tests based on tags while running on CI. e.g.: AVOCADO_TAGS="foo bar baz" make check-acceptance Signed-off-by: Willian Rampazzo <willianr@redhat.com> Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20210923161141.232208-2-willianr@redhat.com>
2021-09-01docs/devel/testing: add -p option to the debug section of QEMU iotestsEmanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210809090114.64834-17-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-01docs/devel/testing: add -valgrind option to the debug section of QEMU iotestsEmanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20210809090114.64834-15-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-01docs/devel/testing: add -gdb option to the debugging section of QEMU iotestsEmanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210809090114.64834-10-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-01docs/devel/testing: add debug section to the QEMU iotests chapterEmanuele Giuseppe Esposito
Introduce the "Debugging a test case" section, in preparation to the additional flags that will be added in the next patches. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210809090114.64834-5-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-08-02docs/devel: Format literals correctlyPeter Maydell
In rST markup, single backticks `like this` represent "interpreted text", which can be handled as a bunch of different things if tagged with a specific "role": https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#interpreted-text (the most common one for us is "reference to a URL, which gets hyperlinked"). The default "role" if none is specified is "title_reference", intended for references to book or article titles, and it renders into the HTML as <cite>...</cite> (usually comes out as italics). Fix various places in the devel section of the manual which were using single backticks when double backticks (for literal text) were intended. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20210726142338.31872-6-peter.maydell@linaro.org
2021-07-13tests/acceptance: Automatic set -cpu to the test vmWainer dos Santos Moschetta
This introduces a new feature to the functional tests: automatic setting of the '-cpu VALUE' option to the created vm if the test is tagged with 'cpu:VALUE'. The 'cpu' property is made available to the test object as well. For example, for a simple test as: def test(self): """ :avocado: tags=cpu:host """ self.assertEqual(self.cpu, "host") self.vm.launch() The resulting QEMU evocation will be like: qemu-system-x86_64 -display none -vga none \ -chardev socket,id=mon,path=/var/tmp/avo_qemu_sock_pdgzbgd_/qemu-1135557-monitor.sock \ -mon chardev=mon,mode=control -cpu host Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20210430133414.39905-2-wainersm@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-07-13Acceptance Tests: support choosing specific distro and versionCleber Rosa
The tests based on the LinuxTest class give the test writer a ready to use guest operating system, currently pinned to Fedora 31. With this change, it's now possible to choose different distros and versions, similar to how other tags and parameter can be set for the target arch, accelerator, etc. One of the reasons for this work, is that some development features depend on updates on the guest side. For instance the tests on virtiofs_submounts.py, require newer kernels, and may benefit from running, say on Fedora 34, without the need for a custom kernel. Please notice that the pre-caching of the Fedora 31 images done during the early stages of `make check-acceptance` (before the tests are actually executed) are not expanded here to cover every new image added. But, the tests will download other needed images (and cache them) during the first execution. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210414221457.1653745-4-crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-06-01Acceptance Tests: add basic documentation on LinuxTest base classCleber Rosa
Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20210412044644.55083-10-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-03-20docs/devel/testing.rst: Fix references to unit testsWainer dos Santos Moschetta
With the recent move of the unit tests to tests/unit directory some instructions under the "Unit tests" section became imprecise, which are fixed by this change. Fixes: da668aa15b99 ("tests: Move unit tests into a separate directory") Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210318174407.2299930-1-wainersm@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-24docs/devel: add forward reference to check-tcgAlex Bennée
For completeness reference the check-tcg tests in the container preamble text. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Cc: Thomas Huth <thuth@redhat.com> Message-Id: <20210222101455.12640-7-alex.bennee@linaro.org>
2021-02-24docs/devel: update the container based testsAlex Bennée
This section has grown a little stale so clean-up the language and examples for current usage: - refer to containers at the top - mention podman can also be used - add podman prerequisites section - move to using "docker-help" for online help - mention the registry and it's purpose - don't refer to out-of-date min-glib image Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20210222101455.12640-6-alex.bennee@linaro.org>
2021-02-24docs/devel: expand on use of containers to build testsAlex Bennée
Expand on the usage of containers for building tests and why we have some that are not used to build QEMU itself. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Cc: Thomas Huth <thuth@redhat.com> Message-Id: <20210222101455.12640-5-alex.bennee@linaro.org>
2021-02-15tests/docker: remove travis containerDaniel P. Berrangé
The travis container that we have no longer matches what travis currently uses. As all x86 jobs are being moved to GitLab CI too, there is no compelling reason to update the travis container. It is simpler to just remove it. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20210209135011.1224992-2-berrange@redhat.com> Message-Id: <20210211122750.22645-9-alex.bennee@linaro.org>
2021-02-08Acceptance Tests: remove unnecessary tag from documentation exampleCleber Rosa
The ":avocado: enable" is not necessary and was removed in 9531d26c, so let's remove from the docs. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20210203172357.1422425-4-crosa@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-01-28Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into stagingPeter Maydell
Block layer patches: - Fix crash on write to read-only devices - iotests: Rewrite 'check' in Python, get rid of 'groups' and allow non-numeric test case names # gpg: Signature made Wed 27 Jan 2021 19:56:00 GMT # gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6 # gpg: issuer "kwolf@redhat.com" # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * remotes/kevin/tags/for-upstream: iotests: rename and move 169 and 199 tests iotests: rewrite check into python iotests: add testrunner.py iotests: add testenv.py iotests: add findtests.py iotests: 146: drop extra whitespaces from .out file virtio-scsi-test: Test writing to scsi-cd device block: Separate blk_is_writable() and blk_supports_write_perm() Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-27iotests: add findtests.pyVladimir Sementsov-Ogievskiy
Add python script with new logic of searching for tests: Current ./check behavior: - tests are named [0-9][0-9][0-9] - tests must be registered in group file (even if test doesn't belong to any group, like 142) Behavior of findtests.py: - group file is dropped - tests are all files in tests/ subdirectory (except for .out files), so it's not needed more to "register the test", just create it with appropriate name in tests/ subdirectory. Old names like [0-9][0-9][0-9] (in root iotests directory) are supported too, but not recommended for new tests - groups are parsed from '# group: ' line inside test files - optional file group.local may be used to define some additional groups for downstreams - 'disabled' group is used to temporary disable tests. So instead of commenting tests in old 'group' file you now can add them to disabled group with help of 'group.local' file - selecting test ranges like 5-15 are not supported more (to support restarting failed ./check command from the middle of the process, new argument is added: --start-from) Benefits: - no rebase conflicts in group file on patch porting from branch to branch - no conflicts in upstream, when different series want to occupy same test number - meaningful names for test files For example, with digital number, when some person wants to add some test about block-stream, he most probably will just create a new test. But if there would be test-block-stream test already, he will at first look at it and may be just add a test-case into it. And anyway meaningful names are better. This commit doesn't update check behavior (which will be done in further commit), still, the documentation changed like new behavior is already here. Let's live with this small inconsistency for the following few commits, until final change. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210125185056.129513-3-vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-27docs/devel: Explain how acceptance tests can be skippedWainer dos Santos Moschetta
Documented under the "Acceptance tests using the Avocado Framework" section in testing.rst how environment variables are used to skip tests. Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20210115210022.417996-1-wainersm@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26docs/devel/testing.rst: Update outdated Avocado URLsPhilippe Mathieu-Daudé
Avocado documentation referred returns 404 error. Update the broken links. Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201010080741.2932406-1-philmd@redhat.com>
2020-10-12docs/devel: update instruction on how to add new unit testsPaolo Bonzini
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-12docs: Move QTest documentation to its own documentEduardo Habkost
The qtest and libqtest doc comments will be parsed to generate API documentation, so move QTest documentation to its own document where the API and format documentation and will be included. Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20201005205228.697463-2-ehabkost@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-10docs: repair broken referencesJohn Snow
In two different places, we are not making a cross-reference to some resource correctly. Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20201009161558.107041-2-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-09-17docs/: fix some comment spelling errorszhaolichang
I found that there are many spelling errors in the comments of qemu, so I used the spellcheck tool to check the spelling errors and finally found some spelling errors in the docs folder. Signed-off-by: zhaolichang <zhaolichang@huawei.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-Id: <20200917075029.313-4-zhaolichang@huawei.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2020-08-21meson: link emulators without Makefile.targetPaolo Bonzini
The binaries move to the root directory, e.g. qemu-system-i386 or qemu-arm. This requires changes to qtests, CI, etc. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-08-21meson: use coverage optionMarc-André Lureau
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-06-16docs: Added details on TSan to testing.rstRobert Foley
Adds TSan details to testing.rst. This includes background and reference details on TSan, and details on how to build and test with TSan both with and without docker. Signed-off-by: Robert Foley <robert.foley@linaro.org> Reviewed-by: Emilio G. Cota <cota@braap.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200609200738.445-13-robert.foley@linaro.org> Message-Id: <20200612190237.30436-16-alex.bennee@linaro.org>
2020-02-04docs/devel: Fix qtest paths and info about check-block in testing.rstThomas Huth
The qtests have recently been moved to a separate subdirectory, so the paths that are mentioned in the documentation have to be adjusted accordingly. And some of the iotests are now always run as part of "make check", so this information has to be adjusted here, too. Message-Id: <20200122134511.23806-1-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2019-12-18tests/vm: Allow to set qemu-img pathWainer dos Santos Moschetta
By default VM build test use qemu-img from system's PATH to create the image disk. Due the lack of qemu-img on the system or the desire to simply use a version built with QEMU, it would be nice to allow one to set its path. So this patch makes that possible by reading the path to qemu-img from QEMU_IMG if set, otherwise it fallback to default behavior. Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20191114134246.12073-2-wainersm@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-12-16Acceptance tests: use avocado tags for machine typeCleber Rosa
The same way the arch tag is being used as a fallback for the arch parameter, let's do the same for QEMU's machine and avoid some boiler plate code. This is now possible because, since Avocado 72.0, it's possible to use tags with names that match the machine types on QEMU. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20191104151323.9883-4-crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2019-09-26docs/devel: add "check-tcg" to testing.rstAlex Bennée
It was pointed out we haven't documented the check-tcg part of the build system. Attempt to rectify that now. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-07-05Merge remote-tracking branch ↵Peter Maydell
'remotes/vivier2/tags/trivial-branch-pull-request' into staging Fix typo, change virtio-rng default to urandom # gpg: Signature made Wed 03 Jul 2019 16:24:53 BST # gpg: using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C # gpg: issuer "laurent@vivier.eu" # gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full] # gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full] # gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full] # Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C * remotes/vivier2/tags/trivial-branch-pull-request: docs/devel/testing: Fix typo in dockerfile path VirtIO-RNG: Update default entropy source to `/dev/urandom` Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-07-04Makefile: Rename the 'vm-test' target as 'vm-help'Philippe Mathieu-Daudé
We already have 'make check-help', use the 'make vm-help' form to display helps about VM testing. Keep the old target to not bother old customs. Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Message-Id: <20190531064341.29730-1-philmd@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-03docs/devel/testing: Fix typo in dockerfile pathLiam Merwick
Signed-off-by: Liam Merwick <liam.merwick@oracle.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Li Qiang <liq3ea@gmail.com> Message-Id: <1561727317-30655-1-git-send-email-liam.merwick@oracle.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-05-02tests/acceptance: look for target architecture in test tags firstCleber Rosa
A test can, optionally, be tagged for one or many architectures. If a test has been tagged for a single architecture, there's a high chance that the test won't run on other architectures. This changes the default order of choosing a default target architecture to use based on the 'arch' tag value first. The precedence order is for choosing a QEMU binary to use for a test is now: * qemu_bin parameter * arch parameter * arch tag value (for example, x86_64 if ":avocado: tags=arch:x86_64 is used) This means that if one runs: $ avocado run -p qemu_bin=/usr/bin/qemu-system-x86_64 test.py No arch parameter or tag will influence the selection of the QEMU target binary. If one runs: $ avocado run -p arch=ppc64 test.py The target binary selection mechanism will attempt to find a binary such as "ppc64-softmmu/qemu-system-ppc64". And finally, if one runs a test that is tagged (in its docstring) with "arch:aarch64": $ avocado run aarch64.py The target binary selection mechanism will attempt to find a binary such as "aarch64-softmmu/qemu-system-aarch64". At this time, no provision is made to cancel the execution of tests if the arch parameter given (manually) does not match the test "arch" tag, but it may be a useful default behavior to be added in the future. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Cornelia Huck <cohuck@redhat.com> Message-Id: <20190312171824.5134-7-crosa@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-05-02tests/acceptance: introduce arch parameter and attributeCleber Rosa
It's useful to define the architecture that should be used in situations such as: * the intended target of the QEMU binary to be used on tests * the architecture of code to be run within the QEMU binary, such as a kernel image or a full blown guest OS image This commit introduces both a test parameter and a test instance attribute, that will contain such a value. Now, when the "arch" test parameter is given, it will influence the selection of the default QEMU binary, if one is not given explicitly by means of the "qemu_img" parameter. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Cornelia Huck <cohuck@redhat.com> Message-Id: <20190312171824.5134-5-crosa@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-05-02tests/acceptance: fix doc reference to avocado_qemu directoryCleber Rosa
The "this directory" reference is misleading and confusing, it's a leftover from when this text was proposed in a README file inside the "tests/acceptance/avocado_qemu" directory. When that text was moved to the top level docs directory, the reference was not updated. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Caio Carrara <ccarrara@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Cornelia Huck <cohuck@redhat.com> Message-Id: <20190312171824.5134-4-crosa@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-02-22tests.acceptance: adds multi vm capability for acceptance testsCaio Carrara
This change adds the possibility to write acceptance tests with multi virtual machine support. It's done keeping the virtual machines objects stored in a test attribute (dictionary). This dictionary shouldn't be accessed directly but through the new method added `get_vm`. This new method accept a list of args (that will be added as virtual machine arguments) and an optional name argument. The name is the key that identify a single virtual machine along the test machines available. If a name without a machine is informed a new machine will be instantiated. The current usage of vm in tests will not be broken by this change since it keeps a property called vm in the base test class. This property only calls the new method `get_vm` with default parameters (no args and 'default' as machine name). Signed-off-by: Caio Carrara <ccarrara@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20190212193855.13223-2-ccarrara@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2019-02-22Acceptance tests: drop usage of ":avocado: enable"Cleber Rosa
The Avocado test runner attemps to find its INSTRUMENTED (that is, Python based tests) in a manner that is as safe as possible to the user. Different from plain Python unittest, it won't load or execute test code on an operation such as: $ avocado list tests/acceptance/ Before version 68.0, the logic implemented to identify INSTRUMENTED tests would require either the ":avocado: enable" or ":avocado: recursive" statement as a flag for tests that would not inherit directly from "avocado.Test". This is not necessary anymore, and because of that the boiler plate statements can now be removed. Reference: https://avocado-framework.readthedocs.io/en/68.0/release_notes/68_0.html#users-test-writers Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Caio Carrara <ccarrara@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20190218173723.26120-1-crosa@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2019-02-11docs/devel/testing: Add -a option to usermod command on docker setupMurilo Opsfelder Araujo
The option -G of usermod command will remove user from other groups not listed, i.e.: $USER will belong only to group 'docker' after following the documentation as is. From usermod(8) manual page: If the user is currently a member of a group which is not listed, the user will be removed from the group. This behaviour can be changed via the -a option, which appends the user to the current supplementary group list. This patch improves the situation by adding the -a option to the usermod command, which will just append user to the supplementary group list. Cc: qemu-trivial@nongnu.org Signed-off-by: Murilo Opsfelder Araujo <muriloo@linux.ibm.com> Message-Id: <20190207184346.6840-1-muriloo@linux.ibm.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-10-30Acceptance tests: add make rule for running themCleber Rosa
The acceptance (aka functional, aka Avocado-based) tests are Python files located in "tests/acceptance" that need to be run with the Avocado libs and test runner. Let's provide a convenient way for QEMU developers to run them, by making use of the tests-venv with the required setup. Also, while the Avocado test runner will take care of creating a location to save test results to, it was understood that it's better if the results are kept within the build tree. Signed-off-by: Cleber Rosa <crosa@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Caio Carrara <ccarrara@redhat.com> Message-Id: <20181018153134.8493-3-crosa@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-10-26docs/devel/testing.rst: add missing newlines after code blockCleber Rosa
The line immediate following a ".. code::" block is considered to contains arguments to the "code directive". The lack of a new line gives me during at parse time: testing.rst:63: (ERROR/3) Error in "code" directive: maximum 1 argument(s) allowed, 3 supplied. .. code:: make check-unit V=1 testing.rst:120: (ERROR/3) Error in "code" directive: maximum 1 argument(s) allowed, 3 supplied. .. code:: make check-qtest V=1 Let's add the missing newlines, both for consistency and to avoid the parsing errors. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-Id: <20181004161852.11673-6-crosa@redhat.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2018-10-16tests: remove gcov-files- variablesPaolo Bonzini
Commit 31d2dda ("build-system: remove per-test GCOV reporting", 2018-06-20) removed users of the variables, since those uses can be replaced by a simple overall report produced by gcovr. However, the variables were never removed. Do it now. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> [thuth: Fixed up contextual conflicts with the patch from Eric] Signed-off-by: Thomas Huth <thuth@redhat.com>
2018-08-15tests/vm: Propagate V=1 down into the make inside the VMPeter Maydell
Invoking 'make vm-build-freebsd' and friends with V=1 should propagate that verbosity setting down into the build run inside the VM. Make sure we do that. This brings it into line with how the container tests handle V=1. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-Id: <20180803085230.30574-4-peter.maydell@linaro.org> Signed-off-by: Fam Zheng <famz@redhat.com>
2018-07-30docs: Describe using images in writing iotestsFam Zheng
Signed-off-by: Fam Zheng <famz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-02-06 20:10:31 +0000