summaryrefslogtreecommitdiff
path: root/python
AgeCommit message (Collapse)Author
2021-06-01python: add tox supportJohn Snow
This is intended to be a manually run, non-CI script. Use tox to test the linters against all python versions from 3.6 to 3.10. This will only work if you actually have those versions installed locally, but Fedora makes this easy: > sudo dnf install python3.6 python3.7 python3.8 python3.9 python3.10 Unlike the pipenv tests (make venv-check), this pulls "whichever" versions of the python packages, so they are unpinned and may break as time goes on. In the case that breakages are found, setup.cfg should be amended accordingly to avoid the bad dependant versions, or the code should be amended to work around the issue. With confidence that the tests pass on 3.6 through 3.10 inclusive, add the appropriate classifiers to setup.cfg to indicate which versions we claim to support. Tox 3.18.0 or above is required to use the 'allowlist_externals' option. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-31-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add .gitignoreJohn Snow
Ignore *Python* build and package output (build, dist, qemu.egg-info); these files are not created as part of a QEMU build. They are created by running the commands 'python3 setup.py <sdist|bdist>' when preparing tarballs to upload to e.g. PyPI. Ignore miscellaneous cached python confetti (mypy, pylint, et al) Ignore .idea (pycharm) .vscode, and .venv (pipenv et al). Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-30-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add Makefile for some common tasksJohn Snow
Add "make venv" to create the pipenv-managed virtual environment that contains our explicitly pinned dependencies. Add "make check" to run the python linters [in the host execution environment]. Add "make venv-check" which combines the above two: create/update the venv, then run the linters in that explicitly managed environment. Add "make develop" which canonizes the runes needed to get both the linting pre-requisites (the "[devel]" part), and the editable live-install (the "-e" part) of these python libraries. make clean: delete miscellaneous python packaging output possibly created by pipenv, pip, or other python packaging utilities make distclean: delete the above, the .venv, and the editable "qemu" package forwarder (qemu.egg-info) if there is one. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-29-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add avocado-framework and testsJohn Snow
Try using avocado to manage our various tests; even though right now they're only invoking shell scripts and not really running any python-native code. Create tests/, and add shell scripts which call out to mypy, flake8, pylint and isort to enforce the standards in this directory. Add avocado-framework to the setup.cfg development dependencies, and add avocado.cfg to store some preferences for how we'd like the test output to look. Finally, add avocado-framework to the Pipfile environment and lock the new dependencies. We are using avocado >= 87.0 here to take advantage of some features that Cleber has helpfully added to make the test output here *very* friendly and easy to read for developers that might chance upon the output in Gitlab CI. [Note: ALL of the dependencies get updated to the most modern versions that exist at the time of this writing. No way around it that I have seen. Not ideal, but so it goes.] Provided you have the right development dependencies (mypy, flake8, isort, pylint, and now avocado-framework) You should be able to run "avocado --config avocado.cfg run tests/" from the python folder to run all of these linters with the correct arguments. (A forthcoming commit adds the much easier 'make check'.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-28-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add devel package requirements to setuptoolsJohn Snow
setuptools doesn't have a formal understanding of development requires, but it has an optional feataures section. Fine; add a "devel" feature and add the requirements to it. To avoid duplication, we can modify pipenv to install qemu[devel] instead. This enables us to run invocations like "pip install -e .[devel]" and test the package on bleeding-edge packages beyond those specified in Pipfile.lock. Importantly, this also allows us to install the qemu development packages in a non-networked mode: `pip3 install --no-index -e .[devel]` will now fail if the proper development dependencies are not already met. This can be useful for automated build scripts where fetching network packages may be undesirable. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-27-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/qemu: add qemu package itself to pipenvJohn Snow
This adds the python qemu packages themselves to the pipenv manifest. 'pipenv sync' will create a virtual environment sufficient to use the SDK. 'pipenv sync --dev' will create a virtual environment sufficient to use and test the SDK (with pylint, mypy, isort, flake8, etc.) The qemu packages are installed in 'editable' mode; all changes made to the python package inside the git tree will be reflected in the installed package without reinstallation. This includes changes made via git pull and so on. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-26-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/qemu: add isort to pipenvJohn Snow
isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret certain "from ..." clauses that are not related to imports. isort < 5.1.1 has a bug where it does not handle comments near import statements correctly. Require 5.1.2 or greater. isort can be run (in "check" mode) with 'isort -c qemu' from the python root. isort can also be used to fix/rewrite import order automatically by using 'isort qemu'. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-25-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: move .isort.cfg into setup.cfgJohn Snow
Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-24-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add mypy to pipenvJohn Snow
0.730 appears to be about the oldest version that works with the features we want, including nice human readable output (to make sure iotest 297 passes), and type-parameterized Popen generics. 0.770, however, supports adding 'strict' to the config file, so require at least 0.770. Now that we are checking a namespace package, we need to tell mypy to allow PEP420 namespaces, so modify the mypy config as part of the move. mypy can now be run from the python root by typing 'mypy -p qemu'. A note on mypy invocation: Running it as "mypy qemu/" changes the import path detection mechanisms in mypy slightly, and it will fail. See https://github.com/python/mypy/issues/8584 for a decent entry point with more breadcrumbs on the various behaviors that contribute to this subtle difference. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-23-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: move mypy.ini into setup.cfgJohn Snow
mypy supports reading its configuration values from a central project configuration file; do so. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-22-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: Add flake8 to pipenvJohn Snow
flake8 3.5.x does not support the --extend-ignore syntax used in the .flake8 file to gracefully extend default ignores, so 3.6.x is our minimum requirement. There is no known upper bound. flake8 can be run from the python/ directory with no arguments. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-21-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add excluded dirs to flake8 configJohn Snow
Instruct flake8 to avoid certain well-known directories created by python tooling that it ought not check. Note that at-present, nothing actually creates a ".venv" directory; but it is in such widespread usage as a de-facto location for a developer's virtual environment that it should be excluded anyway. A forthcoming commit canonizes this with a "make venv" command. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-20-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: move flake8 config to setup.cfgJohn Snow
Update the comment concerning the flake8 exception to match commit 42c0dd12, whose commit message stated: A note on the flake8 exception: flake8 will warn on *any* bare except, but pylint's is context-aware and will suppress the warning if you re-raise the exception. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-19-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add pylint to pipenvJohn Snow
We are specifying >= pylint 2.8.x for several reasons: 1. For setup.cfg support, added in pylint 2.5.x 2. To specify a version that has incompatibly dropped bad-whitespace checks (2.6.x) 3. 2.7.x fixes "unsubscriptable" warnings in Python 3.9 4. 2.8.x adds a new, incompatible 'consider-using-with' warning that must be disabled in some cases. These pragmas cause warnings themselves in 2.7.x. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-18-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: move pylintrc into setup.cfgJohn Snow
Delete the empty settings now that it's sharing a home with settings for other tools. pylint can now be run from this folder as "pylint qemu". Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-17-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add pylint import exceptionsJohn Snow
Pylint 2.5.x - 2.7.x have regressions that make import checking inconsistent, see: https://github.com/PyCQA/pylint/issues/3609 https://github.com/PyCQA/pylint/issues/3624 https://github.com/PyCQA/pylint/issues/3651 Pinning to 2.4.4 is worse, because it mandates versions of shared dependencies that are too old for features we want in isort and mypy. Oh well. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-16-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: Add pipenv supportJohn Snow
pipenv is a tool used for managing virtual environments with pinned, explicit dependencies. It is used for precisely recreating python virtual environments. pipenv uses two files to do this: (1) Pipfile, which is similar in purpose and scope to what setup.cfg lists. It specifies the requisite minimum to get a functional environment for using this package. (2) Pipfile.lock, which is similar in purpose to `pip freeze > requirements.txt`. It specifies a canonical virtual environment used for deployment or testing. This ensures that all users have repeatable results. The primary benefit of using this tool is to ensure *rock solid* repeatable CI results with a known set of packages. Although I endeavor to support as many versions as I can, the fluid nature of the Python toolchain often means tailoring code for fairly specific versions. Note that pipenv is *not* required to install or use this module; this is purely for the sake of repeatable testing by CI or developers. Here, a "blank" pipfile is added with no dependencies, but specifies Python 3.6 for the virtual environment. Pipfile will specify our version minimums, while Pipfile.lock specifies an exact loadout of packages that were known to operate correctly. This latter file provides the real value for easy setup of container images and CI environments. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-15-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add MANIFEST.inJohn Snow
When creating a source or binary distribution via 'python3 setup.py <sdist|bdist>', the VERSION and PACKAGE.rst files aren't bundled by default. Create a MANIFEST.in file that instructs the build tools to include these so that installation from these files won't fail. This is required by 'tox', as well as by the tooling needed to upload packages to PyPI. Exclude the 'README.rst' file -- that's intended as a guidebook to our source tree, not a file that needs to be distributed. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-14-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add directory structure README.rst filesJohn Snow
Add short readmes to python/, python/qemu/, python/qemu/machine, python/qemu/qmp, and python/qemu/utils that explain the directory hierarchy. These readmes are visible when browsing the source on e.g. gitlab/github and are designed to help new developers/users quickly make sense of the source tree. They are not designed for inclusion in a published manual. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-13-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add VERSION fileJohn Snow
Python infrastructure as it exists today is not capable reliably of single-sourcing a package version from a parent directory. The authors of pip are working to correct this, but as of today this is not possible. The problem is that when using pip to build and install a python package, it copies files over to a temporary directory and performs its build there. This loses access to any information in the parent directory, including git itself. Further, Python versions have a standard (PEP 440) that may or may not follow QEMU's versioning. In general, it does; but naturally QEMU does not follow PEP 440. To avoid any automatically-generated conflict, a manual version file is preferred. I am proposing: - Python tooling follows the QEMU version, indirectly, but with a major version of 0 to indicate that the API is not expected to be stable. This would mean version 0.5.2.0, 0.5.1.1, 0.5.3.0, etc. - In the event that a Python package needs to be updated independently of the QEMU version, a pre-release alpha version should be preferred, but *only* after inclusion to the qemu development or stable branches. e.g. 0.5.2.0a1, 0.5.2.0a2, and so on should be preferred prior to 5.2.0's release. - The Python core tooling makes absolutely no version compatibility checks or constraints. It *may* work with releases of QEMU from the past or future, but it is not required to. i.e., "qemu.machine" will, for now, remain in lock-step with QEMU. - We reserve the right to split the qemu package into independently versioned subpackages at a later date. This might allow for us to begin versioning QMP independently from QEMU at a later date, if we so choose. Implement this versioning scheme by adding a VERSION file and setting it to 0.6.0.0a1. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-12-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add qemu package installerJohn Snow
Add setup.cfg and setup.py, necessary for installing a package via pip. Add a ReST document (PACKAGE.rst) explaining the basics of what this package is for and who to contact for more information. This document will be used as the landing page for the package on PyPI. List the subpackages we intend to package by name instead of using find_namespace because find_namespace will naively also packages tests, things it finds in the dist/ folder, etc. I could not figure out how to modify this behavior; adding allow/deny lists to setuptools kept changing the packaged hierarchy. This works, roll with it. I am not yet using a pyproject.toml style package manifest, because "editable" installs are not defined (yet?) by PEP-517/518. I consider editable installs crucial for development, though they have (apparently) always been somewhat poorly defined. Pip now (19.2 and later) now supports editable installs for projects using pyproject.toml manifests, but might require the use of the --no-use-pep517 flag, which somewhat defeats the point. Full support for setup.py-less editable installs was not introduced until pip 21.1.1: https://github.com/pypa/pip/pull/9547/commits/7a95720e796a5e56481c1cc20b6ce6249c50f357 For now, while the dust settles, stick with the de-facto setup.py/setup.cfg combination supported by setuptools. It will be worth re-evaluating this point again in the future when our supported build platforms all ship a fairly modern pip. Additional reading on this matter: https://github.com/pypa/packaging-problems/issues/256 https://github.com/pypa/pip/issues/6334 https://github.com/pypa/pip/issues/6375 https://github.com/pypa/pip/issues/6434 https://github.com/pypa/pip/issues/6438 Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-11-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: create qemu packagesJohn Snow
move python/qemu/*.py to python/qemu/[machine, qmp, utils]/*.py and update import directives across the tree. This is done to create a PEP420 namespace package, in which we may create subpackages. To do this, the namespace directory ("qemu") should not have any modules in it. Those files will go into new 'machine', 'qmp' and 'utils' subpackages instead. Implement machine/__init__.py making the top-level classes and functions from its various modules available directly inside the package. Change qmp.py to qmp/__init__.py similarly, such that all of the useful QMP library classes are available directly from "qemu.qmp" instead of "qemu.qmp.qmp". Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-10-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: Trim line length to below 80 charsJohn Snow
One more little delinting fix that snuck in. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: disable warning for Popen in _launch()John Snow
We handle this resource rather meticulously in shutdown/kill/wait/__exit__ et al, through the laborious mechanisms in _do_shutdown(). Quiet this pylint warning here. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-7-jsnow@redhat.com Message-id: 20210517184808.3562549-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: Disable pylint warning for open() in _pre_launchJohn Snow
Shift the open() call later so that the pylint pragma applies *only* to that one open() call. Add a note that suggests why this is safe: the resource is unconditionally cleaned up in _post_shutdown(). _post_shutdown is called after failed launches (see launch()), and unconditionally after every call to shutdown(), and therefore also on __exit__. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-6-jsnow@redhat.com Message-id: 20210517184808.3562549-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/console_socket: Add a pylint ignoreJohn Snow
We manage cleaning up this resource ourselves. Pylint should shush. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-5-jsnow@redhat.com Message-id: 20210517184808.3562549-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: use subprocess.run instead of subprocess.PopenJohn Snow
use run() instead of Popen() -- to assert to pylint that we are not forgetting to close a long-running program. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-4-jsnow@redhat.com Message-id: 20210517184808.3562549-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: use subprocess.DEVNULL instead of open(os.path.devnull)John Snow
One less file resource to manage, and it helps quiet some pylint >= 2.8.0 warnings about not using a with-context manager for the open call. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-3-jsnow@redhat.com Message-id: 20210517184808.3562549-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/console_socket: avoid one-letter variableJohn Snow
Fixes pylint warnings. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20210527211715.394144-2-jsnow@redhat.com Message-id: 20210517184808.3562549-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Python: add utility function for retrieving port redirectionCleber Rosa
Slightly different versions for the same utility code are currently present on different locations. This unifies them all, giving preference to the version from virtiofs_submounts.py, because of the last tweaks added to it. While at it, this adds a "qemu.utils" module to host the utility function and a test. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210412044644.55083-4-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> [Squashed in below fix. --js] Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210601154546.130870-2-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Python: expose QEMUMachine's temporary directoryCleber Rosa
Each instance of qemu.machine.QEMUMachine currently has a "test directory", which may not have any relation to a "test", and it's really a temporary directory. Users instantiating the QEMUMachine class will be able to set the location of the directory that will *contain* the QEMUMachine unique temporary directory, so that parameter name has been changed from test_dir to base_temp_dir. A property has been added to allow users to access it without using private attributes, and with that, the directory is created on first use of the property. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210211220146.2525771-3-crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-02-15Python: close the log file kept by QEMUMachine before reading itCleber Rosa
Closing a file that is open for writing, and then reading from it sounds like a better idea than the opposite, given that the content will be flushed. Reference: https://docs.python.org/3/library/io.html#io.IOBase.close Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210211220146.2525771-2-crosa@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-01-02python: add __repr__ to ConsoleSocket to aid debuggingAlex Bennée
While attempting to debug some console weirdness I thought it would be worth making it easier to see what it had inside. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20201210190417.31673-6-alex.bennee@linaro.org>
2020-12-10treewide: do not use short-form boolean optionsPaolo Bonzini
They are going to be deprecated, avoid warnings on stdout while the tests run. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-11-23tests: add prefixes to the bare mkdtemp callsAlex Bennée
The first step to debug a thing is to know what created the thing in the first place. Add some prefixes so random tmpdir's have something grep in the code. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20201117173635.29101-3-alex.bennee@linaro.org>
2020-10-20python/qemu/qmp.py: Fix settimeout operationJohn Snow
We enabled callers to interface directly with settimeout, but this reacts poorly with blocking/nonblocking operation; as they are using the same internal mechanism. 1. Whenever we change the blocking mechanism temporarily, always set it back to what it was afterwards. 2. Disallow callers from setting a timeout of "0", which means Non-blocking mode. This is going to create more weird problems than anybody wants, so just forbid it. I opt not to coerce '0' to 'None' to maintain the principal of least surprise in mirroring the semantics of Python's interface. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20201009175123.249009-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/qmp.py: re-raise OSError when encounteredJohn Snow
Nested if conditions don't change when the exception block fires; we need to explicitly re-raise the error if we didn't intend to capture and suppress it. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20201009175123.249009-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python: add mypy configJohn Snow
Formalize the options used for checking the python library. You can run mypy from the directory that mypy.ini is in by typing `mypy qemu/`. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20201009175123.249009-2-jsnow@redhat.com [Edit: Added newline; thanks Bin Meng --js] Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/qmp.py: Preserve error context on re-raiseJohn Snow
Use the "from ..." phrasing when re-raising errors to preserve their initial context, to help aid debugging when things go wrong. This also silences a pylint 2.6.0+ error. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-18-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/console_socket.py: avoid encoding to/from stringJohn Snow
We can work directly in bytes instead of translating back and forth to string, which removes the question of which encodings to use. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-17-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/console_socket.py: Add type hint annotationsJohn Snow
Finish the typing of console_socket.py with annotations and no code changes. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-16-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/console_socket.py: Clarify type of drain_threadJohn Snow
Mypy needs just a little help to guess the type here. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-15-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/console_socket.py: fix typing of settimeoutJohn Snow
The types and names of the parameters must match the socket.socket interface. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-14-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/console_socket.py: Correct type of recv()John Snow
The type and parameter names of recv() should match socket.socket(). OK, easy enough, but in the cases we don't pass straight through to the real socket implementation, we probably can't accept such flags. OK, for now, assert that we don't receive flags in such cases. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-13-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu: Add mypy type annotationsJohn Snow
These should all be purely annotations with no changes in behavior at all. You need to be in the python folder, but you should be able to confirm that these annotations are correct (or at least self-consistent) by running `mypy --strict qemu`. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-12-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu: make 'args' style arguments immutableJohn Snow
These arguments don't need to be mutable and aren't really used as such. Clarify their types as immutable and adjust code to match where necessary. In general, It's probably best not to accept a user-defined mutable object and store it as internal object state unless there's a strong justification for doing so. Instead, try to use generic types as input with empty tuples as the default, and coerce to list where necessary. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20201006235817.3280413-10-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: fix _popen accessJohn Snow
As always, Optional[T] causes problems with unchecked access. Add a helper that asserts the pipe is present before we attempt to talk with it. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-9-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: Add _qmp access shimJohn Snow
Like many other Optional[] types, it's not always a given that this object will be set. Wrap it in a type-shim that raises a meaningful error and will always return a concrete type. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: use qmp.commandJohn Snow
machine.py and qmp.py both do the same thing here; refactor machine.py to use qmp.py's functionality more directly. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20201006235817.3280413-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: Handle None events in events_waitJohn Snow
If the timeout is 0, we can get None back. Handle this explicitly. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-14 19:02:57 +0000