summaryrefslogtreecommitdiff
path: root/packaging
AgeCommit message (Collapse)Author
2023-09-05[stable-2.14] tarfile: Handle deprecation warning for extract and extractall ↵Matt Martz
(#81545) (#81633) * Python 3.11.4 introduces a new parameter 'filter' in extract and extractall in tarfile. Handle deprecation warning message emitted in Python 3.12. * added probing mechanism in ansible-galaxy code to detect broken data filter implementation in tarfile. Fixes: #80832 (cherry picked from commit 1cc5efa) Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2023-08-16[stable-2.14] Remove obsolete text from release.py PR template (#81526) (#81528)Matt Clay
(cherry picked from commit 9afaf2216b0bf38cfbd9a0c93409959f78d01820)
2023-08-14[stable-2.14] Fix f-string whitespace in release script (#81477) (#81479)Matt Clay
(cherry picked from commit 85d3305889e71f70db27081279c89068fbad8b98)
2023-08-14[stable-2.14] Clean up release script (#81488) (#81490)Matt Clay
* Enable mypy for the entire packaging directory * Return CompletedProcess only when capturing output This allows stdout/stderr on CompletedProcess to be `str` instead of `str | None`. The unused args on CompletedProcess have been removed. Overload type hints have been added to reflect these changes. * Relax return type on ensure_venv This improves consistency with its usage, since `run` accepts `env` of `dict[str, t.Any]`. Also removed unnecssary `str()` usage when updating `env`. * Fix type hint on suppress_when * Fix callable annotation * Add type hint for command_parser PyCharm complains about using a protected member, and also that it can't find the type in the type stubs. However, mypy properly recognizes the type. * Avoid unnecessary TypeVar usage. (cherry picked from commit 47ab59753c2a91aef8823ccf4f4b7bc61d2efe03)
2023-08-02[stable-2.14] Docs removal and other backports (#81407)Matt Clay
* Remove straight.plugin dependency (#80084) (cherry picked from commit f587856beb4afa11040418ecf83b0bfd3d528ab6) * Update package-data sanity test (#80344) The test no longer relies on the Makefile. (cherry picked from commit 46362bbd2783d25ddc1edaaf4db7d00627ad7e88) * Remove obsolete release bits (#80347) Releases are now built using the `packaging/release.py` tool. This makes the `Makefile` and associated files in `packaging/release/` and `packaging/sdist/` obsolete. * Use --no-isolation for package-data sanity test (#80377) The dependencies are already in the sanity test venv. This avoids use of unpinned dependencies and a dependency on a network connection. (cherry picked from commit 7fcb9960e65591b42c1b46811dd529bae52ddf85) * Set the minimum setuptools to 45.2.0 (#80649) Also update the package-data sanity test to use the minimum setuptools version. (cherry picked from commit 4d25e3d54f7de316c4f1d1575d2cf1ffa46b632c) * Use package_data instead of include_package_data (#80652) This resolves warnings generated by setuptools such as the following: _Warning: Package 'ansible.galaxy.data' is absent from the `packages` configuration. (cherry picked from commit 5ac292e12d5e1515beb34028346d76bb68398fc8) * Fix os.walk issues in package-data sanity test (#80703) * Remove `docs` and `examples` directories (#81011) * Remove docs dir * Updates to reflect docs removal * Fix integration test * Remove examples dir * Updates to reflect examples removal * Remove build_library and build-ansible.py * Remove refs to build_library and build-ansible.py * Remove obsolete template * Remove obsolete template reference * Remove the now obsolete rstcheck sanity test (cherry picked from commit 72e038e8234051b54552d794a22ebef9681ae3ae) * Omit pre-built man pages from sdist (#81395) Since man pages aren't accessible to users after a `pip install`, there's no need to include them in the sdist. This change makes it trivial to build man pages from source, which makes them much easier to iterate on. It also simplifies creation and testing of the sdist, since it no longer requires building man pages. The new `packaging/cli-doc/build.py` script can generate both man pages and RST documentation. This supports inclusion on the docs site without a dependency on `ansible-core` internals. Having a single implementation for both simplifies keeping the two formats in sync. (cherry picked from commit 691c8e86034f1fe099e4ef54880e633b34f0bc7a)
2023-07-31Fix backport of #81379 (#81386)Matt Clay
Two copies of the file existed in this branch, but only one was updated previously.
2023-07-31[stable-2.14] Include subcommands in generated man pages (#81378) (#81383)Matt Clay
Sub commands of `ansible-galaxy role` and `ansible-galaxy collection` are now documented. (cherry picked from commit 081c60b9d3d9d37e915f11073ac4d047e7888f33)
2023-07-31[stable-2.14] Fix misrendered sections in manpage generation (#81380)Matt Clay
This change fixes bugs in the manpage generator that existed since it was first added. It exposes CLI `ARGUMENTS` value to manpage templates. Before this change, the code contained a typo, causing the `for`-loop iterate over individual characters of the `'ARGUMENTS'` string rather than iterating over a tuple. A missing comma was at fault. The updated code gets rid of the `for`-loop and conditionals since it seems to have been a premature complexity increase and no other things than `'ARGUMENTS'` were ever added into the broken iterable. The functional change is that `arguments` is now always present in the Jinja2 context, unlike being missing sometimes because of the previous design (not that it was ever present, because of the bug! sigh...) The Jinja2 templates perform an `{% if arguments %}` check, letting the template engine silently ignore the missing variable. The clause was always falsy, meaning that the arguments section was not included in the manpages for at least the last 6 years. With this fix, it will be. This patch also deduplicates calling `opt_doc_list` @ generate_man. It was called late in the execution, more times than necessary. This patch makes sure it happens once by putting it at the top of the scope. It fixes rendering library and inventory in manpages. The corresponding Jinja2 templates have blocks wrapped with conditionals like `{% if inventory %}` and `{% if library %}` but said variables were never injected into the context, nor were they even deduced on the Python side of the generator. This means that the conditional clauses were always falsy, never showing the portions of the manpages. The Python script has hints for how the `inventory` variable was to be calculated, which is confirmed through the Git paleontology efforts. The block of code that references to the `inventory` bit was incorrectly checking a variable with a list of nested objects for the presence of a string which was never going to work. This patch fixes this check by verifying the CLI flag against the correct variable containing a list of options and exposes it to the Jinja2 templates. It also exposes the `library` variable in a similar way. The block displaying other binaries in Sphinx CLI docs has been synchronized with the manpage template. Previously, the current binary was displayed also. This patch gets rid of the unwanted trailing comma there too. Finally, the CLI executables list in the manpage template now reuses the same variable as the RST template that doesn't need any post-processing in Jinja2. Before, it was already used in the RST template so this patch aligns both templates to use the same logic as they got out-of-sync over time. PR #80450. (cherry picked from commit a84b3a4e7277084466e43236fa78fc99592c641a) Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
2023-07-28[stable-2.14] Fix command doc lookup in man page generation (#81365) (#81368)Matt Clay
(cherry picked from commit f9d674fcfd178bc2f9fb6f466b9e891290d2e53f)
2023-07-28[stable-2.14] Exclude internal options from man pages and docs (#81360) (#81362)Matt Clay
(cherry picked from commit fead6546715beb2ef61c6a42cc91375d9d7914ab)
2023-06-09[stable-2.14] Remove hacking dir dependency from build backend (#81026)Matt Clay
* Copy man generation files into build backend * Use copied files in build backend
2023-06-09[stable-2.14] Remove docs dir dependency from man page build (#81003) (#81023)Matt Clay
* [stable-2.14] Remove docs dir dependency from man page build (#81003) (cherry picked from commit b3f1290bcdc28a4b5d6d859ff4346ea0784c3ca1) Co-authored-by: Matt Clay <matt@mystile.com> * Update path in Makefile
2023-06-09Remove straight.plugin dependency (#80084) (#81019)Matt Davis
(cherry picked from commit f587856)
2023-05-09[stable-2.14] pep517 backend - Copy symlinks when copying source (#80690) ↵Matt Clay
(#80701) (cherry picked from commit 54576cd794530768a67bddba2f7450b614611e92)
2023-05-01[stable-2.14] Release tool improvements (#80641) (#80680)Matt Clay
* Provide reproducible sdist builds. * Use reproducible wheel builds. * Add PyPI artifact checks. (cherry picked from commit d37678c5ff221ada71afc0cad3ff7b70e0a0ec2f)
2023-04-12[stable-2.14] pep517 backend - Use correct import_module import (#80480) ↵Matt Clay
(#80502) (cherry picked from commit e87802cf2c75c5f2578e58ed2edb3e033623cc78)
2023-03-27[stable-2.14] New upstream release tool (#80179) (#80321)Matt Clay
(cherry picked from commit a6bfa82bd061d1d66e7d67064f2e78774a103b54)
2023-03-21[stable-2.14] 📦 Switch sdist build-system to pure setuptools (#80255) ↵Matt Clay
(#80261) * [stable-2.14] 📦 Switch sdist build-system to pure setuptools (#80255) This patch modifies the in-tree build backend to build sdists that swap out pointers to it in the `pyproject.toml`'s `[build-system]` section. The effect of this is that the first build from source (for example, from a Git checkout) uses our PEP 517 in-tree build backend. But the produced tarball has `build-backend` set to `setuptools.build_meta` which is the native build backend of `setuptools`. So any following builds from that sdist will skip using the in-tree build backend, calling the setuptools' one. The good news is that if the first build generated the manpages, they will be included and won't go anywhere even though, a different build system is in place. Combined with #80253, this will make sure not to modify the current source checkout on that first build. Co-authored-by: Matt Clay <matt@mystile.com> (cherry picked from commit 7097df3eed979446830fc579613ffb9b7e7c19bf) Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com> * Make PEP 517 mutation tests use pinned old setuptools (#80262) * Pin setuptools to lowest supported @ PEP 517 test This allows catching the behavior of builds under old setuptools. * Stop invoking `setup.py install` in tests This is not the part we care about since it involves dealing with the external runtime dependencies rather than building our source distribution. (cherry picked from commit eebfd71a6da396b011e664604c9991543205d780) --------- Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
2023-03-20[stable-2.14] Make isolated source changes @ PEP 517 backend (#80253)Sviatoslav Sydorenko
(cherry picked from commit 888abf5d6e13880ba82d02c5f8a57ca1f90a992f) Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
2023-03-15[backport-2.14] 🎨 Convert RST to manpage in-memory @ PEP 517 (#80237)Sviatoslav Sydorenko
* Add _convert_rst_in_template_to_manpage arg types * 📦 Make manpage build dependencies conditional Previously, said dependencies were declared as unconditionally required even when manpages not needed to be built. This patch Makes it so they are only required when needed. * Correct _generate_rst_in_templates returned type It was marked as Path before this patch but in fact, it's iterable of paths. * 🎨 Convert RST to manpage in-memory @ PEP 517 Previously, the automation was writing a temporary templated RST on disk and calling a helper CLI script on that. But with this change, it happens with less unnecessary I/O. Co-Authored-By: Matt Davis <6775756+nitzmahone@users.noreply.github.com> * 📦Expose sdist manpage build deps unconditionally Due to a bug in pypa/build, the `get_requires_for_build_sdist()` hook is always invoked with `config_settings=None`. This means that we cannot conditionally extend build requirements in said hook. As a workaround, this patch makes hook pretend that `--built-manpages` is always passed. Ref: https://github.com/pypa/build/issues/559. --------- Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com> (cherry picked from commit 67bafafbc0d61751d2ec10b0aeccb6b01482a074) This is a backport of #80098.
2023-03-07[backport-2.14] 📦 Integrate manpage builds into PEP 517 build backend ↵Sviatoslav Sydorenko
(#80129) This patch creates a thin wrapper around the `setuptools`' PEP 517 build backend in-tree. It features an ability to request generating the manpage files in the process of building a source distribution. This toggle is implemented using the `config_settings` mechanism of PEP 517. One must explicitly pass it a CLI option to the build front-end to trigger said behavior. The packagers are expected to use the following call: python -m build --config-setting=--build-manpages This option has no effect on building wheels. 🧪 The change includes integration tests This test runs building and re-building sdists and wheels with and without the `--build-manpages` config setting under the oldest-supported and new `setuptools` pinned. It is intended to preserve the interoperability of the packaging setup across Python runtimes. An extra smoke test also verifies that non PEP 517 interfaces remain functional. PR #79606 Co-authored-by: Matt Clay <matt@mystile.com> (cherry picked from commit 56036013cd09db0f1b20402a69be44345ddafb22)
2021-10-14Remove unofficial packaging.Matt Clay
Unofficial packaging should be maintained externally since it is not tested in CI or used in the official release process. Maintainers of unofficial packages are better equipped with the necessary expertise and testing resources to support alternative packaging.
2021-05-18Remove PyCrypto (#74699)Sam Doran
* Remove PyCrypto from setup.py and packaging script * Remove mention of pycrpto from installation docs * Remove PyCrypto from vault * Remove pycryto constraint and unit test requirement * Remove PyCrypto tests from unit tests * Add docs and fix warning message * Remove section about cryptography library in Ansible Vault docs
2021-04-06renamed: ansible-base.{dirs,install} -> ansible-core.{dirs,install} (#74159)Deric Crago
2020-11-13Rename to ansible-core (#72594)Rick Elrod
Change: - Initial set of changes for renaming to ansible-core - Includes changelog fragment changes from base -> core - Does NOT include docs changes - Modifies detection stuff in setup.py to support ansible<2.9 and ansible-base Test Plan: - ci_complete
2020-06-11Use antsibull-changelog instead of packaged changelog generator (#69313)Felix Fontein
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
2020-05-26fix debian control file copypaste fail (#69715)Rick Elrod
Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-26More fixes for debian ansible-base builds (#69714)Rick Elrod
Change: Fix files that got missed in 04ec72160 Test Plan: Local Ubuntu 20.04 VM Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-15Fixes for building ansible-base .debs, using py3 (#69534)Rick Elrod
Change: - Changes for ansible -> ansible base - Bump to py3 for 2.10 PPA release Test Plan: - Local VM; Jenkins after merge. Tickets: - Refs #57342 Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-15Remove RPM packaging infrastructure (#69554)Rick Elrod
Change: - Nuke `make rpm` and friends from Makefile - Nuke packaging/rpm We are no longer going to be pushing RPMs to releases.ansible.com post-2.10, so this is no longer necessary for us, and users should prefer RPMs from their distro instead. Test Plan: Grepped the Makefile for all of: /rpm/i, /mock/i, /fedora/i Tickets: Refs #69539 Signed-off-by: Rick Elrod <rick@elrod.me>
2020-03-23remove azure extras and extras_require support (#67822)Matt Davis
* remove azure extras and extras_require support * Since Azure will be collectionized, the requirements will float more frequently than Ansible releases; the Azure collection needs to host the requirements now. * Removed the dynamic extras support as well, since Azure was the only thing using it. If we need it again, it's easy to pull back from history. * Mark azure-requirements as orhpaned. This keeps the docs around so that existing links from old test runs remain valid. Co-authored-by: Matt Clay <matt@mystile.com>
2020-01-16Vmss scale in policy (#66512)haiyuan_zhang
* azure_rm_virtualmachinescaleset: add scale_in_policy and terminate_event_notification * azure_rm_virtualmachinescalesetinstance: add vmss instance protection policy support
2020-01-15Update and cleanup Arch packaging (#64747)Martin
2019-11-05debian: install lib in dist-packages (#64412)Gonéri Le Bouder
On Ubuntu Precise and Trusty, the build was failing because the Ansible libs were installed in /usr/lib/python2.7/site-packages. According to the Debian Policy (2.5), they should actually be install in /usr/lib/python2.7/dist-packages This is also CDBS default behaviour since 0.4.131. See: https://www.debian.org/doc/packaging-manuals/python-policy/ch-python.html Closes: #64160
2019-11-01Exclude the ansible-test script from the main rpm package (#64277)Toshio Kuratomi
We want the script to live in the ansible-test rpm subpackage alongside of its libraries Fixes #64275
2019-10-04CentOS8/RHEL8 base don't have all the deps we were specifyingToshio Kuratomi
We used a few packages for tests which don't exist in RHEL8 base. Don't dep on those so those tests will simply skip
2019-09-30Fixes to the rpm dependenciesToshio Kuratomi
* Remove duplicate bundled provides line in the rpm spec file * Remove jmespath and passlib on RHEL7 as these are optional dependencies which aren't shipped with RHEL7
2019-09-18Disable setting of -s in shebangToshio Kuratomi
RPM builds on Fedora and RHEL create a python shebang line with -s This is not good for ansible since ansible has a lot of optional features which need extra dependencies installed. If the user installs those extra dependencies to their home directory or to /usr/local then the -s will keep them from being used.
2019-09-05Fix rpm dependencies for ansible-testToshio Kuratomi
Needs to require ansible = version rather than ansible-version
2019-09-03debian: prepare the ansible-test packageGonéri Le Bouder
- Generate an extra package for `ansible-test`. - Manually install the files because CDBS get confused by our two Python packages - Refresh the package descriptions to match what is done with the RPM
2019-09-03Update the rpm spec file to include ansible-testToshio Kuratomi
* Include ansible-test in a subpackage * Sync with the ansible engine spec file * Addresses https://github.com/ansible/ansible/issues/60380 to the extent that we're likely to fix that.
2019-08-29Revert "New Module: NetApp Account (#59939)"Toshio Kuratomi
This reverts commit 6a8dcb8fcbc0e79484067dc8c5b5dda173e73ce2. Reverting this as it breaks CI for azure (more permissions would be needed for our account)
2019-08-29New Module: NetApp Account (#59939)Chris Archibald
* netapp account module for azure * netapp account module for azure * fixes * fixes * add requirements * add requirements * fix issues * add units * add units * updates * updates * fixes * fixes * fies * fixes
2019-08-27Support Azure IoT hub and related module (#55121)Yuwei Zhou
* Support iothub creation * raise errordetailexception rather than clouderror * add facts * change requirement * compare endpoint * add documentation * add documentation * add iot device facts * modify line ending * add auth method * add iot module * add consumer group * add the test * enhencement of doc * add list consumer groups * fix lint * fix lint * fix doc * fix doc * Update auzre_rm_iothub related document * changed paramter's type * update type * rename facts -> info * fixed sanity * missed during merge
2019-08-27Support automation account managment (#56429)Yuwei Zhou
* add automation account * add test * update sdk version * fix doc * Update azure_rm_automationaccount.py * Update azure_rm_automationaccount_facts.py * Update main.yml * add type define (#64) * add new (#67) * change 'example' to 'sample' * changed 'path' to 'ID'
2019-08-20ansible-doc added a new required element and broke changelog generationToshio Kuratomi
ansible-doc should probably be using subcommands for its exclusive_options...
2019-08-15Fix capitalization of SUSE (#60625)Andreas Jaeger
The openSUSE project was always capitalized openSUSE, and SUSE does not use the lowercase "u" since over 15 years. Let's update the docs for this.
2019-07-18add xmltodict to azure requirements file (#55748)Yunge Zhu
* add xmltodict * fix lint
2019-07-17remove unused and obsolete bsd port makefile (#58633)Brian Coca
freebsd itself has maintained and up to date version of port
2019-07-05Fix unicode error in changelog generationToshio Kuratomi