Commit Graph

173 Commits

Author SHA1 Message Date
Zuul df5756f765 Merge "Add zuul-tenant-conf-check role/job" 2024-02-13 16:09:34 +00:00
James E. Blair 73bdf1f2df Add zuul-tenant-conf-check role/job
This performs static validation of Zuul tenant config files.

Change-Id: I5d439d6cfb963e55d07b2a0058de76f030fe47b3
2024-02-01 15:56:29 -08:00
Tristan Cacqueray 26db5b3b24 Introduce LogJuicer roles
This change adds new roles to run logjuicer in zuul jobs:
  https://github.com/logjuicer/logjuicer

Change-Id: I02824a18285a16c8f0be6bb96b5404aa0d601c16
2024-01-08 16:09:17 +00:00
Zuul 3b3495e255 Merge "Deprecate mirror-workspace-git-repos" 2023-09-26 23:24:36 +00:00
Lukas Kranz ce2bea51d4 Deprecate mirror-workspace-git-repos
This change is preparation for https://review.opendev.org/c/zuul/zuul-jobs/+/887917

In the beginning, there was only prepare-workspace[0] which rsynced repos.

Then we added mirror-workspace-git[1] to make it more efficient by using git operation, but it required some openstack-specific code in project-config to work.

Then we added prepare-workspace-git[2] which completed the git-based sync solution by locating everything requried in zuul-jobs.  It used mirror-workspace-git by reference and added this TODO:

  # TODO(tobiash): we might want to deprecate the role mirror-workspace-git-repos
  # and move it here.

This change completes that TODO by moving the mirror-workspace-git-repos code into prepare-workspace-git and places the repo in a sensible and maintainable state with two simple and good options:

 * prepare-workspace (rsync)
 * prepare-workspace-git (git)

In the unlikely event anyone is still using mirror-workspace-git-repos standalone (OpenStack/OpenDev is not, and that solution was haphazard as described above) they would be well served by a notification that there is a better alternative which is what most of the community actually uses now.

[0] cfffd4431b
[1] 348598e96a
[2] 7cee7156bc

Change-Id: Ib80e0447d49363182fd0d4c4d4e269841bc3aa95
2023-09-25 14:51:26 -07:00
Clark Boylan 378e039dba Remove the nox-py27 job
We added nox somewhat recently and set it up to mimic existing tox jobs.
This meant adding py27 jobs. Since then (in OpenDev at least) only a
single project has used the py27 job: Bindep. Bindep is dropping support
for python2.7 as the need for it has come to an end. Additionally, nox
doesn't work with python2.7 out of the box due to a virtualenv
dependency that ends up being too new for python2.7 venv creation.
Rather than hack around that let's drop python2.7 job support.

Change-Id: I52c07b01ad173304c19b13a10927fdadf9d84170
2023-08-22 14:28:36 -07:00
James E. Blair 78b2911f17 Mark nim jobs unmaintained
The nim test jobs do not work across the board and no one has
stepped up to fix them.

Change-Id: I1bdd9ca95cd4b45c25d2a2645810c19a90e3cc55
2023-08-11 09:12:13 -07:00
Lukas Kranz b192ccee73 Fix blockdiag issue
Blockdiag is incompatible with Pillow >= 10.0
Solution for now, pin Pillow < 10.0

Change-Id: I93b9634f7d2813cb1a17aeda2d8ef2ec188f86f3
2023-07-10 09:06:31 +02:00
Clark Boylan 18b32703ed Add ensure-quay-repo role
This adds a new role that can be used to ensure a quay repo exists
before publishing to it. This is particularly useful for creating public
repos in quay as simply pushing to a repo with quay will create a
private repo by default.

Change-Id: I979f1b9b64f901bb8d54b8991bb9142b18b6330f
2023-04-21 15:09:42 -07:00
Ian Wienand fec27296c8
remove-registry-tag: role to delete tags from registry
This is a role to abstract removal of tags from registries, which is
an operation that practically has to be done via the registry API.

This implements removing tags from the quay and docker API's.

For the common case of working with a repository like
"quay.io/org/project" there is minimal configuration.  However, if you
run a private repository, this is flexible with a few extra variables
to tell the role to use the quay API but your own URL.

By default it clears out old tags from the Zuul promote pipeline.
However if you set registry_tag_remove_tag it will only remove that
one tag.

This is inspired by the current work done in promote-docker-image
role.

Change-Id: I7f2d9d00024e34451e2d20b2c2f8171ecd151943
2023-04-04 09:53:18 +10:00
James E. Blair 466aa92635 Add container build jobs
These jobs use the container build roles.

Change-Id: I13d1987980bc3d0b1c717878a4bc47edc6dcfe1c
2023-03-23 09:47:49 -07:00
okozachenko 0c3b87f20e Add promote-container-image role
This role uses skopeo to perform image operations.

Also update the container roles docs to add missing documentation
for the already existing upload-container-image role.  Clarify
some ambiguity about the registry and repository attributes of
the container images data structure.

Change-Id: Ib66c85daf0edacf0dd797ab34b0d629f99c7111b
Co-Authored-By: James E. Blair <jim@acmegating.com>
2023-03-21 10:17:49 -07:00
Ade Lee 99711abf23 Add ubuntu to enable-fips role
The enable-fips role has been refactored to support both centos/rhel and
Ubuntu.

In addition, for the Ubuntu tasks, a small role is added to enable a
Ubuntu Advantage subscription.  This is required because Ubuntu requires
a subscription to enable FIPS.  This role takes a subscription key as a
parameter (ubuntu_ua_token.token).

In Openstack, this is provided by the openstack-fips job in
openstack/project-config, which will be the base job for OpenStack jobs.
This job will provide the ubuntu_ua_token.token.

Change-Id: I47a31f680172b47584510adb672b68498a85bd32
2023-02-09 19:02:00 +00:00
Clark Boylan 50fd134646 Add nox role and some simple jobs
This is an alternative to tox.

Change-Id: Ib4920acec09c2c980af909e8f9d1eabd1c6d253a
2022-12-19 18:41:12 -08:00
James E. Blair 77b1b24911 Add ensure-nox role
This is an alternative to tox.

Change-Id: I394774836e303584473f0e7d64d496baa393037a
2022-12-15 13:22:49 -08:00
Michael Kelly b0cc01ceac
helm: Add job for linting helm charts
This job runs 'helm lint' on the user specified charts.

Change-Id: Ie6f39bab366b683e773add181de516c3ac913866
2022-11-16 15:09:07 -08:00
Clark Boylan 866ae1c52b Pin py38 jobs to focal
The default base job nodeset is moving from focal to jammy. Jammy
doesn't have python3.8 to run these jobs. Address that by explicitly
forcing these jobs to run on focal.

Change-Id: I57433092ea2afbec4546659ea20f31161cc41a6e
2022-10-25 10:05:49 -07:00
Ian Wienand a791329de3 Pin sphinx to 5.2.3
The new 5.3.0 release of Sphinx has started giving circular reference
errors on some of the included files.  Pin this while we figure it
out.

Change-Id: I7674eb0e08207e1ec3b3941361d1fae75f124ddd
2022-10-18 23:21:15 +00:00
Zuul 21faa1a9d8 Merge "Add the post-reboot-tasks role" 2022-06-17 15:46:53 +00:00
Ade Lee 25caf7ef5e Add the post-reboot-tasks role
This role will do basic checks to confirm that the node is
sufficiently up to continue afer a reboot.

Change-Id: Iebf474c9351e4246d7ab2072b48a50e93dbf0b94
2022-06-06 04:56:14 -07:00
James E. Blair 0c980bbcb7 Make test-prepare-workspace-git role
This is a copy of prepare-workspace-git except that it imports
test-mirror-workspace-git-repos.  This is for base job testing.

Change-Id: I4ef3e4376c9e958761c165836c4fb546157e237a
2022-05-19 11:40:44 -07:00
Zuul 775729a4c0 Merge "Add per-build WinRM cert generation" 2022-04-26 21:52:35 +00:00
James E. Blair 59d7af0e67 Add per-build WinRM cert generation
This adds roles that, similar to add-build-sshkey, create a per-build
WinRM certificate, install it on remote windows nodes, and then switch
to using the certificate in Ansible for authentication.  A second role
is included which can clean up the cert which is useful for static
nodes.

Since winrm certificates must be acessible within the bubblewrap
container, these roles can be used to restrict the system-wide winrm
cert to trusted playbooks while untrusted playbooks will only have access
to the per-build cert (with appropriate configuration of the executor).

Change-Id: I4efe25594c2f543886a000aa02fb0a38683a43cb
2022-04-13 15:04:51 -07:00
Zuul 9761561e5a Merge "Add upload-logs-ibm role" 2022-04-11 22:30:23 +00:00
James E. Blair a8b4bc6ff3 Add upload-logs-ibm role
This role uploads logs to IBM Cloud object storage.

Change-Id: Ibe1131f863a64051b427fcb03b126b1577c4843a
2022-04-11 14:20:49 -07:00
Zuul 70c30231cd Merge "Add tox-py310 job" 2022-04-04 13:51:04 +00:00
Sorin Sbarnea f5dbe71c49 Add tox-py310 job
Change-Id: Ib6a06893c0b4c7e5c988acf7315b17367b330d9e
2022-03-24 13:25:04 +01:00
Ian Wienand ad7093c17b encrypt-file : role to encrypt a file
This is a role that takes some ASCII gpg keys, and encrypts a file
with them.

Change-Id: If2fe7921ff051a1c5d0589f5e32fba26d30ae96c
2022-02-19 08:05:40 +11:00
James E. Blair 90c427d630 Switch docs theme to RTD
To match change I2870450ffd02f55509fcc1297d050b09deafbfb9 in Zuul.

This does not use the versioning feature due to the nature of this repo.

This also corrects a reference which is now an error.

Change-Id: Ia1d31df932b447f11bc588925de9974d4f6dfc7d
2021-12-16 06:48:49 -08:00
Andre Aranha cac1875575 Add fips version of jobs needed for OpenStack
FIPS needs to be enabled before test-setup is run, as enabling
FIPS requires the node to be rebooted, test-setup needs to run and
setup the environment after the reboot.

Change-Id: I6fecb9c6e917d1a36b2b82c1b02098eed4323ac7
2021-11-05 14:17:26 +00:00
Douglas Viroel 9107f3ee7d Add FIPS enable multinode job definition
This patch adds a new multinode job definition that enables
FIPS mode prior to multinode configuration.
In order to enable FIPS mode, the OS boot procedure need to be
changed to enable the appropriate kernel flag. This modification
has effect only after system reboot.
The default behavior of this job is to always enable FIPS mode.

Change-Id: I6f1365837d9ed2ba82c391a20f9094c9ef0e6c4e
Signed-off-by: Douglas Viroel <dviroel@redhat.com>
2021-10-20 11:20:52 -03:00
Zuul 7fcb90c59d Merge "Deprecate EOL Python releases and OS versions" 2021-09-30 17:33:30 +00:00
Jeremy Stanley 483838ceca Deprecate EOL Python releases and OS versions
Update the deprecation policy to indicate that zuul-jobs is no
longer tested with EOL platforms. Also explicitly switch the minimum
Python 3 documented to 3.6, and add a note to the tox-py34 and
tox-py35 jobs mentioning that they're no longer directly tested.
Move those jobs to the deprecated jobs list as well, to help
reinforce the point that their continued use is not recommended.

Change-Id: I2edbf8ea010caf7a7641e0d88f360965fc0b96ab
2021-09-23 17:35:32 +00:00
Jeremy Stanley f8a60d416f Pin to funcparserlib prerelease for new SetupTools
SetupTools 58 dropped support for its old use_2to3 option, which has
started surfacing a number of ancient Python packages in need of
updates. In this case, the last full release of funcparserlib (which
is a transitive dependency by way of blockdiag by way of
sphinxcontrib-blockdiag) was in 2013, but luckily they have an alpha
release which we can pin explicitly and pull in as a temporary
workaround to get docs builds going again.

Change-Id: I6903eeac2c479e2da795c1dbd215cdee33d09fd7
2021-09-17 20:34:37 +00:00
Ade Lee be0415e556 Add role to enable FIPS on a node
Adds role to be used to enable FIPS on test nodes, so that projects
can create jobs that would test when FIPS is enabled.

This is pretty much copied from the same role in ansible zuul jobs, where
it works well. Thanks to Paul Belanger for pointing it out.
https://github.com/ansible/ansible-zuul-jobs/blob/master/roles/enable-fips-mode/tasks/main.yaml

An example showing how this is can be used is in:
https://review.opendev.org/c/openstack/barbican/+/760665

Change-Id: If07b8ddb77368d591659f3a111e3f5306daf6f06
2021-06-11 14:30:39 -04:00
Guillaume Chauvel b517747623 Add ensure-skopeo role
Role copied and modified from ensure-podman

As focal doesn't exist for project atomic ppa [1]
Install is performed from opensuse repository only

[1] http://ppa.launchpad.net/projectatomic/ppa/ubuntu/dists/

Change-Id: I72fc2e68768664b80c39bd47295330131337d8b5
2021-05-25 16:38:00 +02:00
Clint Byrum 7eab57ab1e intercept-job -- self-service SSH access
This role is an attempt to allow self-service SSH access to nodes.

Change-Id: Icb6fb50b779c0bf2296e14436e4746355703f2ae
2021-04-23 15:14:06 +00:00
James E. Blair 120a11ef20 Add upload-logs-azure role
Add support for uploading logs to Azure blob storage.

Change-Id: I0347977324b880123c6ed83ded3c39eb210612e2
2021-03-30 22:27:25 -07:00
Sorin Sbarnea 938da600b7 Add tox-py39 job
Change-Id: I46cac3396bd870eb1f54820921df09946184605a
2021-01-27 10:14:32 +00:00
Paul Belanger 777230be59 Create upload-container-image role
This new role will be used to replace our upload-docker-image role in
the future.

Change-Id: I0e2b0cca6575255520aa6d4d48a12128ab5f46cc
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2020-11-13 15:04:27 -05:00
Albin Vass b209381be0 Add nimble roles and job
Installs nim toolchains using choosenim (similar to rustup),
installs dependencies and builds nim projects using
the package manager nimble.

See:
https://nim-lang.org/
https://github.com/nim-lang/nimble
https://github.com/dom96/choosenim

Change-Id: I95e7e02eb975200aed7680880b945261888de5ca
2020-11-08 09:36:05 +01:00
James E. Blair 5603eb2291 Revert "Refactor fetch-sphinx-tarball to be executor safe"
This reverts commit 51a8ed8e95.

This has a typo ("exector").  The fix is obvious, but the bigger
issue is that it was not caught in testing, even though the main
purpose of the change was to re-enable tests.  We should understand
why it wasn't caught in testing and resolve that before fixing and
unreverting.

Change-Id: I3ed407546fecc52d4a039f7959c0521511e6a00b
2020-10-14 13:54:22 -07:00
Zuul 4b19def07d Merge "Refactor fetch-sphinx-tarball to be executor safe" 2020-10-14 18:51:05 +00:00
Ian Wienand 0b9fad9583 update-json-file: add role to combine values into a .json
Ansible doens't really have a great built-in way to modify a json file
(unlike ini files).  The extant docker role does what seems to be the
usual standard, which is slurp in the file, parse it and then write it
back out.

In a follow-on change (I338616c41a65b007d56648fdab6da2a6a6b909f4) we
need to set some more values in the docker configuration .json file,
which made me think it's generic enough that we can have a role to
basically run read the file, |combine and write it back out.

This adds such a role with various options, and converts the existing
json configuration update in ensure-docker to use it.

Change-Id: I155a409945e0175249cf2dc630b839c7a97fb452
2020-10-05 15:18:58 +11:00
Ian Wienand 51a8ed8e95 Refactor fetch-sphinx-tarball to be executor safe
This reverts commit 69a238df46.

The role is re-written with executor-safe methods.

Depends-On: https://review.opendev.org/753222
Change-Id: I0b52eff66bfdca776e0e5c426bf1fc57deb3fc49
2020-10-05 15:14:31 +11:00
Tristan Cacqueray a086fb4333 ensure-zookeeper: add role to setup zookeeper
This role is lifted from https://src.fedoraproject.org/rpms/zuul/blob/master/f/tests/setup_zookeeper.yml

Co-Authored-By: Fabien Boucher <fboucher@redhat.com>
Change-Id: Iec21d12baddf3de580d1941adade107c7e24fdd9
2020-09-24 23:29:59 +00:00
Albin Vass 17d7322bd3 Update hashicorp jobs file with correct title
Change-Id: Ibc3a3f1f442e97d0c2d7edf3383694bcd5ad7cc0
2020-09-01 05:43:46 +00:00
Ian Wienand 048aff6c98 Add ensure-rust role
Add a role to install Rust via the rustup tool.  It defaults to
installing globally, which avoids having to worry too much about
setting paths for follow-on jobs.

Packaged Rust and the upstream rustup install tool can live together,
and there's various documentation about it.  Thus I've made this such
that we can expand it with packaged Rust support if there is a need,
but I have not implemented that yet.

Change-Id: I32f9b285904a7036f9a80ada8a49fa9cf31b5163
2020-08-25 09:04:43 +10:00
Albin Vass 838b0c8877 Add upload-logs-s3
Change-Id: I6ce64734ed5f20a212e6cb953d09ea2769238bea
2020-07-19 21:22:36 +02:00
Albin Vass 3d4f3a3a28 Add linting rule to enforce no-same-owner policy
Change-Id: I92c66a21be95935d11fc8e9887d9d91c645d28d4
2020-06-18 11:06:45 +02:00