Commit Graph

185 Commits

Author SHA1 Message Date
James E. Blair 73bdf1f2df Add zuul-tenant-conf-check role/job
This performs static validation of Zuul tenant config files.

Change-Id: I5d439d6cfb963e55d07b2a0058de76f030fe47b3
2024-02-01 15:56:29 -08:00
Clark Boylan 378e039dba Remove the nox-py27 job
We added nox somewhat recently and set it up to mimic existing tox jobs.
This meant adding py27 jobs. Since then (in OpenDev at least) only a
single project has used the py27 job: Bindep. Bindep is dropping support
for python2.7 as the need for it has come to an end. Additionally, nox
doesn't work with python2.7 out of the box due to a virtualenv
dependency that ends up being too new for python2.7 venv creation.
Rather than hack around that let's drop python2.7 job support.

Change-Id: I52c07b01ad173304c19b13a10927fdadf9d84170
2023-08-22 14:28:36 -07:00
Clark Boylan 7ae274dff8 Fix nox job python2.7 support
Virtualenv dropped python2.7 support. When we install latest nox it also
installs latest virtualenv. Then when we try to run python2.7 jobs which
will create a python2.7 venv things fail with:

  RuntimeError: failed to find interpreter for Builtin discover of python_spec='python2.7'

The underlying issue is that we are getting a too new virtualenv. Fix
this by adding a pre run step to the python2.7 jobs that downgrades to a
version of virtualenv that should support python2.7 environments.

Change-Id: I0ed9915b25c752cdaae4df19c9a0e0dc1aafd3d7
2023-08-22 10:02:22 -07:00
Clark Boylan 5994ce4049 Gracefully handle use of intermediate registry in container upload role
For symmetry and ease of transition between the docker specific
jobs/roles and generic container jobs/roles it is advantageous to have
the container upload role skip pushing artifacts to the final registry
location if we are relying on the intermediate registry instead.

Update the container upload role to skip pushing to the actual registry
if the promote var is set to intermediate registry. This allows us to
avoid reshuffling all of our jobs as we migrate between the two
implementations.

Change-Id: I3cae9e03517cb0a5ce8e9369bf43fd052cac97ff
2023-05-09 16:18:42 -07:00
Ian Wienand 0a64d51c3d
promote-container-image: add promote_container_image_method
After recent conversations, we've come to the conclusion it will be
good to have two models of promotion

 - using tags, where gate directly uploads to the final repository and
   promote retags the image.

 - from an intermediate-registry, where upload stores the built image
   in an i-r and the promote step uploads to the final registry.

To facilitate this, we add a "promote_container_image_method" flag to
the promote roles.

The documentation is expanded to explain how all this is intended to
work together.

These roles haven't been publicised yet, but this should be a no-op as
it defaults to tags, which is the current operation.

c.f. Ia24bbd101e01ab371ceacfed006b5ff806418a97

Change-Id: I1c25f60f835b1cab983bcdd169eeffc0e250a56c
2023-04-04 09:50:17 +10:00
James E. Blair c14523148a Fix container-image pre playbook container_command default
This variable may be undefined, so use the default documented by
the roles.

Change-Id: I842a596402f30f1c9a1c2369a6dde23c346fd5f5
2023-03-23 12:56:45 -07:00
James E. Blair 466aa92635 Add container build jobs
These jobs use the container build roles.

Change-Id: I13d1987980bc3d0b1c717878a4bc47edc6dcfe1c
2023-03-23 09:47:49 -07:00
Ade Lee 99711abf23 Add ubuntu to enable-fips role
The enable-fips role has been refactored to support both centos/rhel and
Ubuntu.

In addition, for the Ubuntu tasks, a small role is added to enable a
Ubuntu Advantage subscription.  This is required because Ubuntu requires
a subscription to enable FIPS.  This role takes a subscription key as a
parameter (ubuntu_ua_token.token).

In Openstack, this is provided by the openstack-fips job in
openstack/project-config, which will be the base job for OpenStack jobs.
This job will provide the ubuntu_ua_token.token.

Change-Id: I47a31f680172b47584510adb672b68498a85bd32
2023-02-09 19:02:00 +00:00
Clark Boylan 50fd134646 Add nox role and some simple jobs
This is an alternative to tox.

Change-Id: Ib4920acec09c2c980af909e8f9d1eabd1c6d253a
2022-12-19 18:41:12 -08:00
Michael Kelly b0cc01ceac
helm: Add job for linting helm charts
This job runs 'helm lint' on the user specified charts.

Change-Id: Ie6f39bab366b683e773add181de516c3ac913866
2022-11-16 15:09:07 -08:00
Ian Wienand a016a1a565 linters: standardise on newline at end of file
I noticed this by accident when I ran ansible-lint over this repo from
an outside context; it didn't use the .yamllint in here and started
compalining about eof whitespace.

After scratching my head for a bit as to why this didn't fail here, I
realised we've allowed various newlines since the initial commit
I936fe2c997597972d884c5fc62655d28e8aaf8c5.

Remove this and just use the default eof rules, and fixup the
whitespace as required.  This is fairly unimportant, but is nice for
consistency.

Change-Id: Idb46a1f39ba798b0bf70eaa27b4c6b4758ce3d26
2022-07-28 16:19:06 +10:00
Douglas Viroel 9107f3ee7d Add FIPS enable multinode job definition
This patch adds a new multinode job definition that enables
FIPS mode prior to multinode configuration.
In order to enable FIPS mode, the OS boot procedure need to be
changed to enable the appropriate kernel flag. This modification
has effect only after system reboot.
The default behavior of this job is to always enable FIPS mode.

Change-Id: I6f1365837d9ed2ba82c391a20f9094c9ef0e6c4e
Signed-off-by: Douglas Viroel <dviroel@redhat.com>
2021-10-20 11:20:52 -03:00
Sorin Sbarnea 59dad009e9 Bits to keep ansible-lint happy
- moved vars file under vars to avoid failure to load file (newer
  version raise error if you try to put vars outside vars/defaults as
  it will assume they are broken playbooks.
- added missing file-modes on few tasks

Change-Id: I7217469d089b655ee16cd038391cffa4197c8c83
Part-Of: https://review.opendev.org/c/zuul/zuul-jobs/+/773245
2021-03-10 18:04:08 +00:00
Sorin Sbarnea ad3167bd56 Fixes all tasks should be named rule
This prepared for ansible-lint v5 which no longer has exceptions
for task naming, requiring all to be named.

Change-Id: I5e761d1e3836fa270d7afdcf01780320001f820d
Part-Of: #773245
2021-03-08 13:30:32 +00:00
Zuul 0c5dcd17ea Merge "Add nimble roles and job" 2020-11-12 22:44:33 +00:00
Sorin Sbarnea 6b8cc6d468 More E208 (final)
Change-Id: I705d1b10696326f3d4d5bef4b5a88a83f2c3d960
2020-11-10 19:34:45 +00:00
Albin Vass b209381be0 Add nimble roles and job
Installs nim toolchains using choosenim (similar to rustup),
installs dependencies and builds nim projects using
the package manager nimble.

See:
https://nim-lang.org/
https://github.com/nim-lang/nimble
https://github.com/dom96/choosenim

Change-Id: I95e7e02eb975200aed7680880b945261888de5ca
2020-11-08 09:36:05 +01:00
Pierre-Louis Bonicoli 0eb09e7b4f
explicit error when test_command is undef
Don't use a shell builtin: shell builtins aren't available since
'command' ansible module is used. Instead, fail with an explicit
error when the required test_command variable is not set.

Fix this error:

    Ansible output: b'failed: [host] (item=exit 1) => {
        "ansible_loop_var": "item",
        "changed": false,
        "cmd": "exit 1",
        "item": "exit 1",
        "msg": "[Errno 2] No such file or directory: 'exit': 'exit'",
        "rc": 2
    }

Change-Id: I88303f7302d7354ffc8b18e607b28349a9860a57
2020-09-22 13:39:22 +02:00
Zuul 1ba95015ac Merge "Avoid to use 'length' filter with null value" 2020-08-06 13:36:52 +00:00
Pierre-Louis Bonicoli 222056a26f
Avoid to use 'length' filter with null value
This error occurs when variables are null and
DEFAULT_JINJA2_NATIVE is enabled:

    object of type 'NoneType' has no len()

Change-Id: I7c69be60b846c03e30a0a6e8893bf2f3a3d5e5b8
2020-08-03 03:31:29 +02:00
Tristan Cacqueray 61228901c2 dco-license: remove the empty nodeset
The job was relying on a security hole to execute untrusted command
on the executor (localhost). This change prevents failure after the
hole is fixed.

Change-Id: I910fb6e6a2659e77991111c8ce5a6b763556aeec
2020-07-22 14:46:41 +00:00
Tristan Cacqueray 2d16f0fd07 phoronix-test-suite: extract ensure- role from run playbook
This change adds a new phoronix-test-suite/pre.yaml playbook to
ensure-phoronix-test-suite is installed.

Change-Id: I30d8cca408cca006a452db35cb4f40d4af526dec
2020-06-18 13:58:15 +00:00
Ian Wienand 67f223b53a Partial revert "Ensure wheel exists for build-release-python"; move to ensure-pip
This partially reverts commit
3f961ce202.

This alternative installs wheel with the ensure-pip role instead of in
a separate role.  wheel is very closely linked with pip install
operations so this isn't a large overreach of the role.

I suggest this for several reasons; firstly the python-wheel role
doesn't try to install packages, so we end up with mixed system pip
and upstream versions of wheel most of the time.  This is the type of
thing that has proven problematic in the past.  It also installs via
pip --user; something we've already had problems with tox when for
various reasons roles want to run this as non-zuul user.  Using
ensure-pip we keep the packaged versions together.

[1] did try to install wheel with root, but during runtime which
didn't work due to sudo being revoked.  This should work for the
existing build-python-release job, because it already includes
ensure-pip in pre-run via playbooks/python/pre.yaml

I believe our conclusion on the ensure-* roles was that requiring
root/become: for installation is OK, but we should have a no-op path
if the tools are found.  This is consistent with that approach
(i.e. if you want wheel and can't do sudo, you should pre-install it
on your image using whatever you build that with).

This adds a check to the existing "is pip installed" check to also
check if wheel packages are available.  If not we trigger the install
path.

This revealed some issues with RedHat.yaml -- we can always install
Python 3 (packages available for CentOS 7) so remove that check, and
if Ansible is running under Python 2; ensure we install the
dependencies too (not only if it is forced).

Update the documentation to describe that it will enable support for
bdist_wheel, and add a basic sanity test that wheels are produced by
pip.  The existing build-python-release job is kept; although it is
modified to use the playbooks/python/pre.yaml playbook as the build
job does.

Change-Id: I2ab11bb45b6b2a49d54db39195228ab40141185c
[1] https://review.opendev.org/#/c/736001/5/roles/build-python-release/tasks/main.yaml
2020-06-18 12:51:56 +00:00
Sorin Sbarnea 3f961ce202 Ensure wheel exists for build-release-python
We need it to exist for building wheels. It should be
installed in a pre-playbook.

Change-Id: Ieaec3d21911decf11283e82fed97dd26b3a93ffa
2020-06-17 16:59:24 -05:00
Monty Taylor ed3fdf5a8d Add ensure-pip to build-python-release
It assumes setuptools has been installed, which we usually get
via the installation of pip.

Change-Id: I0b8f408a47a78a6a04bdbd083f949bb82713776b
2020-06-16 07:57:26 -05:00
Albin Vass 5bcf93c37d Terraform roles and jobs.
Adds terraform roles to install and execute terraform.

Supports adding an override.tf file to override configuration in CI
which is useful to let zuul handle module reposity authentication
instead of setting up credentials on the remote during the job.

Also returns the execution plan back as a comment for 'terraform plan'
to make it easy for reviewers.

Change-Id: I3b4f2bac7f055a0c0f9cb7888b4146ac9c007d25
2020-06-11 17:26:50 +02:00
Tristan Cacqueray b8e57ffc60 shake-build: add shake build system job
This change adds a new job to run the shake build system.

Change-Id: Iffaf4e0675d91033b1b5d66c52d501f6bdc8acaf
2020-05-28 12:48:16 +00:00
Albin Vass 6292b5ec86 tarball-post.yaml: do not synchronize owner
Change-Id: I6683d80ea11a177da891e3fc529a3125f713656b
2020-05-18 17:05:32 +02:00
Sorin Sbarnea 9baebe3684 yamlint: EOF newlines and comments indent
Fixed two rules which where temporary disabled during introduction
of the linter.

Change-Id: Icd1e1b40b1e8207ab5ff7088a48e8f0a800e3aa8
2020-05-14 08:42:17 +00:00
James E. Blair a3ba8e309a Add zuul_work_dir to run-test-command
To make it so that a repo can host a simple command which is an
integration test of multiple projects, add the semi-standard
zuul_work_dir variable to the job.

Change-Id: I265469ac9c12c3a38a11b243f3ff747d27141f21
2020-05-13 11:31:01 -07:00
Zuul b8a24d635f Merge "Split browser and non-browser targetting js test jobs" 2020-05-12 17:10:19 +00:00
Monty Taylor 0caa85cc48 Split browser and non-browser targetting js test jobs
Some javascript operations, like running tests for web-apps, need
browsers. Javascript packages that are not targetting browsers
do not.

Make a -browser version that installs the browsers and xvfb, and
a non-browser version that does not.

Change-Id: I33c12cb0d9516bdffef7d8d04af4dbcb03ed8355
2020-05-12 11:31:42 -05:00
Zuul 9d18ae4326 Merge "Extract ensure-javascript-build-tool role" 2020-05-12 16:28:17 +00:00
Zuul 7730bf0718 Merge "Add new non-npm specific javascript jobs" 2020-05-12 13:37:06 +00:00
Monty Taylor 5973c9b7b1 Extract ensure-javascript-build-tool role
The pre-playbook for the javascript jobs got complex. Extract it
to a role so we can better document it.

Also - stop installing javascript depends in pre - the depends
declared in a patch should be tested as part of the patch.

Change-Id: I50a483f223620cd3f9ecd82887062cfc9ac64b7f
2020-05-12 07:53:14 +02:00
Monty Taylor 202cce830e Add new non-npm specific javascript jobs
We have a bunch of jobs that are built around the npm role, but
for projects using yarn, that can lead to ignoring yarn.lock.

For projects with a yarn.lock, we can assume the user wants to
use yarn. Make a new js-package-manager role that can detect
if that's the case and otherwise use npm. Make an js_build_tool
parameter that allows the user to override that auto-detection.

Make a whole new suite of jobs that do this behavior, do not have
npm in their name, and default to the latest node LTS, version 14.

Don't install yarn if we're not going to use yarn. Also allow people
who want to use yarn but don't have a yarn.lock to override
js_build_tool everywhere we do that logic.

Mark the old jobs deprecated.

Shift the npm and yarn roles to use the new js-package-manager role
with defaults set.

Change-Id: I8013228ca05607a69f390a9bb75991fc6543f865
2020-05-12 07:50:19 +02:00
Mohammed Naser 1e72ab0a13 tox: run ensure-python first
We currently run ensure-python after running ensure-tox which means that
both ensure-pip and ensure-tox ran against the system Python instead of
the version that's created by ensure-python.

Depends-On: https://review.opendev.org/726549
Change-Id: Id2100bb5f981944223063cf4b2a5a568cc1dc366
2020-05-09 17:56:37 +00:00
Sorin Sbarnea 4af438d136 Made sequence indent consistent
Change-Id: I5ea4232ca4fd6e03d5b5f72eb6704bee84d04ea5
2020-05-07 14:13:55 +01:00
Tristan Cacqueray 4903ecd30b dhall-diff: add new job
This change adds a new dhall-diff job to verify generated config
is idempotent.

Change-Id: I96a335dc78c4fa74564b854997433e5be0b5e633
2020-05-03 19:06:37 +00:00
Zuul 10cd19fdae Merge "ensure-yarn: run ensure-nodejs before" 2020-05-01 17:24:27 +00:00
Zuul 096b1218c3 Merge "haskell-stack-test: add haskell tool stack test" 2020-05-01 16:09:58 +00:00
Tristan Cacqueray cf64b73f78 haskell-stack-test: add haskell tool stack test
This change adds a new job to test stack based haskell project.

Change-Id: Ie3492097b351eec743d98e9ca8bc693c00c11a78
2020-05-01 15:35:33 +00:00
Mohammed Naser 6920617af5 ensure-yarn: run ensure-nodejs before
YARN needs a modern version of NodeJS and if we don't use the role
to get the latest version, we end up failing on most platforms that
ship an old version of it.

This patch also adds testing to validate that ensure-yarn alone works.

Co-Authored-By: Andreas Jaeger <aj@suse.com>
Change-Id: I83ac09bc5751c85c6f1eac550d289a300ca2255e
2020-05-01 17:07:48 +02:00
Albin Vass d0e2016592 Add loop var policy to ansible-lint
This adds a custom ansible-lint rule at .rules/ZuulJobsNamespaceLoopVar.py
that enforces the loop var policy described at:
https://zuul-ci.org/docs/zuul-jobs/policy.html#ansible-loops-in-roles

It also updates existing roles to follow the policy.

Change-Id: I92b2ff56a1c2702542fc07b316f1809087a4c92f
2020-04-29 17:20:59 +02:00
Tristan Cacqueray 2e82d88589 hlint: add haskell source code suggestions job
This change adds a new job to run hlint on haskell source files.

Change-Id: Ibf76c5552acecd68dfc56c4f31d8045ca5b233f1
2020-04-26 12:05:12 +00:00
Tristan Cacqueray d92b39cae2 cabal-test: add initial haskell job
This change adds a new job to test cabal based haskell project.

Change-Id: Idb24eb5f8f9bd1409bae07bf4da415f1d05568e4
2020-04-24 13:12:13 +00:00
Zuul 21928eab52 Merge "Adds roles to install and run hashicorp packer" 2020-04-15 16:01:18 +00:00
vass 32a3ce3f5a Adds roles to install and run hashicorp packer
Change-Id: I82274ef59eda00a735579082bbd37b64c0599c7d
2020-04-13 16:44:32 +02:00
vass da97adbfb7 Use ensure-* roles
The install-* roles in zuul-jobs have been renamed to ensure-*, follow
the rename.

Change-Id: I32984fcd408cf0cd3751c5c5960608bec6e202a2
2020-04-12 15:26:10 +02:00
vass 6dabffe402 Rename install-go to ensure-go for consistency
The old role will be kept and include ensure-go for backwards-compatability.

Change-Id: I4e6345563817057a6c47c124b9ec8b0c62b057d6
2020-04-06 21:10:11 +02:00