Because buildset registries may be used by jobs that finish before other
jobs are finished using the buildset registry we must be careful not to
expose the registry credentials in the jobs that finish sooner.
Otherwise logs for the earlier job runs could potentially be used to
poison the registry for later jobs.
This is likely currently incomplete. Other Zuulians should look over it
carefully to ensure we're covering all the bases here.
The cases I've identified so far are:
* Setting facts that include passwords
* Reading and writing to files that include passwords (as content may be
logged)
* Calling modules with passwords passed as arguments (the module
invocation is logged)
I've also set no_log on zuul_return that passes up credentials because
while the logging for zuul_return is minimal today, I don't want to
count on it remaining that way.
We also use the yet to be merged secret_data attribute on zuul_return to
ensure that zuul_return itself does not expose anything unwanted.
Finally it would be great if others could check over the use of
buildset_registry variables to make sure there aren't any that got
missed. One thing I'm not sure of is whether or not when conditionals
get logged and if we need to be careful about their use too.
Temporarily remove some buildset-regitry jobs which are in a catch-22.
Change-Id: I2dea683e27f00b99a7766bf830981bf91b925265
When using docker buildx to build a container image, use a temporary
registry to receive the built image instead of requiring a buildset
registry.
A multi-arch test is also added with a publication registry
using the same task list to reduce duplication.
Change-Id: Ib20d1c97f6cb63e0ff9d8888ea792d1941cd8690
Co-Authored-By: James E. Blair <jeblair@redhat.com>
The buildx patch unfortunately changed the logic associated with
siblings to set up siblings in a loop one time, rather than to
do a loop of "set up siblings, build, cleanup siblings". This causes
builds to fail when they're using siblings with an error about
siblings dir not having been cleaned up.
Change-Id: I3c45bfa77ec9f2609689e04044c18f066adc9741
Docker has experimental support for building multi-arch
container images with a buildx command. Currently it only
supports pushing to a registry after running and the images
don't end up in the local docker images list. To work around
that, push to the buildset registry then pull back. This
is the inverse of the normal case where we build, then
retag, then push. The end result should be the same.
Change-Id: I6a4c4f9e262add909d2d5c2efa33ec69b9d9364a
When you build from a Dockerfile, it runs in a given "context"; that
is the directory the Dockerfile is in and the directories below it.
It can not access anything outside that context during the build.
When building a container for a project in the gate, you may wish to
install sibling projects that Zuul has checked-out into your container
(i.e. so that Depends-On works). As mentioned, because
/home/zuul/src/<project> is not in the context of the current project,
you will not be able to access this source code during the container
build.
So to help facilitate dependencies, add a siblings: tag which can copy
some or all of the required-projects already specified for the job
into a special sub-directory of the current source.
Because all the code is now in the same context, this will allow build
scripts to be written that look for directories in .zuul-siblings and
can install the source code from there. To further help the scripts,
the ZUUL_SIBLINGS arg is set for the docker build giving the copied
paths.
The test is updated with some paths to test the copy.
Change-Id: I079d823e7194e15b1b496aea0f53f70f6b563f02
We recently added several tasks to the build-docker-image role to
work around docker bugs when using the buildset registry; but this
role doesn't require it so they should be wrapped in a conditional.
Change-Id: Id6e85d07fe34aeb272d7388c778455d5d2a402dd
So that users can specify two docker image builds for the same
repository, but with different tags, ensure that the temporary
change_ tag attached to the image also includes the final tag
name.
This allows this configuration to work:
docker_images:
- repository: foo/image
context: opensuse
tags:
- opensuse-latest
- repository: foo/image
context: ubuntu
tags:
- ubuntu-latest
Change-Id: I917dcf8a74fc864ea06dc70bdb3e212dc170eb48
* In the build-image role, push to the buildset registry if it is defined.
* In the intermediate registry push and pull roles, ensure that the
buildset registry TLS cert is in place. This is a self-signed cert,
and so needs to be written for each run. This happens inside
bubblewrap where we have permission to write to /etc, which is an
ephemeral volume.
Change-Id: I47781d8a7adb93817dfe9266e2f4ad5fd829385c
In certain build projects, multiple Dockerfiles exist (for example,
one per distro) to simplify reading. However, this role is hardwired
to use dockerfiles only named "Dockerfile".
This is a problem, as you can't override the filename neither per
image, or globally.
This should fix the problem, allowing certain images to be build
by providing the dockerfile argument in docker_images, but also
have a globally overridable flag if you are using a different
convention (for example Dockerfile.distro_minordistroversion).
Change-Id: I075c365bc9f4f85f9ada832d22d1f1e213e68e21
When doing the local build, go ahead and apply the tags to the
local image, so that one can use the role for building local
images for testing that will eventually be published with the given
tag.
Change-Id: I0249ddc4f9a8a2e17466f96a5711672282ce025c
These probably should have been prefixed to start with. The roles
are brand new, not publicised, and likely not widely used. I think
we can merge this without announcement or deprecation.
Change-Id: I7825ef6fee1325b6d4fcc179032652eb5530d016