If we build an image with `docker_registry` set, it will fail to
tag the image for the buildset registry since we do not prefix the
image with `docker_registry` when retagging it (if set).
This patch resolves this by adding that prefix if it is set so
it will refer to an image that exists.
Change-Id: Iba29161156c8e8ff4f79a92771456cf105d780fe
Docker has experimental support for building multi-arch
container images with a buildx command. Currently it only
supports pushing to a registry after running and the images
don't end up in the local docker images list. To work around
that, push to the buildset registry then pull back. This
is the inverse of the normal case where we build, then
retag, then push. The end result should be the same.
Change-Id: I6a4c4f9e262add909d2d5c2efa33ec69b9d9364a
This adds a custom ansible-lint rule at .rules/ZuulJobsNamespaceLoopVar.py
that enforces the loop var policy described at:
https://zuul-ci.org/docs/zuul-jobs/policy.html#ansible-loops-in-roles
It also updates existing roles to follow the policy.
Change-Id: I92b2ff56a1c2702542fc07b316f1809087a4c92f
Skopeo has problems with ipv6 address literals just like docker as they
use the same underlying checks for url validity. However, we think we
can fix that by using a port forward from the executor to the buildset
registry so that skopeo can connect via ipv4
Go back to aliases the registries on test nodes via /etc/hosts.
Change-Id: I5f9316ffe84de06cb2fb2b65a7e1c31d9f8b0e35
Co-Authored-By: James E. Blair <jeblair@redhat.com>
This reverts commit 05f20a5396.
Apparently skopeo is properly cloud native too and doesn't support ipv6
either. I think it is pulling the same docker
distribution/reference/regexp.go lib in and using docker's regex.
The error we get from skopeo:
time="2019-04-10T15:15:48Z" level=fatal msg="Invalid source name docker://[2607:ff68:100:54:f816:3eff:fef2:fc69]:5000/zuul/nodepool:latest: invalid reference format"
Change-Id: I6f916574c9f46e8fdd2464465e2b36ecf8719b16
We only need to alias registries on the build nodes when running docker.
We cannot alias them in /etc/hosts in roles that are expected to run on
localhost beacuse /etc/hosts is bindmounted read only on localhost. This
assumes that skopeo handles ipv6 properly (which has not been tested).
If skopeo does not handle ipv6 properly then we'll need additional
fixing.
Change-Id: I40e5b1bac5aeaf2d42aa05a72b9ced72b7d222c0
This reverts commit a307259776.
We need to additionally handle this on localhost (the zuul executor)
where we cannot edit /etc/hosts. At least I don't think we can. We also
need to handle the case where buildset_regsitry is not yet defined.
Change-Id: I4928f7fcf58e88cf360de253f01b16546220aace
Docker doesn't appear to understand properly escaped ipv6 addrs in its
"urls". Address this by adding /etc/hosts entries for any
buildset_registry that is specified by an ip address (v4 or v6). This
allows us to use a named alias instead of the ipv6 address.
An example failure for posterity:
"[2607:ff68:100:54:f816:3eff:fe7c:e98a]:5000/zuul/nodepool:latest" is not a valid repository/tag: invalid reference format
Change-Id: Id865dc7d3382174b61f9eaa76e29b637a85f5142
The approach of having the proxy serve the local data as well as
the remote wasn't working -- it seems that the proxy would always
check upstream and prefer that data even if it had been pushed
locally.
To correct this, separate the data stores of the two registries,
and add both of them to the registry_mirror setting for the
docker daemon. Now we will pull from our buildset registry first,
and fall back on the proxy to talk to upstream if an image is not
found locally.
The proxy is still required in order to mask out the username and
password which dockerd will otherwise use when talking to upstream.
Change-Id: Iab11954a4b5431d3b1a4d4753f519b6b71f64094
When copying images from the intermediate registry to the buildset
registry, use the new push endpoint of the dual-registry system.
Also, use the push endpoint after a docker build to push the
new image to the buildset registry.
Change-Id: I3a11036bb9fb7cb3457a3d744fa83647c1b1b085
* In the build-image role, push to the buildset registry if it is defined.
* In the intermediate registry push and pull roles, ensure that the
buildset registry TLS cert is in place. This is a self-signed cert,
and so needs to be written for each run. This happens inside
bubblewrap where we have permission to write to /etc, which is an
ephemeral volume.
Change-Id: I47781d8a7adb93817dfe9266e2f4ad5fd829385c
Mohammed Naser