It appears that centos-9 stream image builds are broken. We don't
actually care what image we build in this job, so switch to jammy
which should be working.
Change-Id: If574a4b6d26230d7bb98cb2c9eab819a08f10eff
We use latest/stable by default which very recently updated to
1.29/stable. Unfortunately it appears there are issues [0] with this
version on Debian Bookworm which also happens to be the platform we test
on. Our jobs have been consistently failing in a manner that appears
related to this issue. Update the job to collect logs so that we can
better confirm this is the case and rollback to 1.28 which should be
working.
Also update the AWS tests to handle a recent moto release which
requires us to use mock_aws rather than individual mock_* classes.
[0] https://github.com/canonical/microk8s/issues/4361
Change-Id: I72310521bdabfc3e34a9f2e87ff80f6d7c27c180
Co-Authored-By: James E. Blair <jim@acmegating.com>
Co-Authored-By: Jeremy Stanley <fungi@yuggoth.org>
OpenShift has removed their 3.11 container images from dockerhub
and while some of them are available at quay.io, not all of them
are, and it does not appear that we are able to install 3.11 any
more.
To unblock development in nodepool, remove the functional testing.
We will rely on the existing fake-based unit tests now.
Change-Id: Ifb4b02ae1058c7549c30ab915591d4b171e3540e
This upgrades our base container image from bullseye to bookworm.
It also removes some backported packages that were only needed on
bullseye.
Change-Id: I4a009f9f0aaf096f172e3daef7419e6d0c691466
To match Zuul, update the pypi classifiers and testing to indicate
that only Python 3.11 is tested and supported.
Change-Id: Id7d422aaae94961a7ee746e7c69308f04d94954d
Depends-On: https://review.opendev.org/891339
The openshift test job needs a node upon which to install and run
the launcher. Fedora 36 is no longer available and Fedora nodes
are being removed from OpenDev in general, so switch to Debian.
Change-Id: I21ef301eb53112655c4c92f6927e56b3e0d15061
Depends-On: https://review.opendev.org/891212
This fixes an error in the new release of pycodestyle.
It also temporarily stops running the OpenShift test job since
that job requires fedora 36 whose package archive has just been
removed.
Change-Id: I1eee3c26eb79bf866cd5775aa3ebf8d1d8d4ba33
Switch this job to use microk8s, which should be more stable than
minikube on our Ubuntu testing hosts.
Depends-on: https://review.opendev.org/c/zuul/zuul-jobs/+/866953
Change-Id: I8e76581ea5cf4a4241aec48bc59086d7f17cd72a
This adds python 3.11 testing and drops python3.10 in order to keep
testing only the bounds of what Nodepool supports. Note that currently the
python 3.11 available for jammy is based on an RC release. This should
be fine as we do functional testing with a released python 3.11 and that
is what people will consume via the docker images.
Change-Id: Ia3e1c4976e35b254497e60d811f61a5531c69f1a
The default base job nodeset has moved from focal to jammy. Jammy
doesn't have python3.8 to run these jobs. Address that by explicitly
forcing these jobs to run on focal. Also drop the jammy pin from the
linter job as that override is no longer necessary.
Change-Id: Id41d49b09ae214dbea263e536c793070be80ed8f
Co-Authored-By: Clark Boylan <cboylan@sapwetik.org>
This adds python 3.10 testing and updates our docker images to
python3.10.
On the docker image side of things we use opendev's python images which
are based on Debian then have python compiled on top of that external to
Debian python packaging. Debian bullseye ships with python3.9 but our
images come with python3.10. What this means is that we cannot rely on
external wheel caches for Debian bullseye + python3.9 here as we've got
the wrong version of python3.10. The good news is that all of the
dependencies that have historically given us trouble on arm64 in
particular already ship python3.10 aarch64/arm64 wheels. This means we
can rely on pypi as is. This is probably better for us anyway as it
decouples us from relying on additional external resources.
We also update a number of jobs to use nodeset: ubuntu-jammy as this
ubuntu version defaults to python 3.10.
Change-Id: I7fb585bc5ccc52803eea107e76dddf5e9fde8646
Icc9adcbfae7a37c335ce4586741cb27d6e0f5c66 pins the openstacksdk
requirement to <0.99, but this fails when we try to use the zuul
checkout of master openstacksdk in the siblings job.
For now, we just need to drop it from the siblings job.
Change-Id: I6e755ef7d2fc95f7475315ed1be1c5d13e9d0f0b
The rbac.authorization.k8s.io/v1beta1 API was deprecated and replaced
with rbac.authorization.k8s.io/v1. Version 1.22.0 of Kuberenetes removes
the deprecated API which means Nodepool needs to stop using it.
According to the docs [0] version 1.8 and newer support the new APIs.
To address this we update our RBAC client instance to use the non beta
version client and update our version specification in the manifests to
drop the beta version.
Add a release note indicating that K8s 1.8 and newer is now required.
Additionally we unpin minikube in testing to ensure we have test
coverage of this change against newer Kuberentes.
[0] https://kubernetes.io/docs/reference/using-api/deprecation-guide/#rbac-resources-v122
Story: 2009974
Change-Id: Ic21725efc8088e7dfb0777c2c96c742182cdbf93
Drop support for Python 3.6 as it's EOL and the current version of the
IBM cloud SDK requires 3.7+.
Change-Id: Icdac247e36568ae1372188000914fa00ee071baf
As described in the dependent change, the testing done here is better
done by the quickstart jobs these days. The dependent change has
removed the tox environment this calls in Zuul. This removes the job
definiton and related files from nodepool.
Change-Id: I17e1002012e9ac6abc434454af989f1da1c379b7
Depends-On: https://review.opendev.org/c/zuul/zuul/+/826772
This re-enables the siblings job in nodepool but omits master
openstacksdk which is currently in an incompatible state. This
also fixes the dib gate, which uses this job as the basis for
its functional testing.
Change-Id: Id268993dd88079f54516a020555b3e2b40de8394
The OpenStackSDK project is in the process of making a breaking
API change and has stopped running this job on master. Without
an interest in adjusting Nodepool to match before the release,
it is no longer useful to run this job on this repo. Stop running
it until the sdk project is ready to re-establish co-gating.
We also set a constraint to avoid installing SDK 1.0.0 since we
expect that to include the breaking API change.
Change-Id: If9f45e24a71f349a85e94150e6a4d9ee9672173b
Typically we would drop py38 testing at this point, but since Zuul is
still running on py38 we keep py38 here until both can be running py39.
Change-Id: Ic89f1379e50c2b6d73f0fa4687635cf20ae3dec1
Similar to Zuul (I71182e9d3e6e930977a9f983b37743ee3300ec91), the base
images have updated to Bullseye.
This updates various things to get a building Bullseye image.
We have upgrade to 3.9-based images here because OpenDev builds ARM64
wheels for a bullseye+arm64 combo, which we use to speed up the ARM64
cross-build (we do not have any repository of <3.7|3.8>+bullseye ARM64
wheels, so it makes it difficult to use these combos as the
cross-build can take a very long time)
Depends-On: https://review.opendev.org/c/openstack/diskimage-builder/+/806318
Change-Id: I21cfbd3935e48be4b92591ea36c7eed301230753
Version 1.23.0 is failing; create_namespace_role gets a 404. It's
unclear whether this is due to the k8s version change
(v1.21.2 -> v1.22.1) or minikube itself (v1.22.0 -> v1.23.0).
Until we learn more, pin to the previously working k8s version to
unblock the rest of Nodepool.
Change-Id: I3a5a0040e57eae3bb6e5a7c093343232c5324902
This fixes libffi bindep installation on Ubuntu Focal
The Python 3.6 tox tests are switched back to bionic, as Focal nodes
don't have Python 3.6.
Additionally, we squashed the following change into this to unblock
the gate:
Remove nodepool-functional-openstack
This test installs devstack and then nodepool on a bionic host (in
contrast to the -containers variant that builds a container from the
Dockerfile and installs/runs that).
Firstly, devstack support for Bionic is going away soon so we have to
update this. We don't really need to test if we run ontop of a plain
Bionic/Focal host. We have tox jobs testing various Python versions
for compatability, so running on here isn't providing any extra
coverage. DIB can't build many things on plain Bionic/Focal due to
updates or incompatabilities in "alien" versions of RPM, Zypper,
debootstrap, etc. The container incorporates fixes as required and is
where anyone is going to put attention if there are build issues;
hence we're not testing anything useful for image building paths.
Finally we also have nodepool-zuul-functional, which brings up Zuul
and nodepool on a plain Bionic host anyway. Per the prior reasons,
that covers basically the same thing this is providing anyway.
openstacksdk is using this on older branches, but is switched to using
the container job in the dependent changes.
Depends-On: https://review.opendev.org/c/openstack/openstacksdk/+/788414
Depends-On: https://review.opendev.org/c/openstack/openstacksdk/+/788416
Depends-On: https://review.opendev.org/c/openstack/openstacksdk/+/788418
Depends-On: https://review.opendev.org/c/openstack/openstacksdk/+/788420
Depends-On: https://review.opendev.org/c/openstack/diskimage-builder/+/788404
(was : Change-Id: I87318e9101b982f3cafcf82439fdcb68767b602b)
Change-Id: Ifc74e6958f64be70386cdb3e05768d94db75c3bb
This is coming in right around the 1 hour mark. Update it to match
the check/gate image build jobs.
Change-Id: I753172d44e0be5662042b508b0ab6db531f730e4
Require TLS Zookeeper connections before making the 4.0 release.
Change-Id: I69acdcec0deddfdd191f094f13627ec1618142af
Depends-On: https://review.opendev.org/776696
We did this in zuul already so drop python 3.5 in nodepool as
well. The py35 job now fails with [1].
[1] ERROR: Package 'keystoneauth1' requires a different Python: 3.5.2 not in '>=3.6'
Change-Id: Ica2a8c5a5c0037190fe23302304de000bd7ad319
This is running under Python 3.5, and some dependencies have started
dropping Python 3.5 support making the job break. Run it on the
default node.
Change-Id: I7982ca5cc4ecaf00143c82f79f52015e13e07f5f
Fedora 30 is old. Fedora 32 is available, lets use it. We use the new
ensure-zookeeper role as Fedora 32 does not have zookeeper packages
anymore.
Co-Authored-By: Tristan Cacqueray <tdecacqu@redhat.com>
Depends-On: https://review.opendev.org/752757
Change-Id: I3b429be1fa90cc8a546e5331a4d9b38a4659a591
We updated python-base and python-builder to include arm64 images in
support of nodepool's arm64 python-builder image. In doing so we have
discovered a number of issues, but the biggest is slowness of building
python packages in an emulated environment.
In order to speed up package builds we consume the OpenDev linaro
cloud arm64 wheel cache. This doesn't have wheels for every package we
need, but for the things that it does have it will speed up our builds.
One of the risks with this setup is that we're relying on wheels built
for openstack on arm64 and those follow openstack's contraints. In order
to mitigate this risk we set pip install's --prefer-binary flag in the
pip.conf. This means that if openstack's constraints lag what is
availale on pypi we should use the existing wheels as long as they are
valid version according to requirements rather than trying to build from
sdist.
Co-Authored-By: James E. Blair <corvus@inaugust.com>
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
Change-Id: I3b358721eebbceafc12daf9d706306634048b196
We set this for the check and gate pipelines but not release which
made us very surprised and sad.
We made project vars to avoid this problem. This patch uses them.
Change-Id: Ia64323e2222b5a2f442eceb310172826556b0c1a
Now that the underlying images are correctly being built for arm,
we see that several dependencies are missing wheels, and compiling
them under emulation takes a very long time. Until we resolve that,
only build container images for amd64.
When devstack is enabled with TLS support
(If607caf301211181b4f37a2c7012f875de3d285c) the cloud config has a
reference to the CA in /opt/stack/data; thus we need to map it into
the builder/launcher container. We missed this with testing in
Iacc71e9e744249c7ce585ab7131cc9e905d600ff because the image build
failed and thus the container-based job didn't run.
Change-Id: Ica9aecd457f8b6abcfd3cdbdb425d411f98ffeff
We have published the retroactive 3.13.0 tag to dockerhub, so we
can now restore the full set of release jobs to be ready for the
next release.
Change-Id: Ie1019af226569cb5570ef3fc56fa3b739c5973ea
When we tag a release of Nodepool, run the upload-docker-image job, but
tell it to do so in a non-promote mode. This will cause it to
directly upload the images to Docker Hub with the final tags applied.
We want to do this in the release pipeline so that we get images
built with the correct version number based on the git tag.
When run in a pipeline with a tag attribute set, this will use all
three lengths of the version number (3.19.0, 3.19, 3) as tags instead
of the single 'latest' tag which is what is applied in the promote
pipeline.
This temporarily comments out the other release jobs so that we can
manually enqueue the most recent tag and retroactively build and
publish it to Docker Hub. Once this works, we will re-enable those
two jobs.
Apparently we also need to use python3 in this job now. We were
already doing that for Zuul. The default will be switching soon,
and we can remove that var entry then.
Change-Id: Id85bc3f56a9c18f1e6f4b14e6edefe76eabeff3e
James E. Blair