Commit Graph

119 Commits

Author SHA1 Message Date
James E. Blair f0b4930ee5 Remove devstack plugin functional test jobs
The dependent commits mark the point at which all external usage
of these jobs has been replaced with the new jobs, so these can
be removed.

Change-Id: Ie51f9c177668190a55563fca944b825253d90f3f
Depends-On: https://review.opendev.org/667225
Depends-On: https://review.opendev.org/667992
Depends-On: https://review.opendev.org/667993
2019-06-27 17:21:52 +00:00
Zuul 3412764a98 Merge "Add Debian Buster boot tests" 2019-06-12 20:40:31 +00:00
Monty Taylor d85fb24734 Update devstack settings and docs for opendev
The most important thing is that the URL for glean is actually
wrong, which causes jobs to fail.

Also, change documentation while we're changing things.

Depends-On: https://review.opendev.org/655133
Change-Id: I127a33bfc4a3e33249120dcc7ef6142ce4e52a97
2019-04-23 13:03:21 +00:00
Ian Wienand 4379810a07 Add Debian Buster boot tests
Infra has a mirror for Debian Buster now, add boot tests

Depends-On: https://review.openstack.org/649496
Change-Id: Ib1567b2576631c078fe11d0f250aeb4e6f9fa0b3
2019-04-04 10:48:53 +11:00
Zuul 44ae87c310 Merge "Support userdata for instances in openstack" 2019-01-23 14:36:19 +00:00
Zuul bf5a206c08 Merge "Add dogpile.cache master to the -src tests" 2019-01-22 21:15:25 +00:00
Sagi Shnaidman d5027ff6a9 Support userdata for instances in openstack
Use "userdata" from Nova API to pass cloud-init config to nova
instances in openstack.

Change-Id: I1c6a1cbc5377d268901210631a376ca26f4887d8
2019-01-22 19:14:52 +02:00
Ian Wienand 515255f7a8 Add dogpile.cache master to the -src tests
This installs dogpile.cache from upstream master for the -src job.  We
recently saw breakage with openstacksdk and new versions of
dogpile.cache, this would help catch this in the future as both will
be installed from master.

Depends-On: https://review.openstack.org/625467
Depends-On: https://review.openstack.org/626075
Change-Id: I7ac90804504ff37b72d089cf0cd2c7c48858da19
2019-01-15 10:32:44 +11:00
Zuul 2427bfb277 Merge "Add Fedora 29 testing" 2019-01-09 05:54:07 +00:00
Tobias Henkel 52fe25da25 Add unmanaged vm to nodepool functional tests
In order to increase test coverage we should add an unmanaged vm to
the functional tests.

Change-Id: I3908c74ea6e581d61180e1b238e4d0e869751cf3
Depends-On: https://review.openstack.org/625923
2018-12-19 20:23:37 +00:00
Clark Boylan 0d17a2527d Run devstack zookeeper on tmpfs
We've seen connection losses to zookeeper during functional testing on
some cloud regions. Move zookeepers data dir to tmpfs to avoid io to the
disk costs.

Change-Id: If45d5cd7a2b63dbe7f20e7ef528b65ba41af7117
2018-12-18 15:20:47 -08:00
Zuul f13c736f52 Merge "Update devstack test to Fedora 28" 2018-11-27 19:44:12 +00:00
Ian Wienand 4422d53529 Add Fedora 29 testing
Replace Fedora 28 with Fedora 29 functional testing.

Note this changes our Red Hat platforms to use NetworkManager for
interface configuration, rather than legacy scripts.  Fedora 29 has
split the legacy scripts into a new package and it marked for future
removal.  NetworkManager is the default on Centos 7 and will also be
on Centos 8, so it makes sense to use it there too.

Depends-On: https://review.openstack.org/619120
Change-Id: I640838c68a05f3b22683c1e90279725a77678526
2018-11-23 17:49:17 +11:00
Zuul b0730feb62 Merge "move 01-nodepool-setup to a later phase" 2018-11-21 19:29:26 +00:00
Ian Wienand 7679b2c14b Update devstack test to Fedora 28
Running a bit behind on this transition ... s/27/28/ to update to
Fedora 28.  This is the default in dib now.

Change-Id: I648ab9d9ba4bba7323c432c65f3ef056703f4303
2018-10-31 11:30:00 +11:00
Monty Taylor 34dcacfdba
Make functional src jobs actually install from source
devstack will happily install openstacksdk from source here just from
required-projects, but we install nodepool into a virtualenv, and that
is just doing a pip install.

Add an install_openstacksdk function that installs openstacksdk into the
virtualenv so that the devstack magic for installing from source works.

Change-Id: I68c16d8601f6fd96863f205094d27c0da09ef73c
2018-10-21 18:14:12 -05:00
Matthew Thode fbc3735632
move 01-nodepool-setup to a later phase
installs.d installs packages at 01-package-installs, this causes sudo
(installed from package-installs.yaml) to be installed AFTER
01-nodepool-setup is run.  Move it later as the files it sets up are not
needed until after reboot.

Change-Id: I238e517c983793589faa2c1e2dc0b13c70529d4a
2018-10-12 19:11:06 -05:00
Ian Wienand 7015bd9af4 Add instance boot properties
This allows us to set parameters for server boot on various images.
This is the equivalent of the "--property" flag when using "openstack
server create".  Various tools on the booted servers can then query
the config-drive metadata to get this value.

Needed-By: https://review.openstack.org/604193/

Change-Id: I99c1980f089aa2971ba728b77adfc6f4200e0b77
2018-09-21 16:29:16 +10:00
David Shrewsbury bade82d446 Fix plugin and examples for using openstacksdk
These appear to have been missed in: https://review.openstack.org/572829

Change-Id: I5c008c369b3789c3ae79ce89726194ab715767a9
2018-07-17 15:02:59 -04:00
Markos Chandras 06bd892248 Add opensuse-150 to nodepool dsvm
openSUSE Leap 15 is the latest version of openSUSE, bring an image
online to validate we can properly build it.

Depends-On: https://review.openstack.org/#/c/572424/
Change-Id: Ib0f48d9788aafd763e857c2d33784c4f75af4c17
2018-06-06 11:13:08 +01:00
Zuul 9e5df7325b Merge "Update logging format for devstack jobs" 2018-05-09 23:19:43 +00:00
Paul Belanger 508cee4c50
Update logging format for devstack jobs
Because we run nodepool-launcher under systemd, we actually get double
datetime logging. We can remove the timestamps from nodepool's
logging.conf and depend on systemd.

Change-Id: I708c4dd78dd4134b36aa4beecde237e728e2cd08
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-05-08 12:44:39 -04:00
Paul Belanger 35d589296c
Remove debian-jessie from nodepool dsvm testing
Now that debian-stretch is working as expected, we can remove
debian-jessie.

Change-Id: If897757023772bb4549e40e7fcd048998175fb5b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-05-04 12:16:34 -04:00
Paul Belanger 842a2fdf04
Use AFS mirrors for debian-stretch testing
We do have AFS mirrors in place for debian-stretch, we should be using
them to help speed up builds.

Change-Id: Ie0eb818edbd76326d12a4e98cf709271b8096409
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-03 17:24:22 -04:00
Ian Wienand ee78684521
Test growroot in boot tests
Test that we see the root partition grow.

Increase the root disk size to 5gb, and check that the booted vm has
grown the disk to at least that.  Add disk size tracking so we can
more clearly see what's being built into the images.

Change-Id: I377beffc4896e03f0c2d01c0061c5f8652e8b1d1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-02 13:38:53 -04:00
Paul Belanger ddc27435b0
Enable AFS mirrors for ubuntu-bionic testing
We now have AFS mirrors in place for ubuntu-bionic, allow the nodepool
dsvm job to use them.

Change-Id: If79639154613e2a6a08fe3a269c17335f5b6451f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-03-30 22:29:30 -04:00
Paul Belanger 3088534dc1
Add debian-stretch to nodepool-functional-py35-debian-src
Increase our test coverage for debian-stretch, as this is the latesst
stable version of debian.

Change-Id: I05cbfe9735eb0b3900203fbd423f68483b1cbf5d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-03-30 11:12:14 -04:00
Matthew Thode 593ce04f72 add gentoo systemd image for testing
Depends-On: https://review.openstack.org/548604
Change-Id: I7efb78b509713fa021419ba50a3d8878b284293f
2018-03-29 16:36:17 +11:00
Paul Belanger ee085dfd00
Support ubuntu-bionic (18.04) with nodepool dsvm
Since ubuntu-bionic is the next LTS version of Ubuntu, start work to
ensure we can build DIBs for it.

We'll also need to bring online AFS mirrors.

Change-Id: I2e523eee4e99e5aea3d57d7ad224dbec39e2e4a5
Depends-On: https://review.openstack.org/485748
Depends-On: https://review.openstack.org/545611
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-26 09:23:34 -05:00
Paul Belanger 17dfd27874
Source /etc/ci/mirror_info.sh for mirrors
Switch to using /etc/ci/mirror_info.sh for mirror information.

Change-Id: I66f8075a7fe6fe41d981b307b2cb99dddada95a3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-19 23:35:51 -05:00
Paul Belanger 1d3989ca35
Add opensuse-tumbleweed to nodepool dsvm testing
We'd like to bring opensuse-tumbleweed online for openstack-infra, so
enable testing of tumbleweed to help catch things.

Change-Id: Id0a1fbf6f3c7df63402670f8c54f7fb3af652ae5
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-15 20:16:36 -05:00
Paul Belanger 796324f15e
Use $NODEPOOL_KEY_NAME for dsvm testing
Follow up patch to previous changes and removes hardcoded 'root'
key-name.

Change-Id: If1ac2b0a55898607409216a3ffa1bb45831ad2fc
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-15 20:12:20 -05:00
Paul Belanger 04c586e644
Take 2: Add functional test for key-name and glean
Validate that glean will properly use the key-name from nodepool, and
allow a root user to ssh into the node.

It seems when we merge feature/zuulv3 back into master, we lost this
logic. It is still helpful to actually SSH into a node to validate
glean and diskimage-builder worked as expected.

See: https://review.openstack.org/455770/

Change-Id: I03f7f04be6c7889f94abed2d9e0a56d7e05ad90f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-15 16:53:21 -05:00
Paul Belanger c857a38650 Switch to fedora-27 for nodepool dsvm
Fedora-27 is the latest supported version, switch to it. We also have
fedora-27 DIBs live in openstack-infra.

Change-Id: Iff9e01a8aab4ba50c42e5e72fabbe8cb20bc821c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-15 08:41:03 +11:00
Paul Belanger 230c4ff792 Clean up old images for testing
Both fedora-25 and ubuntu-precise are nolonger tested. Remove dead
logic.

Change-Id: Ib54827bfea01cec39bd161a5a94fee350556102f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-15 08:41:03 +11:00
Ian Wienand de24f25e58 Add native distro test jobs
This implements legacy-dsvm-nodepool-[distro]-src jobs as native jobs.
This seems like the best home for them, as they are run by multiple
externals such as glean and diskimage-builder.

Previously the defaults were set such that with no arguments, the
trusty test ran for nodepool-functional-py35.

To facilitate better templating, this turns all builds off by default.
The nodepool functional test is moved to a base template, and then
nodepool-functional-py35 explicitly builds and boots Xenial now
(trusty doesn't seem very useful).

The check_devstack_plugin.sh script runs after devstack, so needs to
source the stackrc file to pull in the variables about what images are
paused or not.

Additionally, the path for the script is fixed to nodepool so we can
run the job from other projects.

The redhat, ubuntu and suse legacy tests are re-implemented for their
respective builds.

This also highlighted that the opensuse test wasn't actually doing
anything.  This actually adds the configuration to build opensuse-423.

Needed-By: https://review.openstack.org/543270
Needed-By: https://review.openstack.org/543328
Needed-By: https://review.openstack.org/543329
Needed-By: https://review.openstack.org/543330
Change-Id: I203d149a1d63ac8358e1c8b878d9c2bc0ba67c02
2018-02-13 19:05:48 +11:00
James E. Blair baa831192f Store build logs automatically
This updates the builder to store individual build logs in dedicated
files, one per build, named for the image and build id.  Old logs are
automatically pruned.  By default, they are stored in
/var/log/nodepool/builds, but this can be changed.

This removes the need to specially configure logging handler for the
image build logs.

Change-Id: Ia7415d2fbbb320f8eddc4e46c3a055414df5f997
2018-02-09 07:50:20 -08:00
David Shrewsbury 8d910aca97 Convert from legacy to native devstack job
Changes nodepool-functional-py35 and nodepool-functional-py35-src
to use the native devstack job and not the legacy job.

Change-Id: I3d97d83360816001da0f599d95b13eae3abb6c96
2018-01-24 11:45:10 -05:00
Paul Belanger 4c533c8718
Drop python2 virtualenv for devstack
We no longer support python2, so we can just test using python3
virtualenv.

Change-Id: Id01f470e2d3424c3789a8f05f9eb05ac783f8363
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-21 14:19:01 -05:00
Paul Belanger 1f175a4754
Add support to test fedora-26
Fedora 26 is now the latest version, so lets start supporting it for
nodepool devstack testing.

  https://fedoraproject.org/wiki/Releases/26/Schedule

Change-Id: I9cde430a8cda53357851ab527446f6b32919907e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-07-12 07:06:45 -04:00
Paul Belanger 9de3191baf
EOL ubuntu-precise for dsvm job
No need to build ubuntu-precise any more, it is EOL.

Change-Id: I16ef944d01c699971a136600da8e7e3c710919a3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-07-11 11:57:22 -04:00
Paul Belanger 1a804c7859
Add console-log to config-validate
Sadly, I missed this on our previous commit. Also update coverage from
nodepool dsvm job.

Change-Id: I6966957ac8162a588531c38bd69a93fb58a15258
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-05-29 15:31:54 -04:00
Jenkins db7dd563a8 Merge "Fetch server console log if ssh connection fails" into feature/zuulv3 2017-05-26 18:35:33 +00:00
David Shrewsbury 08926eb1de Do not request nodes for which we have no images
Set the min-ready value based on whether or not we are building
images for those node labels.

Change-Id: I774d39d0147343b07a9ba653fa875d7fd5a3a416
2017-05-26 12:16:58 -04:00
Paul Belanger 969a838d83 Drop -e for pip install for devstack plugin.sh
Stop using -e for pip install, we shouldn't need to write anything to
the git repo.

Depends-On: Ie68d7da0c0e075826fdba77ba59e05bddb932d7b
Change-Id: Idd82eb792201a02a6645ffafd958d86583e5b010
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-05-26 13:17:22 +00:00
Clark Boylan 2852fd598a Revert "Use devstack's zookeeper support"
This reverts commit e5f5840c27.

Devstack has removed zk support from itself in favor of etcd3. We must
manage our own zk destiny.

Change-Id: Ie68d7da0c0e075826fdba77ba59e05bddb932d7b
2017-05-25 16:32:40 -07:00
Paul Belanger 6e34b65d62
Support both python2 and python3 virtualenv
Now that nodepool support python3, allow our dsvm job to create the
appropriate virtualenv for the version of python we'd like to test.

Change-Id: Ib573bfcfd367a983d510238f2f6456ffd710a4ab
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-05-24 14:14:43 -04:00
Paul Belanger e4e98123d3
Fetch server console log if ssh connection fails
Currently, if the ssh connection fails, we are blind to what the
possible failures are.  As a result, attempt to fetch the server
console log to help debug the failure.

This is the continuation of I39ec1fe591d6602a3d494ac79ffa6d2203b5676b
but for the feature/zuulv3 branch. This was done to avoid merge
conflicts on the recent changes to nodepool.yaml layout.

Change-Id: I75ccb6d01956fb6052473f44cce8f097a56dd16a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-05-23 12:53:44 -04:00
David Shrewsbury 13ad80ca31 Update devstack plugin for security groups
The nova client no longer supports security group actions. Use the
openstack client instead.

Change-Id: I6a7740340caa2a7607e95744de31801fa12955cf
2017-05-15 13:35:31 -04:00
Ian Wienand 10f1f9e2bd Use full path to socat in devstack plugin
Now devstack is systemd-ified, make sure this has full path to avoid
it complaining.

  [/etc/systemd/system/devstack@statsd.service:6] Executable path is not absolute, ignoring: socat -u udp-recv:8125 -

Change-Id: I47ebf5abb86bc202e5bf90b0cc318764cbb0d062
(cherry picked from commit 4170b3eb3b)
2017-05-02 09:34:00 -07:00