This follows the previous change and is intended to have little
or no behavior changes (only a few unit tests are updated to use
different placeholder values). It updates all textual references
of build numbers to build ids to better reflect that they are
UUIDs instead of integers.
Change-Id: I04b5eec732918f5b9b712f8caab2ea4ec90e9a9f
This updates the OpenStack driver to use the statemachine framework.
The goal is to revise all remaining drivers to use the statemachine
framework for two reasons:
1) We can dramatically reduce the number of threads in Nodepool which
is our biggest scaling bottleneck. The OpenStack driver already
includes some work in that direction, but in a way that is unique
to it and not easily shared by other drivers. The statemachine
framework is an extension of that idea implemented so that every driver
can use it. This change further reduces the number of threads needed
even for the openstack driver.
2) By unifying all the drivers with a simple interface, we can prepare
to move them into Zuul.
There are a few updates to the statemachine framework to accomodate some
features that only the OpenStack driver used to date.
A number of tests need slight alteration since the openstack driver is
the basis of the "fake" driver used for tests.
Change-Id: Ie59a4e9f09990622b192ad840d9c948db717cce2
There was only one caller to the change_node_state function, and
it was the one-line hold function. Remove the unecessary indirection.
Change-Id: I13efc73edf807c8cb6b430f207bf7cb067520ba3
With the static driver it is possible to have multiple static nodes
defined that only differ by their username. In order to be able to
distinguish them, include the username in the output of the
"nodepool list --detail" output.
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I702ccbf412731ef3400eb722386629356a244334
This allows nodes to be set in an idle state
so that they will not have jobs scheduled
while e.g. maintenance tasks are performed.
This is probably most useful for static nodes.
Change-Id: Iebc6b909f370fca11fab2be0b8805d4daef33afe
This lets operators see the user_data and driver_data in the node
list when passing --detail. That can be useful for identifying
issues with the metastatic driver, or any other debugging which
would benefit from seeing that data.
Change-Id: I2f36ce98a183b7a8e289376f2228b6370900a057
This augments the dib-request list (which shows what images have
manual build requests) with information about whether the image
is paused. The resulting command is renamed to "image-status".
Change-Id: If75a8757b4ec93563e47bfdf0a239a9c21660c45
Image build requests can now be retrieved through the /dib-request-list
endpoint or via the dib-request-list sub-command. The list will show the
age of the request and if it is still pending or if there is already a
build in progress.
Change-Id: If73d6c9fcd5bd94318f389771248604a7f51c449
Users of the "nodepool" command don't need to see the component
registry logs at info level (which output at least a line for each
connected component). Set the minimum level to warning to avoid
that.
The component registry may still be useful for command-line use
in the future, so we leave it in place rather than disabling it
entirely.
Change-Id: I8c0937d7304ddc536773cf74fc40bbf6e79918d4
We have made many improvements to connection handling in Zuul.
Bring those back to Nodepool by copying over the zuul/zk directory
which has our base ZK connection classes.
This will enable us to bring other Zuul classes over, such as the
component registry.
The existing connection-related code is removed and the remaining
model-style code is moved to nodepool.zk.zookeeper. Almost every
file imported the model as nodepool.zk, so import adjustments are
made to compensate while keeping the code more or less as-is.
Change-Id: I9f793d7bbad573cb881dfcfdf11e3013e0f8e4a3
The launcher implements deletes using threads, and unlike with
launches, does not give drivers an opportunity to override that
and handle them without threads (as we want to do in the state
machine driver).
To correct this, we move the NodeDeleter class from the launcher
to driver utils, and add a new driver Provider method that returns
the NodeDeleter thread. This is added in the base Provider class
so all drivers get this behavior by default.
In the state machine driver, we override the method so that instead
of returning a thread, we start a state machine and add it to a list
of state machines that our internal state machine runner thread
should drive.
Change-Id: Iddb7ed23c741824b5727fe2d89c9ddbfc01cd7d7
This adds a CLI commend to set a flag in ZK for images indicating
that the image should be paused. This can be used to quickly pause
the building and uploading of one or more images globally. This
will effectively be boolean OR'd with the pause value for diskimage
builds in the config file.
In particular, this can be used to pause images for short durations,
either because a fix is imminent, or to allow the system to remain
stable while a configuration change goes through the CI/CD workflow.
Change-Id: I21a573dfc337c51f319afe3695d5446b2c91d70b
Change the 'info' command output to include image upload data.
For each image, we'll now output each build and the uploads for the build.
Change-Id: Ib25ce30d30ed718b2b6083c2127fdb214c3691f4
Having python files with exec bit and shebang defined in
/usr/lib/python-*/site-package/ is not fine in a RPM package.
Instead of carrying a patch in nodepool RPM packaging better
to fix this directly upstream.
Change-Id: I5a01e21243f175d28c67376941149e357cdacd26
We broke nodepool configuration with
I3795fee1530045363e3f629f0793cbe6e95c23ca by not having the labels
defined in the OpenStack provider in the top-level label list.
The added check here would have found such a case.
The validate() function is reworked slightly; previously it would
return various exceptions from the tools it was calling (YAML,
voluptuous, etc.). Now we have more testing (and I'd imagine we could
do even more, similar vaildations too) we'd have to keep adding
exception types. Just make the function return a value; this also
makes sure the regular exit paths are taken from the caller in
nodepoolcmd.py, rather than dying with an exception at whatever point.
A unit test is added.
Co-Authored-By: Mohammed Naser <mnaser@vexxhost.com>
Change-Id: I5455f5d7eb07abea34c11a3026d630dee62f2185
Now that there is no more TaskManager class, nor anything using
one, the use_taskmanager flag is vestigal. Clean it up so that we
don't have to pass it around to things anymore.
Change-Id: I7c1f766f948ad965ee5f07321743fbaebb54288a
We currently only need to setup the zNode caches in the
launcher. Within the commandline client and the builders this is just
unneccessary work.
Change-Id: I03aa2a11b75cab3932e4b45c5e964811a7e0b3d4
In order to support static node pre-registration, we need to give
the provider manager the opportunity to register/deregister any
nodes in its configuration file when it starts (on startup or when
the config change). It will need a ZooKeeper connection to do this.
The OpenStack driver will ignore this parameter.
Change-Id: Idd00286b2577921b3fe5b55e8f13a27f2fbde5d6
node_list takes an argument "detail" which adds a rather arbitrary
list of results to the output. This comes from the command-line,
where we're trying to keep width under a certain length; but doesn't
make as much sense here (especially for json).
For dashboard type applications, replace this with a simple "fields"
parameter which, if set, will only return those fields it sees in the
common text output function.
Note, this purposely doesn't apply to the JSON output, as it expected
client-side filtering is more appropriate there. We could also add
generic field support to the command-line tools, if considered
worthwhile.
Add some documentation on all the end-points, and add info about these
parameters.
Change-Id: Ifbf1019b77368124961e7aa28dae403cabe50de1
All the _list functions return the same thing; save the results and
use a common output function to generate the json or text output.
Change-Id: I9cb44b09de2cb948e7381ef10302b21040433a2c
This command will display all ZooKeeper data for a given provider,
and provide and option to remove all of the data from ZooKeeper.
This can be useful when an operator must permanently remove a
pre-existing provider from nodepool and cannot cleanly shutdown
the services otherwise.
Example:
nodepool info rax
nodepool info --erase rax
Change-Id: I527aae5ff89aac864f984af050abb83e7bc3ac04
The pep8 rules used in nodepool are somewhat broken. In preparation to
use the pep8 ruleset from zuul we need to fix the findings upfront.
Change-Id: I9fb2a80db7671c590cdb8effbd1a1102aaa3aff8
This command has lost its usefulness in v3. Leaked instances
are automatically cleaned up by the CleanupWorker thread.
Change-Id: I99dced6c655fe865012d0d54f39bfc16b789d1a2
When execute command "nodepool" without any argument,
an internal error TypeError and traceback info
will be raised out now. Output help message to instead.
Change-Id: If96761fd2342c76a2c69a5bb16bd526279502ac0
If we issue a 'nodepool delete' command for a node whose provider
is not defined on the current launcher, we would get an error
trying to pull up that provider info. Only pull that provider info
when using the --now option.
Change-Id: Ibe05b93b3c8a2b9105261c4df01c9c6a43352f9e
The nodepool secure file is currently not being used for anything. By
having a default value in the nodepool command the config will try to
load this file. When the file then doesn't exist the program will exit.
This basically means you have to have an empty file at
/etc/nodepool/secure.conf to be able to use nodepoolcmd. Launcher does
not specify a default for secure, so follow that lead and remove the
default.
Change-Id: I244f097bcd1443588c03257394c5e8768ab3e785
Signed-off-by: Jamie Lennox <jamielennox@gmail.com>
The output of the 'list' command has gotten quite large. Simplify
the default output, and output the complete info when the --detail
option is given.
Also adds/modifies the list_nodes tests to assert that the column
output changes when the new option is used.
Change-Id: I60f849225c8c7fce1c524f132e54da58b25ae752
This change moves OpenStack related code to a driver. To avoid circular
import, this change also moves the StatsReporter to the stats module so that
the handlers doesn't have to import the launcher.
Change-Id: I319ce8780aa7e81b079c3f31d546b89eca6cf5f4
Story: 2001044
Task: 4614
This change adds a generic Provider meta class to the common
driver module to support multiple implementation. It also renames
some method to better match other drivers use-cases, e.g.:
* listServers into listNodes
* cleanupServer into cleanupNode
Change-Id: I6fab952db372312f12e57c6212f6ebde59a1a6b3
Story: 2001044
Task: 4612
str(e.message) is not valid for python3, but str(e) is.
Change-Id: Ifb3e13ee035371e0534c7bf5dd1587690f3dd6f5
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
For python3, dict.values() no longer is a list, but a view. So, we
need to explicitly enforce a list since we expect it for kazoo.
Change-Id: Ifa8d4101995b0933fc3f7e07cc78336edff598b9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The syntax for imports has changed for python3, lets use the new
syntax.
Change-Id: Ia985424bf23b44e492f51182179d2e476cdcccbb
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Since we are working towards python3 support, lets rename nodepool.py
to launcher.py to make relative imports nicer, otherwise we'd have to
use:
from . import foo
Change-Id: Ic38b6a8c2bf25d53625e159cb135b71d383b700c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Start working on low hanging fruit for python3 support.
Change-Id: I6d52c3a4ad4d383437925ecae6002de7528c9be0
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
So that an operator can choose to hold a node while it is in use,
update the 'nodepool hold' command to wait for the lock if that
is required. The locking algorithm is ordered, so as long as this
command is run while the node is locked while in use, this command
will be able to obtain the lock on the node before the node
deleter does.
This means it will sit in the foreground until the node is locked,
which, in some cases, may be hours. Perhaps that's not ideal,
but processes are easy to background in those cases, or abort if
the operator changes their mind. It does provide an avenue for
notification that the node has been unlocked by returning to the
shell when complete.
Change-Id: I89721cbe1e53b30bb0e2b5f5440870df958cf2c1
When using the CLI to place a hold on a node the return value of getNode
is not checked. This leads to an AttributeError on the None object.
Other commands simply return a message and hold should as well.
Change-Id: I11bdd47644c8535f99500989c63993e04215f464
This tries to capture common operation tasks in the documentation. It
also clears up some related items about what is necessary to have a
functioning Nodepool installation and what the dib-image-delete command
does.
Story: 2000790
Change-Id: I397fc4879fa84ffc667ddda0aff9c107eee0d694
James E. Blair