We still had some variations in how services stop. Finger, merger,
and scheduler all used signal.pause in a while loop which is
incompatible with stopping via the command socket (since we would
always restart the pause). Sending these components a stop or
graceful signal would cause them to wait forever.
Instead of using signal.pause, use the thread.join methods within
a while loop, and if we encounter a KeyboardInterrupt (C-c) during
the join, call our exit handler and retry the join loop.
This maintains the intent of the signal.pause loop (which is to
make C-c exit cleanly) while also being compatible with an internal
stop issued via the command socket.
The stop sequence is now unified across all components. The executor
has an additional complication in that it forks a process to handle
streaming. To keep a C-c shutdown clean, we also handle a keyboard
interrupt in the child process and use it to indicate the start of
a shutdown. In the main executor process, we now close the socket
which is used to keep the child running and then wait for the child
to exit before the main process exits (so that the child doesn't
keep running and emit a log line after the parent returns control
to the terminal).
Change-Id: I216b76d6aaf7ebd01fa8cca843f03fd7a3eea16d
This is a new reconfiguration command which behaves like full-reconfigure
but only for a single tenant. This can be useful after connection issues
with code hosting systems, or potentially with Zuul cache bugs.
Because this is the first command-socket command with an argument, some
command-socket infrastructure changes are necessary. Additionally, this
includes some minor changes to make the services more consistent around
socket commands.
Change-Id: Ib695ab8e7ae54790a0a0e4ac04fdad96d60ee0c9
We currently read the config in the fingergw app and put the config
into the fingergw object. This gets ugly when adding more config
options so move evaluation of the config file into the FingerGateway
class. This is a preparation for adding ssl support to the
FingerGateway which will need more config options.
Depends-On: https://review.opendev.org/663413
Change-Id: I83f3863586b85f8befd84eb8f6079fa35ee3a8cb
Currently, the ZooKeeper connection is initialized directly in the cmd
classes like zuul.cmd.scheduler or zuul.cmd.merger and then passed to
the server instance.
Although this makes it easy to reuse a single ZooKeeper connection for
multiple components in the tests it's not very realistic.
A better approach would be to initialize the connection directly in the
server classes so that each component has its own connection to
ZooKeeper.
Those classes already get all necessary parameters, so we could get rid
of the additional "zk_client" parameter.
Furthermore it would allow us to use a dedicated ZooKeeper connection
for each component in the tests which is more realistic than sharing a
single connection between all components.
Change-Id: I12260d43be0897321cf47ef0c722ccd74599d43d
The ZooKeeperClient now provides a fromConfig() method that parses all
necessary configuration values to instantiate a ZooKeeperClient.
Previously, this needed to be done in every component to initialize the
connection to ZooKeeper.
Change-Id: I5fa4ddab5f85c658291f1262ee0392a60086846e
Part of point 5 in https://etherpad.openstack.org/p/zuulv4
Connection is idle for now.
Also update component documentation.
Change-Id: I97a97f61940fab2a555c3651e78fa7a929e8ebfb
The commands are managed as entry-points so remove
ununecessary shebangs. Also lib/re2util.py does not
require a shebang as well.
zuul_return.py does not have a main and is not supposed
to be run directly.
Ununecessary shebangs for non executable script causes
rpmlint issues.
Change-Id: I6015daaa0fe35b6935fcbffca1907c01c9a26134
We're presuming, by default, that a user named "zuul" exists, which
is not the case in some environments. Change the default to avoid
dropping privileges, and require that this be explicitly set in order
to do so.
Change-Id: Ia677d2615dd9292a809df4c8859a60b7f4df6243
This reverts commit 00d7ea51fd. It
intended to refactor common code paths for signal handling. However in
our dockerized deployment this seems to completely break signal
handling. Thus it needs to be reverted.
Change-Id: Id5731557ff9a363c7a3d9438a8efcd476e38380c
Almost all zuul apps use the method term_handler for SIGINT and
SIGTERM. Defining this centrally in ZuulDaemonApp makes this much
simpler and without repitition.
Change-Id: I68f8d1bf52b0e16340818d2bcc44cd9fc5868ca7
This adds the zuul-fingergw app that should be run as root (so that
it can connect to the standard finger port 79), but changes user privs
immediately after binding that port.
Common streaming functions have been moved to streamer_utils.py to
be shared among modules.
Support for CommandSocket has been included.
Change-Id: Ia35492fe951e7b9367eeab0b145d96189d72c364
James E. Blair