This change, while substantial, is mostly organizational.
Currently, connections, sources, triggers, and reporters are
discrete concepts, and yet are related by virtue of the fact that
the ConnectionRegistry is used to instantiate each of them. The
method used to instantiate them is called "_getDriver", in
recognition that behind each "trigger", etc., which appears in
the config file, there is a class in the zuul.trigger hierarchy
implementing the driver for that trigger. Connections also
specify a "driver" in the config file.
In this change, we redefine a "driver" as a single class that
organizes related connections, sources, triggers and reporters.
The connection, source, trigger, and reporter interfaces still
exist. A driver class is responsible for indicating which of
those interfaces it supports and instantiating them when asked to
do so.
Zuul instantiates a single instance of each driver class it knows
about (currently hardcoded, but in the future, we will be able to
easily ask entrypoints for these). That instance will be
retained for the life of the Zuul server process.
When Zuul is (re-)configured, it asks the driver instances to
create new connection, source, trigger, reporter instances as
necessary. For instance, a user may specify a connection that
uses the "gerrit" driver, and the ConnectionRegistry would call
getConnection() on the Gerrit driver instance.
This is done for two reasons: first, it allows us to organize all
of the code related to interfacing with an external system
together. All of the existing connection, source, trigger, and
reporter classes are moved as follows:
zuul.connection.FOO -> zuul.driver.FOO.FOOconnection
zuul.source.FOO -> zuul.driver.FOO.FOOsource
zuul.trigger.FOO -> zuul.driver.FOO.FOOtrigger
zuul.reporter.FOO -> zuul.driver.FOO.FOOreporter
For instance, all of the code related to interfacing with Gerrit
is now is zuul.driver.gerrit.
Second, the addition of a single, long-lived object associated
with each of these systems allows us to better support some types
of interfaces. For instance, the Zuul trigger maintains a list
of events it is required to emit -- this list relates to a tenant
as a whole rather than individual pipelines or triggers. The
timer trigger maintains a single scheduler instance for all
tenants, but must be able to add or remove cron jobs based on an
individual tenant being reconfigured. The global driver instance
for each of these can be used to accomplish this.
As a result of using the driver interface to create new
connection, source, trigger and reporter instances, the
connection setup in ConnectionRegistry is much simpler, and can
easily be extended with entrypoints in the future.
The existing tests of connections, sources, triggers, and
reporters which only tested that they could be instantiated and
have names have been removed, as there are functional tests which
cover them.
Change-Id: Ib2f7297d81f7a003de48f799dc1b09e82d4894bc
The addFailTest method needed to be updated to catch up with some
test infrastructure changes. Rather than consulting the merger's
git repo for the ZUUL_REF to find out if a job which is supposed to
fail is present for a given build, use the new FakeBuild.hasChanges
method to inspect the launcher's prepared git repo.
Also rename the method as 'Job' is more descriptive than 'Test' and
document it.
Change-Id: I3224b8a01d49cfa06b799a8028c1bf0d455d25b1
Use the newer form of spelled-out category names in tests since
that's what we're writing in config files now.
Change-Id: Ib679e54b53131280956cbda20b84f5602a4953c8
When connections are set up in tests, multiple Gerrit connections which
are configured to connect to the same fake Gerrit server share a change
database so that changes sent to Gerrit via one connection are reflected
back to Zuul on another. They also share an event queue so that events
injected on one are seen by another.
Unfortunately, that part doesn't work, and in fact, events are only seen
by one of the gerrit connections. This happens to work since it doesn't
matter which gerrit connection receives an event, which is why we haven't
noticed the problem in tests.
Where we do see the problem in Zuulv3 is in shutting down the connections.
When a GerritConnection is stopped, a sentinal object (None) is added to
the event queue. When the GerritConnection gets an event from the queue,
it first checks whether it has stopped before processing that event.
Because in tests (but not in production) multiple GerritConnections share
an event queue, the connection that adds the None object to the queue
may not be the one that receives it, which causes the test to raise an
exception and not stop correctly.
We did not notice this in v2 because the order in which the Queue.Queue
class decides to awaken a thread is deterministic enough that the thread
which submitted the sentinel was always the one that received it. In
v3, the thread order is sufficiently different that the thread for the
*other* connection is reliably the one which receives it.
To correct this, stop using a shared queue between the differing
GerritConnection objects, and instead add a helper method to the testcase
class which will add an event to every connection for a given server.
Change-Id: Idd3238f5ab8f5e09e295c0fa028e140c089a2a3f
Test reporting back to gerrit as a different user.
Test multiple gerrit instances.
Squashed with: Assign fake connections to test class dynamically
As multiple connections can be configured for testing purposes, it makes
more sense to be able to access them all comfortably.
Drawback of this change is, that the dynamicism makes the code less
readable and less obvious.
Previously: If50567dd5a087e9fe46cd4e30f9e0562cda794d7
Co-Authored-By: Jan Hruban <jan.hruban@gooddata.com>
Change-Id: I4652799c8b9a626bd2f7f6996d09c95da2953936
This is a large refactor and as small as I could feasibly make it
while keeping the tests working. I'll do the documentation and
touch ups in the next commit to make digesting easier.
Change-Id: Iac5083996a183d1d8a9b6cb8f70836f7c39ee910
James E. Blair