We're seeing many test failures due to executors unregistering during
high system load and thus causing timeouts in test cases. During tests
we expect the system to be busy so increase the load_multiplier in
tests.
Change-Id: I54a05adc9e7cb9efaf20b70e59a59cefd44e21e9
On the way towards a fully scale out scheduler we need to move the
times database from the local filesystem into the SQL
database. Therefore we need to make at least one SQL connection
mandatory.
SQL reporters are required (an implied sql reporter is added to
every pipeline, explicit sql reporters ignored)
Change-Id: I30723f9b320b9f2937cc1d7ff3267519161bc380
Depends-On: https://review.opendev.org/621479
Story: 2007192
Task: 38329
This adds a tenant option to use the Zuul web build page as the
URL reported to the code review system when a build completes.
The setting is per-tenant (because it requires that the tenant
have a working SQL reporter configured in all pipelines) and
defaults to false, since we can't guarantee that. In the future,
we expect to make SQL reporting implicit, then this can default
to true and eventually be deprecated.
A new zuul.conf option is added and marked required to supply
the root web URL. As we perform further integration with the web
app, we may be able to deprecate other similar settings, such
as "status_url".
Change-Id: Iaa3be10525994722d020d2aa5a7dcf141f2404d9
Some options of zuul web should be initialized as int format,
and it may cause TypeError when accessing zuul web dashboard.
Change-Id: I4923d3047516c39b7e1005e189924fb4dd6fd1df
Add a relative_priority field to node requests and continuously
adjust it for each queue item based on the contents of queues.
This allows for a more fair distribution of build resources between
different projects. The first item in a pipeline from a given
project (or, in the case of a dependent pipeline, group of projects)
has equal priority to all other first-items of other projcets in
the same pipeline. Second items have a lower priority, etc.
Depends-On: https://review.openstack.org/620954
Change-Id: Id3799aeb2cec6d96a662bfa394a538050f7ea947
The automatic statsd configuration based on env variables has
proven cumbersome and counter-intuitive. Move its configuration
into zuul.conf in preparation for other components emitting stats.
Change-Id: I3f6b5010d31c05e295f3d70925cac8460d334283
This option was to maintain backwards compatability with the report
format used by the jenkins gerrit trigger plugin. We no longer
need to worry about that. With the addition of various reporters
which may need to report information in different formats, we
should probably either move this to the gerrit driver, or make it
customizable.
For now, let's go ahead and remove the option and use the behavior
we would have if it were set to true, which has been the case for
OpenStack infra's and all of our third-party ci users for some
time.
Change-Id: I8f040e03b7074cf59eced6fca38e6b221051d93e
This has been moved to the job definition (with the expectation
that it will be set by a site-local base job).
Change-Id: Ibdb5968a090c335d5a8449466599da38d321d55e
To avoid confusion with nodepool-launcher, we've decided to rename
zuul-launcher to zuul-executor.
Change-Id: I7d03cf0f0093400f4ba2e4beb1c92694224a3e8c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
In case a user runs a merger and a launcher on the same host, make
sure that they don't share a git directory (used by the launcher's
internal merger). They could end up colliding.
Incidentally, that's basically the configuraton used in tests, so
update the test configuration likewise.
Change-Id: I64a690c706d00583973bd2d542a5f42ae6e9ef36
Problem is that merge jobs triggered by timer don't supply
"refspec" event attribute, and, naturally, nothing is merged
but the repo is updated.
But, in such case, no connection info is passed to merger,
and GIT_SSH is not set.
So, the merge job will fail (perhaps, until some
other change events with "refspec" defined will
make this merger to set GIT_SSH properly)
Change-Id: I6dd96ae0bdcdf4c3db4c931a8c6ef7d5edfc1257
This is a large refactor and as small as I could feasibly make it
while keeping the tests working. I'll do the documentation and
touch ups in the next commit to make digesting easier.
Change-Id: Iac5083996a183d1d8a9b6cb8f70836f7c39ee910
Have zuul send signed credentials as part of the job for workers to
consume and upload assets to a defined location.
Zuul currently doesn't care about logs however this change will
suggest a LOG_PATH to builders as a unqiue destination prefix
allowing zuul to know preemptively the destination.
The workers are still required to send a URL of the final location.
Change-Id: I042cdd2dd2407f381cafcabc5c6b83d9b9a9eb00
Connect it to Zuul via Gearman. Any number of mergers may be
deployed.
Directly find the pipeline for a build when processing a result,
so that the procedure is roughly the same for build and merge
results.
The timer trigger currently requires the gerrit trigger also be
configured. Make that explicit inside of the timer trigger so
that the scheduler API interaction with triggers is cleaner.
Change-Id: I69498813764753c97c426e42d17596c2ef1d87cf
This is in preparation to move to separate merger process, which
will not have access to the trigger. Similar functionality can
be obtained using the new replication feature, and docs have been
updated to that effect.
Change-Id: I9397f7eb8466af464c8e7adb02d0a8d3eff04f9f
This patch adds the zuul_url option in zuul conf file
in order to pass ZUUL_URL to Jenkins, which will be used
by devstack-vm-gate-wrap.sh.
Documentation added in launchers.rst and zuul.rst
explaining how this new option works.
Change-Id: I840423cc06fdfdacd301d30be3e0b3e589e563e9
Utilises the new reporter plugin architecture to add support for
emailing success/failure messages based on layout.yaml.
This will assist in testing new gates as currently after a job has
finished if no report is sent back to gerrit then only the workers
logs can be consulted to see if it was successful. This will allow
developers to see exactly what zuul will return if they turn on
gerrit reporting.
Change-Id: I47ac038bbdffb0a0c75f8e63ff6978fd4b4d0a52
It is possible the host system does not have git properly configured,
which results in merge failures because the git client is complain. For
example:
GitCommandError: 'git merge FETCH_HEAD' returned exit status 128:
*** Please tell me who you are.
Run
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
Now we can pass user.name and user.email settings to git, if configured
to do so.
Change-Id: I896194d8d1f5334026954b02f3a1a8dd82bed2ac
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/29015
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Remove the Jenkins launcher and add a new Gearman launcher (designed
to be compatible with Jenkins) in its place.
See the documentation for how to set up the Gearman Plugin for
Jenkins.
Change-Id: Ie7224396271d7375f4ea42eebb57f883bc291738
If url_pattern is set in zuul.conf, use it instead of the build
url supplied by jenkins when reporting a change to gerrit.
Change-Id: Ic66b1efb5c7eae0dbbfd1927ee2827d13729b568
Reviewed-on: https://review.openstack.org/11908
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
And the default -- so that Zuul doesn't need any special gerrit
privileges to run out of the box.
Change-Id: I36411b299c8363d509498ece8ea21f88665ca39a
Reviewed-on: https://review.openstack.org/11589
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Add git repo management and merging. When collecting changes to
be tested together, merge or cherry-pick those changes into the
zuul-managed repos, and create a unique ref for that configuration.
Pass the ref to Jenkins instead of the string description of the
changes, so that Jenkins only needs to checkout that one ref.
This moves the complexity of merging and managing multiple commits
out of Jenkins and into Zuul.
The GERRIT_CHANGES variable is deprecated (along with the rest of
the GERRIT_* variables) and will be removed in a future patch
(which will contain a documentation update).
Change-Id: I126c9030223c07a30f7092e2273ebd7605d9f3df
Reviewed-on: https://review.openstack.org/11349
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Add support for testing, and some initial functional tests.
Some threads in Zuul now have proper shutdown methods, since
that's needed for testing. The threads that are faked in the
testing infrastructure still lack shutdown methods.
Fix some bugs uncovered by tests:
Reverse the order of commits that are given to Jenkins (the
commits that preceded the one being tested were listed in
revers order, this corrects that).
Return 0 in reportChange if the change has already been
reported. The old return value of True was misinterpreted
as an error.
Change-Id: I4979d0efd8581061a2b856b892d4ecdb75284a1b
Reviewed-on: https://review.openstack.org/10572
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
James E. Blair