summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
3 hoursMerge "Don't exclude config if excluded in earlier tenant"HEADmasterZuul
11 hoursAllow extra data in artifact schema validationJames E. Blair
The schema validator for zuul.artifacts returned via zuul_return did not allow for extra return data (outside of the zuul hierarchy). Correct the validator to allow that, and update the test to exercise it. Change-Id: Ia8d7ed212b03d978bec1a7de1a2647b0c87b7cf9 Notes (review): Code-Review+2: Clark Boylan <cboylan@sapwetik.org> Code-Review+2: Jeremy Stanley <fungi@yuggoth.org> Workflow+1: Jeremy Stanley <fungi@yuggoth.org> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 19 Feb 2019 23:33:56 +0000 Reviewed-on: https://review.openstack.org/638038 Project: openstack-infra/zuul Branch: refs/heads/master
12 hoursMerge "Combine zuul.artifacts lists in zuul_return"Zuul
12 hoursMerge "Switch artifact return back to list"Zuul
13 hoursCombine zuul.artifacts lists in zuul_returnJames E. Blair
In order to make it simple for multiple independent playbooks to return artifacts, append the values of zuul.artifacts in zuul_return rather than the usual method of overwriting lists. Change-Id: I09e6076b4bb354023c5414d149d9bfa59fb3ea4b Notes (review): Code-Review+2: Clark Boylan <cboylan@sapwetik.org> Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: Jeremy Stanley <fungi@yuggoth.org> Workflow+1: Jeremy Stanley <fungi@yuggoth.org> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 19 Feb 2019 21:56:50 +0000 Reviewed-on: https://review.openstack.org/638005 Project: openstack-infra/zuul Branch: refs/heads/master
15 hoursSwitch artifact return back to listJames E. Blair
This is a partial revert of f12453f6cbb452451bcce0cef9c4da4b957fd4a5. The use case that change was designed to address is poorly served by that change. The intent was to make it easier to return multiple artifacts in multiple playbooks independently by relying on the dictionary merge behavior of zuul_return. However, in the entirely likely case of artifacts with generated names, it becomes difficult because Ansible does not run jinja on dictionary keys. Therefore, revert to the previous list behavior. A subsequent change will add a feature to zuul_return to address the underlying issue of returning multiple artifacts from different playbooks. Change-Id: I0581aa68fcef320ab27c11ddd6338a15eef38ceb Notes (review): Code-Review+2: Monty Taylor <mordred@inaugust.com> Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: Jeremy Stanley <fungi@yuggoth.org> Workflow+1: Jeremy Stanley <fungi@yuggoth.org> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 19 Feb 2019 21:49:55 +0000 Reviewed-on: https://review.openstack.org/637979 Project: openstack-infra/zuul Branch: refs/heads/master
15 hoursMerge "Clarify project vs repository in getPullBySha"Zuul
15 hoursMerge "Switch to LRU based sha to PR cache"Zuul
15 hoursMerge "Test GithubShaCache"Zuul
15 hoursMerge "Rename project to project_name in getPullBySha"Zuul
16 hoursMerge "Add dockerized test setup"Zuul
16 hoursClarify project vs repository in getPullByShaClark Boylan
There continues to be some slightly confusing var terminology in getPullBySha. Rename the project_name suffix variable to repository from project to match the github3 term. Change-Id: I5e34fcb80248532f97b470d7f829dd5c4577e5f3 Notes (review): Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: Monty Taylor <mordred@inaugust.com> Workflow+1: Monty Taylor <mordred@inaugust.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 19 Feb 2019 18:55:43 +0000 Reviewed-on: https://review.openstack.org/637956 Project: openstack-infra/zuul Branch: refs/heads/master
16 hoursMerge "Don't request PR issue data"Zuul
16 hoursSwitch to LRU based sha to PR cacheClark Boylan
This updates the github driver to cache PRs by sha using cachetool's LRUCache. We make this change because we need to cache closed PRs so can't rely on the action of closing a PR to remove the PR from the cache. Since we don't have a good method of evicting entries we fall back to LRU with a reasonable cache size (2k commits). Change-Id: I5fb6c8b33f9eed221a8b84e537f02e7dccf2d2df Notes (review): Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: Monty Taylor <mordred@inaugust.com> Code-Review+2: James E. Blair <corvus@inaugust.com> Workflow+1: James E. Blair <corvus@inaugust.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 19 Feb 2019 18:51:37 +0000 Reviewed-on: https://review.openstack.org/637615 Project: openstack-infra/zuul Branch: refs/heads/master
17 hoursFix typo in build api endpointJames E. Blair
This corrects a copy-pasta typo from the previous loop. Change-Id: If692a716a69f0fb7c48cc4c79a1704ea2847eeec Notes (review): Code-Review+2: James E. Blair <corvus@inaugust.com> Workflow+1: James E. Blair <corvus@inaugust.com> Code-Review+2: Jeremy Stanley <fungi@yuggoth.org> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 19 Feb 2019 17:30:40 +0000 Reviewed-on: https://review.openstack.org/636227 Project: openstack-infra/zuul Branch: refs/heads/master
18 hoursTest GithubShaCacheClark Boylan
This adds a small test class for the GithubShaCache. Change-Id: I5d6763e10bf1f8228af038fcbfa7c2ad65778508 Notes (review): Code-Review+2: James E. Blair <corvus@inaugust.com> Code-Review+2: Jeremy Stanley <fungi@yuggoth.org> Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Workflow+1: Tobias Henkel <tobias.henkel@bmw.de> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 19 Feb 2019 18:51:35 +0000 Reviewed-on: https://review.openstack.org/637228 Project: openstack-infra/zuul Branch: refs/heads/master
18 hoursRename project to project_name in getPullByShaClark Boylan
This is a consistency change suggested by tobiash. When we implemented the PR cache work here we noticed there was inconsistency around var naming in this method compared to getPull which made things a bit confusing. Change-Id: I7460df6dbb93ee27e29950bf95ec25cf49e669be Notes (review): Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: Monty Taylor <mordred@inaugust.com> Workflow+1: Monty Taylor <mordred@inaugust.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 19 Feb 2019 18:51:34 +0000 Reviewed-on: https://review.openstack.org/637218 Project: openstack-infra/zuul Branch: refs/heads/master
33 hoursDon't request PR issue dataClark Boylan
The github api docs show that labels is now part of the PR dict returned in pulls API responses. github3 doesn't directly expose this as an attribute but we can use the as_dict representation to access any json field. Use this field on the PR dict repr to access the labels of a PR and stop fetching the issue of a PR for that info. Change-Id: Ib14bb387dbd233ff252cf57be7f0517770ade037 Notes (review): Code-Review+2: Jeremy Stanley <fungi@yuggoth.org> Code-Review+2: Monty Taylor <mordred@inaugust.com> Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Workflow+1: Tobias Henkel <tobias.henkel@bmw.de> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 19 Feb 2019 17:42:58 +0000 Reviewed-on: https://review.openstack.org/636728 Project: openstack-infra/zuul Branch: refs/heads/master
34 hoursMerge "Add comment about extra issues request"Zuul
34 hoursMerge "Re-use the github PR object when fetching reviews"Zuul
39 hoursMerge "connections: remove unused getSchema procedure"Zuul
39 hoursMerge "connections: always register the scheduler"Zuul
40 hoursMerge "web: prevent status update loop in background"Zuul
40 hoursMerge "Add spec for multi ansible version support"Zuul
43 hoursMerge "Make UnsafeTag self registering"Zuul
2 daysconnections: remove unused getSchema procedureTristan Cacqueray
This change removes unused code. Change-Id: Ic2fcf447b9a2e5c2e97749a5d1630e0ac91dd318 Notes (review): Code-Review+2: James E. Blair <corvus@inaugust.com> Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Workflow+1: Tobias Henkel <tobias.henkel@bmw.de> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Mon, 18 Feb 2019 18:58:02 +0000 Reviewed-on: https://review.openstack.org/637445 Project: openstack-infra/zuul Branch: refs/heads/master
2 daysconnections: always register the schedulerTristan Cacqueray
The Driver base class implements a noop registerScheduler function that can be called un-conditionally. Change-Id: I570f8b2f261bb07d81f99c216c19575ef8cc59ab Notes (review): Code-Review+2: James E. Blair <corvus@inaugust.com> Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Workflow+1: Tobias Henkel <tobias.henkel@bmw.de> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Mon, 18 Feb 2019 18:54:41 +0000 Reviewed-on: https://review.openstack.org/637444 Project: openstack-infra/zuul Branch: refs/heads/master
5 daysAdd dockerized test setupTobias Henkel
We currently have the test-setup.sh script that installs and configures the services that are needed to run the zuul test suite. However this needs mysql, postgres and zookeeper installed into the system. It also messes with tmpfs mounts of zookeeper. This adds a docker-compose based test setup that can ease the setup of the test requirements on a developer machine. Change-Id: Ib0e3f42898c6e9258625b03bcc363e91f4b7ee83 Notes (review): Code-Review+2: James E. Blair <corvus@inaugust.com> Code-Review+2: Clint 'SpamapS' Byrum <clint@fewbar.com> Workflow+1: Clint 'SpamapS' Byrum <clint@fewbar.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 19 Feb 2019 18:21:35 +0000 Reviewed-on: https://review.openstack.org/636424 Project: openstack-infra/zuul Branch: refs/heads/master
5 daysAdd comment about extra issues requestJames E. Blair
We identified a potential improvement to github3.py which, when complete, may be able to eliminate an extra github api call. Change-Id: Ic6a9daf3f0dd87655d08ad6aa42d89cd02c7225f Notes (review): Code-Review+2: Clark Boylan <cboylan@sapwetik.org> Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Workflow+1: Tobias Henkel <tobias.henkel@bmw.de> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Mon, 18 Feb 2019 23:36:19 +0000 Reviewed-on: https://review.openstack.org/636706 Project: openstack-infra/zuul Branch: refs/heads/master
5 daysRe-use the github PR object when fetching reviewsJames E. Blair
We inadvertently fetch the PR object twice because of the way we were fetching reviews. Keep it around so we can make one less API call. Co-Authored-By: Clark Boylan <cboylan@sapwetik.org> Change-Id: If5260278adb525566d99eedaecaf8b4f5077d43e Notes (review): Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: James E. Blair <corvus@inaugust.com> Code-Review+2: Clark Boylan <cboylan@sapwetik.org> Workflow+1: Clark Boylan <cboylan@sapwetik.org> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Mon, 18 Feb 2019 23:36:17 +0000 Reviewed-on: https://review.openstack.org/636705 Project: openstack-infra/zuul Branch: refs/heads/master
5 daysMerge "Cache github PR shas"Zuul
6 daysCache github PR shasJames E. Blair
Maintain a cache of sha->PR which we update every time we fetch a pull_request object from github. Use this in the status event handler to map the status event to relevant PRs. Change-Id: Ie811329429e1b672ce012767a475447f80832ee8 Notes (review): Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: James E. Blair <corvus@inaugust.com> Workflow+1: James E. Blair <corvus@inaugust.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Thu, 14 Feb 2019 22:42:56 +0000 Reviewed-on: https://review.openstack.org/636764 Project: openstack-infra/zuul Branch: refs/heads/master
6 daysMerge "Mark as unsafe commit message at inventory"Zuul
6 daysMake UnsafeTag self registeringTobias Henkel
YAMLObject classes can self-register themselves in the yaml dumper and loader via metaclass magic. Use it so we don't execute statements during import. Change-Id: I31f097701f863e3f2298028d76054e80dfc2d23e Notes (review): Code-Review+2: James E. Blair <corvus@inaugust.com> Code-Review+2: Paul Belanger <pabelanger@redhat.com> Workflow+1: Paul Belanger <pabelanger@redhat.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Mon, 18 Feb 2019 15:20:50 +0000 Reviewed-on: https://review.openstack.org/637023 Project: openstack-infra/zuul Branch: refs/heads/master
7 daysMark as unsafe commit message at inventoryQuique Llorente
If you run zuul at a commit with some jinja2 stuff in the comment it fails, to bypass this this review tag the inventory yaml zuul message with !unsafe ansible yaml tag [1]. Closes-Bug: https://storyboard.openstack.org/#!/story/2004896 [1] https://docs.ansible.com/ansible/latest/user_guide/playbooks_advanced_syntax.html#unsafe-or-raw-strings Change-Id: Ic11c253cf23cc4d1fb80993f5722f37e4c22f6df Notes (review): Code-Review+2: Jeremy Stanley <fungi@yuggoth.org> Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: James E. Blair <corvus@inaugust.com> Workflow+1: James E. Blair <corvus@inaugust.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Thu, 14 Feb 2019 18:32:37 +0000 Reviewed-on: https://review.openstack.org/633930 Project: openstack-infra/zuul Branch: refs/heads/master
7 daysweb: remove build and job_name filter from the buildset routeTristan Cacqueray
This change fixes the buildset route limit which incorrectly match the number of build instead of the number of buildset. Change-Id: Ia5775a32e646a7107a6dd04328ac36ec681b3b50 Notes (review): Code-Review+2: Clint 'SpamapS' Byrum <clint@fewbar.com> Code-Review+2: James E. Blair <corvus@inaugust.com> Workflow+1: James E. Blair <corvus@inaugust.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Thu, 14 Feb 2019 00:09:11 +0000 Reviewed-on: https://review.openstack.org/636504 Project: openstack-infra/zuul Branch: refs/heads/master
8 daysImprove event logging in githubconnectionJames E. Blair
This moves event processing to its own class, so that it's easier to bundle all of the data related to an event along with an event-specific logger. This logs the delivery ID for every line when we're preparing the event. It also logs the start time and queue length as well as the end time, even on error. Change-Id: I941a74ecbdb418cf94537ca9f8f1917a5e38dd33 Notes (review): Code-Review+2: Monty Taylor <mordred@inaugust.com> Code-Review+2: James E. Blair <corvus@inaugust.com> Workflow+1: James E. Blair <corvus@inaugust.com> Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Workflow+1: Tobias Henkel <tobias.henkel@bmw.de> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 12 Feb 2019 22:02:03 +0000 Reviewed-on: https://review.openstack.org/636429 Project: openstack-infra/zuul Branch: refs/heads/master
8 daysweb: prevent status update loop in backgroundTristan Cacqueray
When leaving the status page while it is waiting for status data, the callback happens when the status page is not mounted, resulting in the update loop timer not being cleared. This change prevents that by setting the visible attribute to false when leaving the page, so that the update is not scheduled when the page is not visible. Change-Id: I9f3db68bb77c52d0f477b8322bbf52bd3cc81fe1 Notes (review): Code-Review+2: Clint 'SpamapS' Byrum <clint@fewbar.com> Code-Review+2: James E. Blair <corvus@inaugust.com> Workflow+1: James E. Blair <corvus@inaugust.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Mon, 18 Feb 2019 18:18:33 +0000 Reviewed-on: https://review.openstack.org/636343 Project: openstack-infra/zuul Branch: refs/heads/master
8 daysMerge "Update SQLAlchemy url in docs"Zuul
8 daysMerge "web: add /{tenant}/buildsets route"Zuul
8 daysMerge "Add a note about sqlalchemy metadata"Zuul
9 daysAdd a note about sqlalchemy metadataJames E. Blair
It's not obvious why we're renaming the attribute here. Leave a clue for later explorers. Also, make test_artifacts handle artifacts being returned in a different order. Change-Id: Iffa216105f7e4951ea57fc073dacaa4040f82040 Notes (review): Code-Review+2: Clark Boylan <cboylan@sapwetik.org> Code-Review+2: Monty Taylor <mordred@inaugust.com> Workflow+1: Monty Taylor <mordred@inaugust.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 12 Feb 2019 01:18:43 +0000 Reviewed-on: https://review.openstack.org/636189 Project: openstack-infra/zuul Branch: refs/heads/master
9 daysMerge "Return artifacts as dicts and add metadata"Zuul
9 daysDon't exclude config if excluded in earlier tenantTobias Henkel
Consider the following scenario where tenant-one includes org/project1 without any config. Zuul currently doesn't load the config of org/project1 in tenant-two as well. - tenant: name: tenant-one source: gerrit: untrusted-projects: - org/project1: include: [] - tenant: name: tenant-two source: gerrit: untrusted-projects: - org/project1 This is caused because we cache an empty config if we exclude all config from the repo the same way as if the repo wouldn't have any config. This causes the cat job to be skipped when loading tenant-two. This can be solved by marking the empty config that will be cached as 'this has no config because it has been skipped'. Then we can detect that case and still trigger a cat job if we need the config in a later tenant. Change-Id: I1e7e68f11ca4fb7d1e52eee6d4c9a9d8331b012c Notes (review): Code-Review+2: James E. Blair <corvus@inaugust.com> Code-Review+2: Joshua Hesketh <josh@nitrotech.org> Workflow+1: Tobias Henkel <tobias.henkel@bmw.de> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Wed, 20 Feb 2019 06:41:03 +0000 Reviewed-on: https://review.openstack.org/636146 Project: openstack-infra/zuul Branch: refs/heads/master
12 daysReturn artifacts as dicts and add metadataJames E. Blair
A recent attempt to use the artifact return feature of zuul_return exposed some rough edges. These two changes should make it much easier to use. First, return artifacts as a dictionary instead of a list. This requires that they have unique names (which is no bad thing -- what would two artifacts named "docs" mean anyway?). But mainly it allows the dict merging behavior of zuul_return to be used, so that one playbook may use zuul_return with some artifacts, and another playbook may do the same, without either needing to load in the values of the other first (assuming, of course, that they use different artifact names). Second, add a metadata field. In the database and API, this is JSON serialized, but in zuul_return and zuul.artifacts, it is exploded into separate fields. This lets jobs do things like associate versions or tags with artifacts without having to abuse the url field. Change-Id: I228687c1bd1c74ebc33b088ffd43f30c7309990d Notes (review): Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: Clark Boylan <cboylan@sapwetik.org> Workflow+1: Clark Boylan <cboylan@sapwetik.org> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Mon, 11 Feb 2019 18:07:11 +0000 Reviewed-on: https://review.openstack.org/634825 Project: openstack-infra/zuul Branch: refs/heads/master
13 daysweb: add /{tenant}/buildsets routeTristan Cacqueray
This change adds a /buildsets route to return the buildsets. Change-Id: I0f951267e4531c05ec31d1af62dc56ea0f96257f Notes (review): Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: James E. Blair <corvus@inaugust.com> Workflow+1: James E. Blair <corvus@inaugust.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 12 Feb 2019 02:12:44 +0000 Reviewed-on: https://review.openstack.org/630035 Project: openstack-infra/zuul Branch: refs/heads/master
14 daysUpdate git connection loggingJames E. Blair
Put the logging under the zuul hierarchy, and also add the ref to the event representation. Both of these changes are in aid of better visibility of git events. Change-Id: Iac00be7536ad236e366cd793366ac6819357c17c Notes (review): Code-Review+2: Monty Taylor <mordred@inaugust.com> Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Workflow+1: Tobias Henkel <tobias.henkel@bmw.de> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Thu, 07 Feb 2019 19:29:51 +0000 Reviewed-on: https://review.openstack.org/635204 Project: openstack-infra/zuul Branch: refs/heads/master
14 daysMerge "Fix error reporting for special task failures"Zuul
14 daysMerge "Retry jobs on winrm send_input failed"Zuul
14 daysUpdate SQLAlchemy url in docsMonty Taylor
The existing url does not work. Change-Id: Ie5bef54be0e4122a10d0042c2ee2f5328f68d043 Notes (review): Code-Review+2: Tobias Henkel <tobias.henkel@bmw.de> Code-Review+2: James E. Blair <corvus@inaugust.com> Workflow+1: James E. Blair <corvus@inaugust.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 12 Feb 2019 05:59:28 +0000 Reviewed-on: https://review.openstack.org/635148 Project: openstack-infra/zuul Branch: refs/heads/master