The prepare_workspace_sync_required_projects_only variable allows
users to define which projects to sync to the node. This can prevent
syncing of unnecessary repositories. For some builds e.g. the
depends-on repositories dont need to be synced. The projects are
filtered based on the 'required' flag present in each zuul.project
entry and the required projects list also does not contain projects
which are present due to Depends-On or gate queue sequencing.
Having unnecessary repos in the workspace can for example also break
the analysis phase of bazel.
Change-Id: I3cc36cbfc60c81956caf5137da63973aeade4e21
Co-Authored-By: James E. Blair <jim@acmegating.com>
Co-Authored-By: Bernhard Berg <bernhard.berg@bearingpoint.com>
This change is preparation for https://review.opendev.org/c/zuul/zuul-jobs/+/887917
In the beginning, there was only prepare-workspace[0] which rsynced repos.
Then we added mirror-workspace-git[1] to make it more efficient by using git operation, but it required some openstack-specific code in project-config to work.
Then we added prepare-workspace-git[2] which completed the git-based sync solution by locating everything requried in zuul-jobs. It used mirror-workspace-git by reference and added this TODO:
# TODO(tobiash): we might want to deprecate the role mirror-workspace-git-repos
# and move it here.
This change completes that TODO by moving the mirror-workspace-git-repos code into prepare-workspace-git and places the repo in a sensible and maintainable state with two simple and good options:
* prepare-workspace (rsync)
* prepare-workspace-git (git)
In the unlikely event anyone is still using mirror-workspace-git-repos standalone (OpenStack/OpenDev is not, and that solution was haphazard as described above) they would be well served by a notification that there is a better alternative which is what most of the community actually uses now.
[0] cfffd4431b
[1] 348598e96a
[2] 7cee7156bc
Change-Id: Ib80e0447d49363182fd0d4c4d4e269841bc3aa95
Use zuul_workspace_root as the destination folder for preparing and
mirroring the workspace, which points to ansible_user_dir by default.
(As is currently the case).
The override of this variable can be useful when using a static node
with multiple slots to use a separate workspace for each slot.
Change-Id: If50abc7ccc2dfb6f4ea9f9d5c6a5091ff6967aef
Currently the incremental push of the git repos to the nodes is split
into a part in project-config and a part in zuul-jobs and only works
within the OpenStack context. The prepare-workspace-git role is an
attempt to make this generically available in zuul-jobs.
It first checks for each needed project if it's cached. If yes it's
cloned to the destination, if not it's just initialized as an empty
git repo. After that we can generically use git operations to sync the
correct state of the repos to the destination.
Change-Id: I87938a02d51e561b25ca2cb20f53d62f3cd3ae73
Lukas Kranz