Freze Zuul job variables when starting a build so that jinja
templates can not be used to expose secrets. The values will be
frozen by running a playbook with set_fact, and that playbook
will run without access to secrets. After the playbook
completes, the frozen variables are read from and then removed
from the fact cache. They are then supplied as normal inventory
variables for any trusted playbooks or playbooks with secrets.
The regular un-frozen variables are used for all other untrusted
playbooks.
Extra-vars are now only used to establish precedence among all
Zuul job variables. They are no longer passed to Ansible with
the "-e" command line option, as that level of precedence could
also be used to obtain secrets.
Much of this work is accomplished by "squashing" all of the Zuul
job, host, group, and extra variables into a flat structure for
each host in the inventory. This means that much of the variable
precedence is now handled by Zuul, which then gives Ansible
variables as host vars. The actual inventory files will be much
more verbose now, since each host will have a copy of every "all"
value. But this allows the freezing process to be much simpler.
When writing the inventory for the setup playbook, we now use the
!unsafe YAML tag which is understood by Ansible to indicate that
it should not perform jinja templating on variables. This may
help to avoid any mischief with templated variables since they
have not yet been frozen.
Also, be more strict about what characters are allowed in ansible
variable names. We already checked job variables, but we didn't
verify that secret names/aliases met the ansible variable
requirements. A check is added for that (and a unit test that
relied on the erroneous behavior is updated).
Story: 2008664
Story: 2008682
Change-Id: I04d8b822fda6628e87a4a57dc368f20d84ae5ea9
It is possible to circumvent the use of `allowed-projects` in
untrusted projects by creating a change which `Depends-On` a
change which alters a project definition. This behavior may be
unexpected, so documentation has been updated with warnings to
avoid relying on it in sensitive cases.
It may have been possible to expose a secret, or use resources
protected by a secret, if a job using a secret was defined in an
untrusted project on a system with an independent pre-merge
post-review pipeline -- that is, a pipeline with `post-review` set
to true, `manager` set to `independent`, and which operated on
changes before they merged.
To prevent disclosure or use in this situation, `allowed-projects`
is now automatically set to the current project when a secret is
used in a job defined in an untrusted project, and it can not be
overridden.
The test_trusted_secret_inheritance_gate test is removed because
it only tested that jobs with secrets in an untrusted repo were
able to run in a trusted repo. That is no longer possible.
Change-Id: I77f6a011bca08a2433137dc29597b7cc2757adb1
Story: 2004837
Task: 29037
Adjust the secret-inheritance test so that it verifies secrets
on base jobs are available.
Change-Id: Ifad128213ad95b5b30ffdc1be02c46c48f7c1c46
Co-Authored-By: Tristan Cacqueray <tdecacqu@redhat.com>
Move this into configuration files so that we can test the
functionality end-to-end rather than relying on internal APIs
which are frequently changing.
Change-Id: If1f75cf332732af31386e597b607e45253ecee1f
James E. Blair