Commit Graph

32 Commits

Author SHA1 Message Date
James E. Blair 509880073e Remove command.warn usage
This is no longer present in Ansible 9.

Removing these upsets ansible-lint, so those errors are ignored.

The base roles job has bitrotted on centos-7 and bionic due to
a bad voluptuous release used in an stestr test.  That is fixed in
this change as well.

Change-Id: I67886d5ad82ab590979f82bd102d6f974b9d4421
2024-02-14 15:17:52 -08:00
Ian Wienand 8c6512107c
Standarise block/when ordering
Newer ansbile-lint finds "when" or "become" statements that are at the
end of blocks.  Ordering these before the block seems like a very
logical thing to do, as we read from top-to-bottom so it's good to see
if the block will execute or not.

This is a no-op, and just moves the places the newer linter found.

Change-Id: If4d1dc4343ea2575c64510e1829c3fe02d6c273f
2022-11-07 10:37:53 +11:00
Ian Wienand 6d23d20f2f linters: add names to blocks
This is preparation for a later version of ansbile-lint, which finds
missing names on blocks.  This seems a reasonable rule, and the
Ansible manual says [1]

  Names for blocks have been available since Ansible 2.3. We recommend
  using names in all tasks, within blocks or elsewhere, for better
  visibility into the tasks being executed when you run the playbook.

This simply adds a name tag for blocks that are missing it.  This
should have no operational change, but allows us to update the linter
in a follow-on change.

[1] https://docs.ansible.com/ansible/latest/user_guide/playbooks_blocks.html

Change-Id: I92ed4616775650aced352bc9088a07e919f1a25f
2022-07-27 17:13:39 +10:00
Ian Wienand f76cfbab11 linters: fix spaces between filters
This fixes a number of places where we do not have spaces between
filters.  I think that this is a reasonable rule for readability (I
also think it probably was enforced, but maybe later versions got
better at detecting it?).

These are detected by a later version of Ansible lint; this change
should have no operational change to any roles but prepares us to
update in a follow-on change.

Change-Id: I07e1a109b87adce86f483d14d7e02fcecb8313d5
2022-07-27 17:13:39 +10:00
Sorin Sbarnea 6b8cc6d468 More E208 (final)
Change-Id: I705d1b10696326f3d4d5bef4b5a88a83f2c3d960
2020-11-10 19:34:45 +00:00
Ian Wienand 6f43acde41 fetch-sphinx-tarball: don't run merge-output-to-logs
The merge-output-to-logs role is not doing anything in this test.  As
described inline, this merges files in
zuul.executor.work_root/<docs,artifacts> into the
zuul.executor.log_root directory so they are available in change
results.

Since this job doesn't publish anything there, this role is unused.

merge-output-to-logs currently can't run because it tries to run shell
scripts on the executor.  Thus we can remove this unused role and
restore the job.

Change-Id: I1afc905aa8d9c420bed316e99760ad7ad1d838ce
2020-10-15 14:44:03 +11:00
Albin Vass 28b4fb43ef fetch-sphinx-tarball: do not synchronize owner
Change-Id: I0784cb37f29aa66149df470482cc75dcd028474c
2020-05-18 17:05:32 +02:00
Tristan Cacqueray 6f940da935 fetch-sphinx-tarball: introduce zuul_use_fetch_output
This change enables using fetch-sphinx-tarball role along with
the fetch-output role. By default the role still synchronizes
artifacts back to the executor.

Change-Id: I7452f34bccdca49e256053f9630f77953b27f99c
2020-05-14 12:03:58 +00:00
Tristan Cacqueray 6ac719ba2f fetch-sphinx-tarball: add missing zuul_success default
This change prevents an issue in test-jobs where zuul_success is undefined.

Change-Id: I408e9f957ff3151d39f7b5b2ea038255570baf4d
2020-05-12 16:54:32 +00:00
Zuul ff77dd22a1 Merge "Enable yamllint" 2020-05-05 08:46:58 +00:00
Sorin Sbarnea 33461bbecc Enable yamllint
Adds yamllint to the linters with a minimal configuration, some
rules are disabled to allow us to fix them in follow-ups, if
we agree on them.

Fixes invalid YAML file containing characters inside block.

Fixes few minor linting issues.

Change-Id: I936fe2c997597972d884c5fc62655d28e8aaf8c5
2020-05-04 17:47:11 +01:00
Andreas Jaeger c2fba020e3 Fix fetch-sphinx-tarball fails
I added previously a check that makes docs job fail in post if there's no
html output. This now leads to a POST_FAILURE when the normal build fails.

So, a FAILURE turned into POST_FAILURE, see
https://zuul.opendev.org/t/zuul/build/afbc8c39929a46039aa4d987971cdb4f .

Update the check in the fetch-sphinx-tarball role that it only fails
if the playbooks passed.

Change-Id: Iebd0bffcc5caaa6df97d432a712d83b64db8e780
2020-05-04 10:09:47 +02:00
Albin Vass 0c7673054a Use zj_sphinx_pdf instead of item
Change-Id: I8ed9c0e1ebb8cf43335ee49f11a320ddabb29eb5
2020-05-01 23:06:14 +02:00
Clark Boylan 1075767b18 Fix some item usage where var is zj_* now
In the fetch-sphinx-tarball role our handling of pdfs didn't fully
migrate the loop control var 'item' renaming. Update to replace now
incorrect 'item' with correct 'zj_' prefixed var names.

Change-Id: Ia5cb68ccbc410d88349e9dd6e3fc2551d5eff5e2
2020-05-01 12:08:49 -07:00
Zuul f38704dd6c Merge "fetch-sphinx-tarbal: allow to follow symlinks" 2020-05-01 16:08:58 +00:00
Sorin Sbarnea 27f9df6a8b fetch-sphinx-tarbal: allow to follow symlinks
Fix bug which prevented find from finding builds when the
folder was a symlink. This can happen if a project using a different
build layout tries to expose the docs at the expected location.

Change-Id: Ib7782b60cef42d2e9fd3077791f06b0f918d7c45
2020-05-01 16:13:00 +01:00
Albin Vass 9062289151 Check for loop_control in with_ type loops
Change-Id: I191265df7709a6262b44a428d78fe28ffaeb4b75
2020-05-01 13:45:34 +02:00
Zuul fd91b63c64 Merge "fetch-sphinx-tarball: Do not keep owner of archived files" 2020-04-27 19:20:04 +00:00
Zuul aac9dc0289 Merge "fetch-sphinx-tarball: use remote_src true" 2020-04-27 19:09:20 +00:00
Sorin Sbarnea b906d000fe Add testing of fetch-sphinx-tarball role
We discovered that role fails on some platform because of the missing
bzip2 package.  We can avoid this by using gzip which is much more
likely to exist on machines (and exists in the container images for
most operating systems).

Change-Id: Ia33a0f77343df809cd3c591a4a988e8e3712c133
Co-Authored-By: Mohammed Naser <mnaser@vexxhost.com>
2020-04-23 13:52:48 +01:00
vass a6d211cd21 fetch-sphinx-tarball: Do not keep owner of archived files
Since we cannot create users this causes an error if the user is not
the same on both the host that produced the files and the executor.

Change-Id: I74464033001666c82ccfa702799e9236029df816
2020-04-20 14:02:02 +02:00
vass bfcb8a9df3 fetch-sphinx-tarball: use remote_src true
Since ansible copies files as a tempfile to the remote host by default
setting remote_src: true when the files are on the same host gives better
error messages since otherwise the source path is not reflecting the real path.

See: http://paste.openstack.org/show/792400/
Where the file in "dest:" field of unarchive is copied to a tempfile
instead of just unarchiving the tarball directly.

Change-Id: Id7478a5a3171fdd2d3bc7531e66a09414e711d3a
2020-04-20 13:28:04 +02:00
Andreas Jaeger 4f25b964f9 Fail fetch-sphinx-tarball if no html exists
If no html is generated but a PDF, the role will succeed. This is wrong,
we always need HTML for the publish step where the logic is: Publish
HTML and optionally PDF.

I run into this when updating tox.ini and breaking html output, the jobs
passed but promote job failed.

Change-Id: I44d15da4c8e1ec0d1f7284daed69dbf5caa40bf0
2020-04-20 12:33:00 +02:00
Andreas Jaeger 5eb4c257f5 fetch-sphinx: Exclude doctrees directory
The sphinx .doctrees directory is not needed for publishing. By default
sphinx puts it below html. Some projects add to sphinx-build a "-d"
option to put the content elsewhere and not publish it.

This change assures that a .doctrees directory below "html" will not put
stored as artifact and published later.

Statistics: In all of OpenDev, we have:
* 511 repos putting .doctrees elsewhere and thus not publishing it
* 631 repos  no caring and thus publishing .doctrees

The size of .doctrees directory is not neglectable: 2MB for
system-config, 14 MB for nova.

Change-Id: I63b581164aab84352d9e80278cf4a8d634ff28af
2020-01-21 08:32:26 +01:00
Andreas Jaeger 0ca632b19d Switch PDF file name to doc-PROJECT.pdf
Further work on the PDF generation showed that many projects have
project logos as svg files with the name PROJECT.svg, those get
converted to PROJECT.pdf and collide with the default name used here.

Change the default to "doc-PROJECT.pdf" to have a unique name.

This is a change in the default but we only added this 11 days ago, so
I hope it's fine to change.

Change-Id: I4f2224633fd4edbc3121af53d0202796cf8a0a60
2019-09-03 14:40:39 +02:00
Andreas Jaeger c74b03649a Switch PDF fetching to known files
Some repos produce intermediate PDF files, for example from svg files,
and thus we have multiple PDF files.

Change the PDF fetching to a list of pdf_files which can be supplied,
the default is the name of the repos with ".pdf" attached, like
"octavia.pdf".

https://review.opendev.org/#/c/667249/3 illustrates the problem.

Change-Id: I5d3b6d6c2884ba66bdd35639b023d6d76f64ac01
2019-08-24 12:24:07 +02:00
Andreas Jaeger 601711dfa3 Handle sphinx PDF files
This enhances fetch-sphinx-tarball to fetch PDF files as well if
they are published

For now, it collects a single PDF file since sphinx only generates one.
The name of the PDF file is not known in advance.

The publishing is done with https://review.opendev.org/678142.
This will then be used by https://review.opendev.org/664555.

Change-Id: I872ab1f696495dc40cb845e0eee5f6f0ffe892d8
2019-08-23 08:30:05 +02:00
Andreas Jaeger 22a17fa19b Revert "Partial sync fetch-sphinx-tarball with fetch-sphinx-output"
We don't need this here - let's change openstack-manuals instead.

This reverts commit 75ae975235.

Change-Id: I9611950a391823efd2f7d2187da0992f00e0e454
2019-08-14 20:13:23 +00:00
Andreas Jaeger 75ae975235 Partial sync fetch-sphinx-tarball with fetch-sphinx-output
Add and use variable sphinx_output_dir as it's used in
fetch-sphinx-output.

Change-Id: I3fa7b1982463816d16ca6515edffb9e73b8cdbe5
2019-08-14 19:43:36 +02:00
James E. Blair ea00b6aa5a Use human-readable names for artifact returns
This will cause links to appear on the build page with text like
"Docs preview site".

Change-Id: I65f8990b65607e72190a871409c3e31014ce86df
2019-08-12 07:13:27 -07:00
Jeremy Stanley 2711d10186 Be consistent about spaces before and after vars
With the arrival of ansible-lint 4, Jinja2 variable expansions must
include spaces before and after the variable name inside the
brackets.

Adjust the new violations accordingly and remove the rule
206 exclusion.

Change-Id: Ib3ff7b0233a5d5cf99772f9c2adc81861cf34ffa
2019-08-07 20:17:41 +01:00
James E. Blair 189c2471ec Add fetch-sphinx-tarball role
This is a new artifact-aware method of fetching sphinx tarballs,
meant to be used in docs promote jobs.

Change-Id: I9a01af9e36b959c4fa42f3c41b548c87bf2e1759
2019-03-21 14:04:11 -07:00