This is an attempt to reorganize docs based on what we've learned so far:
* Audience is important -- help users find the job syntax reference without
getting bogged down in how to run zookeeper.
* Having distinct tutorials, howtos, and reference documentation is helpful.
* Grouping by subject matter is important; users shouldn't have to open tabs
with howto, reference, and tutorial to synthesize all the information on
a subject.
This reorg reduces the use of explicit reference/howto/tutorial/discussion
divisions since in some cases they never got sufficiently fleshed out (eg,
user tutorials), and in others the information was spread too thinly across
them all (eg authentication). However, those distinctions are still useful,
and the new organization reflects that somewhat.
I have made only some changes to content (generally in introductory sections
in order to make things make sense) and added a new "about Zuul" page. We
should still go through the documentation and update it and tweak the organization
further. This is mostly an attempt to get a new framework in place.
The theme is switched from alabaster to RTD. That's because RTD has really
good support for a TOC tree in the side bar with expansion. That makes a big
difference when trying to navigate large documentation like this. The new
framework is intended to have very good left-hand navigation for users.
Change-Id: I5ef88536acf1a1e58a07827e06b07d06588ecaf1
Reorganizing docs as recommended in:
https://www.divio.com/blog/documentation/
This is simply a reorganization of the existing documents and changes
no content EXCEPT to correct the location of sphinx doc references.
Expect followup changes to change document names (to reflect the new
structure) and to move content from existing guides (e.g., to move the
pipeline/project/job structure definitions out of the "Project Configuration"
reference guide into their own reference documents for easier locatability).
All documents are now located in either the "overview", "tutorials",
"discussions", or "references" subdirectories to reflect the new structure
presented to the user. Code examples and images are moved to "examples" and
"images" root-level directories.
Developer specific documents are located in the "references/developer"
directory.
Change-Id: I538ffd7409941c53bf42fe64b7acbc146023c1e3
The Service Workers seem to be consistently causing issues for people
that are strange, meaning many of our deployers are disabling them.
Since they aren't super necessary for the Zuul use case, change the
default behavior to be to disable them instead of enable them.
Change-Id: Iea8348a3b007badaae74fc1837b55bb0b076ac65
So that people can re-use the Dockerfiles to build zuul images
but with different flags set, plumb the env vars through here
as ARG entries.
Also, fix 2 doc references that were misspelled.
Change-Id: I320a496eadf4132fc0583dd48a87024a2ff61a07
In some cases we need to disable the service worker which does
advanced caching. For example this is the case when using a
redirect-based authentication proxy in front of zuul. In this case it
can be that the main page is serviced by the service worker without
the option to prevent that with cache headers. When that happens the
first request that hits the server and gets redirected to a login page
is an api request. This breaks zuul-web completely until the browser
cache is cleared.
A solution to this problem is to optionally disable the service
worker. This makes it possible to disable caching of any entry pages
forcing the redirect to happen before any api requests.
This change makes it possible to disable the service worker at compile
time by setting the REACT_APP_DISABLE_SERVICE_WORKER=true variable
without the need of on-the-fly patching of zuul itself before
installation.
Change-Id: I537b2c43b556cf2c3696683bf10dd06e152ec11f
Revert "Fix publish-openstack-javascript-content"
This reverts commit ca199eb9db.
This reverts commit 1082faae95.
This appears to remove the tarball publishing system that we rely on.
Change-Id: Id746fb826dfc01b157c5b772adc1d2991ddcd93a
This reverts commit 36aecc1229.
This reverts commit 683f50ed55.
This caused zuul.openstack.org to attempt to GET "https://api/status".
Change-Id: Ib25356f7ea5bfeec84e91195ac161d497f74d73d
Since we got started in all of this angular business back in the good
old storyboard days of yore, the angular folks cut a major release
(ok, 5 major releases). The old v1 angular is known as angularjs now, and
starting at v2 the new codebase is just 'angular'. While angularjs is
still supported for now, angularjs vs. angular seems to be more like
zuulv2 vs. zuulv3 - the developers really want people to
be on the >=v2 series, and they spent a good deal of time fixing issues
from the original angularjs.
The notable differences are the angular is a bit more explicit/verbose,
and that it uses typescript instead of plain javscript. The increased
verbosity wasn't the most popular with some fans of the original angularjs,
but for those of us who aren't breathing it every day the verbosity is
helpful.
There is a recommended code organization structure which has been used.
For zuul, there are notable changes to how the http client and location
service work, so the code related to those has been reworked.
$http has been reworked to use HttpClient - which defaults to grabbing
the remote json and which can do so in a typesafe way.
$location has been reworked to use the angular-routing module, which allows us
to pull both URL and Query String parameters in a structured manner. We
can similary pass query parameters to our output http requests.
Since routing is the new solution for $location, extract the navigation
bar into a re-usable component.
Add tslint config for the typescript. Keep running eslint on our
remaining plain javascript files, at least until we've got them all
transitioned over. Use the angular tslint config as a base, but also
adopt the rule from standardjs that says to not use semicolons since
they are not actually needed.
The main.ejs file is a webpack template, not an angular template. Move
it to web/config with the other webpack files to make that clear.
Add a job that builds the zuul dashboard with the ZUUL_API_URL set to
point to software factory. This should allow us to see a live test with
a multi-tenant scheme.
Depends-On: https://review.openstack.org/572542
Change-Id: Ida959da05df358994f4d11bb6f40f094d39a9541
Co-Authored-By: Tristan Cacqueray <tdecacqu@redhat.com>
Co-Authored-By: Artem Goncharov <artem.goncharov@gmail.com>
The docker instructions and build:docker helper command were put in
place when we first started working on this stack to try to reduce the
burden on people from the new toolchain. However, build:docker results
in root-owned files in zuul/web/static, and in general is complex and
fragile for not very much benefit now that nodeenv is in place for tox
runs.
The docker instructions in the docs, as well as the instructions for
people with old npm are similarly removed, as nodeenv handles these
cases as well.
In general, we should be encouraging people to get familiar with the
tools rather than hiding them.
Change-Id: I931d1101e025ba1236f4abc1d5e088ba5d113a5a
There is some cruft in here, a section that could use more explaining,
and also the examples should use the console code-block entry instead of
bash.
Change-Id: Id88f6ce766139d48889d312b9f9b7d236fb7a646
Add a wrapper installation script that will install npm and yarn into the
current virtualenv. If yarn is installed globally, the script is a
no-op.
If yarn is not installed globally, whenever tox thinks it needs to
create or re-create a virtualenv, install nodeenv then use it to
install yarn.
This removes the use of the zuul-tox-py35 job because it should now just work
properly with the normal job. It does not remove the job itself because
it's still used in tox-py35-on-zuul.
Change-Id: If360a3f0c6b3d74498f8aa063d8b1ae87daff101
We want the value of the PWD variable so need ${PWD} instead of $(PWD)
which will give us the results of executing the PWD command.
Change-Id: I158ffc22bd1ecc5c446dfbe403a840948896003f
The python service is "zuul-web", but the javascript portion is
currently also being built as "zuul-web". While the javascript code is
intended to be served by zuul-web, not having its own name has made it
awkward to be clear about which codebase is being discussed.
npm has also added namespacing to published modules, which allows
grouping modules and is indicated by the @ prefix.
Update the config to call the javascript code '@zuul-ci/dashboard'. This
will allow us to add an npm publication job, although we need to set up
some accounts for that first.
Change-Id: I6e31ae23f74f887448f0a48147d91c156025e925
yarn drives package and dependency management. webpack handles
bundling, minification and transpiling down to browser-acceptable
javascript but allows for more modern javascript like import statements.
There are some really neat things in the webpack dev server. CSS
changes, for instance, get applied immediately without a refresh. Other
things, like the jquery plugin do need a refresh, but it's handled just
on a file changing.
As a followup, we can also consider turning the majority of the status page
into a webpack library that other people can depend on as a mechanism
for direct use. Things like that haven't been touched because allowing
folks to poke at the existing known status page without too many changes
using the tools seems like a good way for people to learn/understand the
stack.
Move things so that the built content gets put
into zuul/web/static so that the built-in static serving from zuul-web
will/can serve the files.
Update MANIFEST.in so that if npm run build:dist is run before the
python setup.py sdist, the built html/javascript content will be
included in the source tarball.
Add a pbr hook so that if yarn is installed, javascript content will be
built before the tarball.
Add a zuul job with a success url that contains a source_url
pointing to the live v3 data.
This adds a framework for verifying that we can serve the web app
urls and their dependencies for all of the various ways we want to
support folks hosting zuul-web.
It includes a very simple reverse proxy server for approximating
what we do in openstack to "white label" the Zuul service -- that
is, hide the multitenancy aspect and present the single tenant
at the site root.
We can run similar tests without the proxy to ensure the default,
multi-tenant view works as well.
Add babel transpiling enabling use of ES6 features
ECMAScript6 has a bunch of nice things, like block scoped variables,
const, template strings and classes. Babel is a javascript transpiler
which webpack can use to allow us to write using modern javascript but
the resulting code to still work on older browsers.
Use the babel-plugin-angularjs-annotate so that angular's dependency
injection doesn't get borked by babel's transpiling things (which causes
variables to otherwise be renamed in a way that causes angular to not
find them)
While we're at it, replace our use of var with let (let is the new
block-scoped version of var) and toss in some use of const and template
strings for good measure.
Add StandardJS eslint config for linting
JavaScript Standard Style is a code style similar to pep8/flake8. It's
being added here not because of the pep8 part, but because the pyflakes
equivalent can catch real errors. This uses the babel-eslint parser
since we're using Babel to transpile already.
This auto-formats the existing code with:
npm run format
Rather than using StandardJS directly through the 'standard' package,
use the standardjs eslint plugin so that we can ignore the camelCase
rule (and any other rule that might emerge in the future)
Many of under_score/camelCase were fixed in a previous version of the patch.
Since the prevailing zuul style is camelCase methods anyway, those fixes
were left. That warning has now been disabled.
Other things, such as == vs. === and ensuring template
strings are in backticks are fixed.
Ignore indentation errors for now - we'll fix them at the end of this
stack and then remove the exclusion.
Add a 'format' npm run target that will run the eslint command with
--fix for ease of fixing reported issues.
Add a 'lint' npm run target and a 'lint' environment that runs with
linting turned to errors. The next patch makes the lint environment more
broadly useful.
When we run lint, also run the BundleAnalyzerPlugin and set the
success-url to the report.
Add an angular controller for status and stream page
Wrap the status and stream page construction with an angular controller
so that all the javascripts can be bundled in a single file.
Building the files locally is wonderful and all, but what we really want
is to make a tarball that has the built code so that it can be deployed.
Put it in the root source dir so that it can be used with the zuul
fetch-javascript-tarball role.
Also, replace the custom npm job with the new build-javascript-content
job which naturally grabs the content we want.
Make a 'main.js' file that imports the other three so that we just have
a single bundle. Then, add a 'vendor' entry in the common webpack file
and use the CommonsChunkPlugin to extract dependencies into their own
bundle. A second CommonsChunkPlugin entry pulls out a little bit of
metadata that would otherwise cause the main and vendor chunks to change
even with no source change. Then add chunkhash into the filename. This
way the files themselves can be aggressively cached.
This all follows recommendations from https://webpack.js.org/guides/caching/https://webpack.js.org/guides/code-splitting/ and
https://webpack.js.org/guides/output-management/
Change-Id: I2e1230783fe57f1bc3b7818460463df1e659936b
Co-Authored-By: Tristan Cacqueray <tdecacqu@redhat.com>
Co-Authored-By: James E. Blair <jeblair@redhat.com>
James E. Blair