Tox v4 behaves significantly differently than v3, and some of the
more complex things we do with tox would need an overhaul to
continue to use it. Meanwhile, nox is much simpler and more
flexible, so let's try using it.
This adds a noxfile which should be equivalent to our tox.ini file.
We still need to update the docs build (which involves changes to
base jobs) before we can completely remove tox.
Depends-On: https://review.opendev.org/868134
Change-Id: Ibebb0988d2702d310e46c437e58917db3f091382
Update the TESTING.rst file to reflect that just installing and
starting zookeeperd is no longer sufficient now that we require TLS
and auth for the connection, as well as running database servers.
Suggest the container-based setup script instead. Also improve that
script to allow it to be invoked as a normal user, with root command
escalation tool choice (e.g. "sudo") supplied through a ROOTCMD
environment variable, so that things created inside the git worktree
like the CA don't end up root-owned.
Related, the tox-docker plugin previously suggested in the document
is no longer a viable option for the same reasons, so clean up the
plumbing for it in tox.ini as well.
Change-Id: Iac32799425a5bd4b1bdbf56f34a2310241ac4499
This job has been broken for a long time, and the paths it tests are
better covered by the quickstart tests which bring up nodepool with
Zuul.
It's a bit of an odd job because nodepool sets itself up, but then the
test calls the tox "nodepool" environment in Zuul here. So remove the
job inclusion, but also the tox/unit-tests being run by the job (it's
already failing and non-voting on nodepool, so this won't affect
things).
Change-Id: I8483b1d66a6a58d4bd2f2fce82b023d8f0446ae7
In case for whatever reason there is already another
MySQL server running on the standard port developers
may want to run the MySQL test-server on a different
port. This can be done by specifying an environment
variable "ZUUL_MYSQL_PORT":
export ZUUL_MYSQL_PORT=3307
tools/test-setup-docker.sh
tox -e py3
Change-Id: Id32bac1a87619803d973b066762d525bd9a4f913
The number of schedulers can be defined via the ZUUL_SCHEDULER_COUNT
environment variable (also forwarded by tox).
Disclaimer: running the tests with multiple schedulers will still fail
as not all test support multiple schedulers yet.
Change-Id: Ie11467dcf0fc36a5e167adcbd29ac4f256e0b3c5
Previously we removed the CPU limit from stestr to speed up testing.
However, based on log data we have occasionaly failures now due to
exceeding system load limits in the Zuul executor governors.
Based on logs we appear to be just over that limit and a small pullback
may be sufficient to make things reliable. Previously we were dividing
the total cpu count by 2, but now lets try reducing it by 1 instead. On
OpenDev's CI systems this should have us using 7 test processes instead
of 4 when we were dividing by 2.
Change-Id: I1ded37b799780c52dbab6a84c1fde9fae099a08e
This is 2 changes:
* Run dstat in unit test jobs so that we can get an idea of
system performance/utilization.
* Remove the stestr concurrency cap. With 8 cores, we have
enough headroom to run the test dependencies (sql/zk) while
the tests are running too. Use all the CPU that's available.
Change-Id: I9f250865f7043fdbb1fa8a01f1bc9508490accc1
SQLAlchemy 2.0 will introduce a number of changes. Thankfully Current
sqlalchemy has deprecation warnings and can be run with future flags set
to True to enforce 2.0 behavior. We use these tools to prepare Zuul for
the SQLAlchemy 2.0 release.
In tox.ini configure the environment to always emit DeprecationWarnings
for modules that touch sqlalchemy.
Update sqlconnection to use the new location for declarative_base and
set future=True on our Engine and Session objects.
Finally update the database migration tests to use transaction based
connections for executing raw sql statements. Also we switch to the
exec_driver_sql method for that. SQLAlchemy 2.0 will not do implicit
autocommiting and doesn't support executing strings directly.
https://docs.sqlalchemy.org/en/14/changelog/migration_20.html has tons
of info on these changes. Reviews should probably pay attention to the
transaction behavior changes as well as any alembic code that might also
need updating.
Change-Id: I4e7a56d24d0f52b6d5b00a8c12fed52d6fae92ef
We aren't generally using type annotations, and mypy itself is
somewhat flawed, occasionally producing false positives and rarely
catching errors. Stop running it.
Change-Id: I6f24457f7d99ca11ec9228e505e6edec558baf9e
We're seeing occasional npm install failures in jobs; retry the
"yarn install" command if it fails.
Also, pass the YARN_REGISTRY env var in the tox remote job.
Change-Id: Ic80dccf0869ce805368f678eafcfdf48f8bf9bea
This mirrors the configuration in Nodepool for using TLS-enabled
ZooKeeper in tests. We use the ensure-zookeeper role in order
to get a newer ZooKeeper than is supplied in bionic.
Change-Id: I14413fccbc9a6a7a75b6233d667e2a1d2856d894
This allows mypy to produce the same results regardless how is called.
For example IDEs will produce the same results as tox -e linters
instead of displaying violations which the core team decided to
disable.
Change-Id: I20eeafc0549f78e3d4f9636793cfe031389b6bbe
Define unit testing for the zuul-client CLI. The actual
testing job will be defined in a subsequent patch to
the zuul-client project.
These tests are stored in the zuul repository rather than in
zuul-client's because we rely on zuul's test framework to bootstrap
zuul and zuul-web, which is not available through zuul's python
package. This is similar to what is done for nodepool's functional
testing with zuul.
Change-Id: I156162279c5d638ef39d22fcfb043655890c2d6e
Depends-On: https://review.opendev.org/751291
Opendev runs a caching mirror for npm packages. Use it when running
yarn install during tox tests. Due to the way yarn.lock files work at
the moment we need to patch the lockfile on the fly to inject the
mirror urls [1]. This can be handlel inside the pip.sh where we patch
the lockfile, run yarn and reset it afterwards.
[1] https://github.com/yarnpkg/yarn/issues/2566
Change-Id: Id1f2daac8327f3110e6beeb2d6391baccfd3ce34
We missed including DOCKER_* to tox passenv and that prevented
use of docker for users that are using other hosts. Most notable
option is DOCKER_HOST but any docker specific options should be
passed, so the docker commands keep their behaviors.
This also sorts passenv list, to ease maintenance.
Change-Id: Iac436da1202066541fc98eb94a2a228e6cf8b337
mypy depends on typed-ast which relatively recently added python 3.8
support. Unfortunately the mypy we were using requires an older
typed-ast and fails under python 3.8. Fix this by bumping mypy up to a
level where minimal code changes are necessary.
The code changes we made ignore type conflicts due to redefinition on
conditional imports. Comments include link back to a bug describing this
issue and workaround in mypy.
Change-Id: Iec69a27b16a1e09eb6bfbcf8d68deb1ae68d42a3
pep8 is legacy, we now use flake8 and the virtual environment also runs
mypy and openapi-spec-validator.
Rename the virtualenv from 'pep8' to 'linters'.
Use tox-linters instead of tox-pep8.
tox-pep8 skips test-setup.sh Ia140b331ae1502fcf1f685faa6dace117d77d35d
Done for tox-linters by Idc72c7d1a88fff8e738c896e2482b5cb1d844600
Depends-On: Idc72c7d1a88fff8e738c896e2482b5cb1d844600
Change-Id: I1fbd8335a2353f750f49b6a763eefc6f399a7143
Replace our tox-py37 job with tox-py38, extend the list of trove
classifiers for Python versions in package metadata, and replace the
"py35" in the tox.ini envlist with just "py3" so that folks running
`tox` unqualified on their systems will use whatever python3
interpreter they have on hand (odds are it's in our supported range
these days). Also uncap python-daemon so we use a version compatible
with Python >=3.8.
Change-Id: I4b382bb89847d3ae2b20292d923c3c2211dfb254
We currently set the OS_LOG_DEFAULTS in tox to an empty string if it's
unset. This essentially overwrites the default levels defined in our
base test class to debug level for everything. This makes the tox logs
hard to read because especially kazoo and gear are very chatty in the
debug level. Further the excessive logging often exceeds the lengh
limitation of stestr when the test fails.
To mitigate this remove OS_LOG_DEFAULTS from setenv and leave it in
passenv so we take the zuul test case default when it's not defined
from outside.
Change-Id: I8f634f1ca5cc028018f181a07e204b55788759fa
Linters are only used by the 'pep8' virtual environment, we do not need
them in other test environment. Also 'pep8' only requires the three
linters it used and the other modules are not needed.
Move the pep8 dependencies to the pep8 testenv.
Change-Id: I3a8436cbff9e2386cf0e0f5b0d1742283e647b56
Running bindep doesn't rely on the zuul source or any of its
dependencies installed, so set usedevelop to False ensuring only
bindep and its dependencies are installed (which is much quicker and
less likely to raise unrelated dependency errors in the process).
Change-Id: I886857e22ed953efdaab5b6247d1e15039806f48
This change adds a swagger description of the REST API. The description is
rendered in the sphinx user documentations and in the web interface.
Change-Id: I753524f40a09874dab5952f14ab17025525bbab9
We're regularly seeing the test_plugins unit test exceed our 260s
hard timeout fixture on slow job nodes, so increase the base timeout
from 4 minutes to 6 to accommodate this. Also bump the wait timeout
in test_plugins from 3 to 4 minutes.
Change-Id: I46a0f84b43c3e7c15919f3681d50e32aa7d3b89f
Since the multi-ansible work the test-logs.sh script is broken and
needs to be fixed to use the managed ansible.
Change-Id: I271f45d9f096a05949900ba1dd2bf8321c459815
To see if reducing the concurrency of tests makes them more reliable,
set them to half the number of cpus (by default, stestr uses the
number of cpus).
Change-Id: I7977c853b759cf980a6717a70876857ef7e9f77e
The cover target wasn't working because the various ansible versions
weren't getting installed. We can fix that by removing the custom pip
install command for the cover target and instead rely on the default
install command which handles this for us.
Change-Id: Ieb7adce16356f0f080ceac580fbddab0d2dbc537
This allows us to start a zookeeper instance via docker as part of the
standard tox process, avoiding issues like those described in [1]. It's
exposed as a separate target, to ensure users can use local
installations if necessary. This requires bumping the minimum version of
tox to 3.2 so we can make use of the 'requires' config option [2].
[1] https://review.openstack.org/#/c/648399/
[2] https://tox.readthedocs.io/en/latest/config.html#conf-requires
Change-Id: Ie848d409533b2a53523cd930299acf21ac10e13c
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Yes, this is purely stylistic and does screw up git log, but it's a heck
of a lot easier to work with in this way.
Change-Id: Ie20ad2fc62be83181826cbdad64ba951d43534e8
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Means we don't need to set this for the default factors (py3x), meaning
we get new versions for free going forward.
Change-Id: Idba181e96adab16181a7f776093c9dfe33e071c6
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
These were added in I5ce1385245c76818777aa34230786a9dbaf723e5 but are
not necessary since I680777e5406368d35366602ac37e029db91c8a5c, the
latter reverting most of the former. Removed them.
Change-Id: Ic820ac288cc8b97b417ba9fa25328c30c99562a3
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The OS_TEST_TIMEOUT setting is used to create a TestTimeout fixture
which aborts tests. This is currently set to 150s which is too low
often for the test_plugin and test_playbook tests.
Change-Id: Ic638dfdad513f3a61e4026c3667b1ca18d464fc9
On some newer distros like Fedora we need to be able to run tox with
py37. For the py3* environments we need to use the pip wrapper so
declare this as well.
Change-Id: I666098238383ebc70982772dbeb449ab37b3b37c
We currently install ansible during the pre_command phase in tox. This
breaks the following use case:
tox -e py36 --notest
ttrun <test case>
This is broken now because ttrun doesn't run pre_commands. Instead we
already have a pip wrapper that does the nodeenv installation. We can
use this to also call zuul-manage-ansible on zuul installation.
Change-Id: I680777e5406368d35366602ac37e029db91c8a5c
Adding more ansible versions lead to longer runtimes so we need to
either run those tests concurrently or split them per ansible version
and run one job per ansible version.
Change-Id: I41574d8f5c0f7f5fa2c5272044d6b387def5bfa7
As a first step towards supporting multiple ansible versions we need
tooling to manage ansible installations. This moves the installation
of ansible from the requirements.txt into zuul. This is called as a
setup hook to install the ansible versions into
<prefix>/lib/zuul/ansible. Further this tooling abstracts knowledge
that the executor must know in order to actually run the correct
version of ansible.
The actual usage of multiple ansible versions will be done in
follow-ups.
For better maintainability the ansible plugins live in
zuul/ansible/base where plugins can be kept in different versions if
necessary. For each supported ansible version there is a specific
folder that symlinks the according plugins.
Change-Id: I5ce1385245c76818777aa34230786a9dbaf723e5
Depends-On: https://review.openstack.org/623927
Without this, basepython overrides the python in all envs, even the
builtin ones. That means that py35 and py36 would both be testing with
whatever the default python3 is on the system, not 3.5 and 3.6
specifically.
Change-Id: I1af5bd99934638244ea29375fc4f170e5616f01b
Benjamin Schanzel