Commit Graph

2 Commits

Author SHA1 Message Date
James E. Blair 153f8a90cc Support multiple semaphores
This allows jobs to request multiple semaphores.  Zuul will wait until
all are available before acquiring them* and will not start a job unless
all have been acquired.

This is useful for jobs which require access to mulitple limited resources
(especially if other jobs require access to a subset or superset of those
same resources).

* Implementation note: for efficiency, we actually do acquire them
  one-by-one but then release any which have been acquired if they are
  not all available.  This all happens very quickly within a single
  attempt to start a job.  We don't hold semaphores while we wait for
  others as that could cause deadlocks.

To be consistent with other job attributes which accept lists, this
deprecates job.semaphore and replaces it with job.semaphores.

Change-Id: I295a891a2d02b904820d8f60afe8ef862693b75d
2021-05-25 17:52:23 -07:00
Tobias Henkel ae887dab58
Improve resource usage with semaphores
Currently when jobs use semaphores they first get and lock the build
nodes and then aquire the semaphore. If there are many jobs waiting
for the semaphore this can block a substantial part of the available
resources. In order to make this safe default to acquire the semaphore
before requesting the nodes.

However in some cases when jobs with a semaphore shall run as fast as
possible in a consecutive manner then it might be preferrable to
accept some waste of resources. In order to support this use case the
job using a semaphore can override this behavior and still acquire the
semaphore after getting the nodes.

Change-Id: Id6f582ec29219d280d05319d1b822c7934437b7a
2018-11-20 15:20:59 +01:00