BuildSteps¶
There are a few parent classes that are used as base classes for real buildsteps. This section describes the base classes. The "leaf" classes are described in Build Steps.
BuildStep¶
-
class
buildbot.process.buildstep.
BuildStep
(name, locks, haltOnFailure, flunkOnWarnings, flunkOnFailure, warnOnWarnings, warnOnFailure, alwaysRun, progressMetrics, useProgress, doStepIf, hideStepIf)¶ All constructor arguments must be given as keyword arguments. Each constructor parameter is copied to the corresponding attribute.
-
name
¶ The name of the step.
-
locks
¶ List of locks for this step; see Interlocks.
-
progressMetrics
¶ List of names of metrics that should be used to track the progress of this build, and build ETA's for users. This is generally set in the
-
useProgress
¶ If true (the default), then ETAs will be calculated for this step using progress metrics. If the step is known to have unpredictable timing (e.g., an incremental build), then this should be set to false.
-
doStepIf
¶ A callable or bool to determine whether this step should be executed. See Common Parameters for details.
-
hideStepIf
¶ A callable or bool to determine whether this step should be shown in the waterfall and build details pages. See Common Parameters for details.
The following attributes affect the behavior of the containing build:
-
haltOnFailure
¶ If true, the build will halt on a failure of this step, and not execute subsequent tests (except those with
alwaysRun
).
-
flunkOnWarnings
¶ If true, the build will be marked as a failure if this step ends with warnings.
-
flunkOnFailure
¶ If true, the build will be marked as a failure if this step fails.
-
warnOnWarnings
¶ If true, the build will be marked as warnings, or worse, if this step ends with warnings.
-
warnOnFailure
¶ If true, the build will be marked as warnings, or worse, if this step fails.
-
alwaysRun
¶ If true, the step will run even if a previous step halts the build with
haltOnFailure
.
A few important pieces of information are not available when a step is constructed, and are added later. These are set by the following methods; the order in which these methods are called is not defined.
-
setBuild
(build)¶ Parameters: build -- the Build
instance controlling this step.This method is called during setup to set the build instance controlling this slave. Subclasses can override this to get access to the build object as soon as it is available. The default implementation sets the
build
attribute.
-
build
¶ The build object controlling this step.
-
setBuildSlave
(build)¶ Parameters: build -- the BuildSlave
instance on which this step will run.Similarly, this method is called with the build slave that will run this step. The default implementation sets the
buildslave
attribute.
-
buildslave
¶ The build slave that will run this step.
-
setDefaultWorkdir
(workdir)¶ Parameters: workdir -- the default workdir, from the build This method is called at build startup with the default workdir for the build. Steps which allow a workdir to be specified, but want to override it with the build's default workdir, can use this method to apply the default.
-
setStepStatus
(status)¶ Parameters: status ( BuildStepStatus
) -- step statusThis method is called to set the status instance to which the step should report. The default implementation sets
step_status
.
-
step_status
¶ The
BuildStepStatus
object tracking the status of this step.
-
setupProgress
()¶ This method is called during build setup to give the step a chance to set up progress tracking. It is only called if the build has
useProgress
set. There is rarely any reason to override this method.
-
progress
¶ If the step is tracking progress, this is a
StepProgress
instance performing that task.
Execution of the step itself is governed by the following methods and attributes.
-
startStep
(remote)¶ Parameters: remote -- a remote reference to the slave-side SlaveBuilder
instanceReturns: Deferred Begin the step. This is the build's interface to step execution. Subclasses should override
start
to implement custom behaviors.The method returns a Deferred that fires when the step finishes. It fires with a tuple of
(result, [extra text])
, whereresult
is one of the constants frombuildbot.status.builder
. The extra text is a list of short strings which should be appended to the Build's text results. For example, a test step may add17 failures
to the Build's status by this mechanism.The deferred will errback if the step encounters an exception, including an exception on the slave side (or if the slave goes away altogether). Normal build/test failures will not cause an errback.
-
start
()¶ Returns: None
orSKIPPED
, optionally via a Deferred.Begin the step. Subclasses should override this method to do local processing, fire off remote commands, etc. The parent method raises
NotImplementedError
.When the step is done, it should call
finished
, with a result -- a constant frombuildbot.status.results
. The result will be handed off to theBuild
.If the step encounters an exception, it should call
failed
with a Failure object. This method automatically fails the whole build with an exception. A common idiom is to addfailed
as an errback on a Deferred:cmd = RemoteCommand(args) d = self.runCommand(cmd) def suceed(_): self.finished(results.SUCCESS) d.addCallback(succeed) d.addErrback(self.failed)
If the step decides it does not need to be run,
start
can return the constantSKIPPED
. In this case, it is not necessary to callfinished
directly.
-
finished
(results)¶ Parameters: results -- a constant from results
A call to this method indicates that the step is finished and the build should analyze the results and perhaps proceed to the next step. The step should not perform any additional processing after calling this method.
-
failed
(failure)¶ Parameters: failure -- a Failure
instanceSimilar to
finished
, this method indicates that the step is finished, but handles exceptions with appropriate logging and diagnostics.This method handles
BuildStepFailed
specially, by callingfinished(FAILURE)
. This provides subclasses with a shortcut to stop execution of a step by raising this failure in a context wherefailed
will catch it.
-
interrupt
(reason)¶ Parameters: reason (string or Failure
) -- why the build was interruptedThis method is used from various control interfaces to stop a running step. The step should be brought to a halt as quickly as possible, by cancelling a remote command, killing a local process, etc. The step must still finish with either
finished
orfailed
.The
reason
parameter can be a string or, when a slave is lost during step processing, aConnectionLost
failure.The parent method handles any pending lock operations, and should be called by implementations in subclasses.
-
stopped
¶ If false, then the step is running. If true, the step is not running, or has been interrupted.
This method provides a convenient way to summarize the status of the step for status displays:
-
describe
(done=False)¶ Parameters: done -- If true, the step is finished. Returns: list of strings Describe the step succinctly. The return value should be a sequence of short strings suitable for display in a horizontally constrained space.
Note
Be careful not to assume that the step has been started in this method. In relatively rare circumstances, steps are described before they have started. Ideally, unit tests should be used to ensure that this method is resilient.
Build steps support progress metrics - values that increase roughly linearly during the execution of the step, and can thus be used to calculate an expected completion time for a running step. A metric may be a count of lines logged, tests executed, or files compiled. The build mechanics will take care of translating this progress information into an ETA for the user.
-
setProgress
(metric, value)¶ Parameters: - metric (string) -- the metric to update
- value (integer) -- the new value for the metric
Update a progress metric. This should be called by subclasses that can provide useful progress-tracking information.
The specified metric name must be included in
progressMetrics
.
The following methods are provided as utilities to subclasses. These methods should only be invoked after the step is started.
-
slaveVersion
(command, oldVersion=None)¶ Parameters: - command (string) -- command to examine
- oldVersion -- return value if the slave does not specify a version
Returns: string
Fetch the version of the named command, as specified on the slave. In practice, all commands on a slave have the same version, but passing
command
is still useful to ensure that the command is implemented on the slave. If the command is not implemented on the slave,slaveVersion
will returnNone
.Versions take the form
x.y
wherex
andy
are integers, and are compared as expected for version numbers.Buildbot versions older than 0.5.0 did not support version queries; in this case,
slaveVersion
will returnoldVersion
. Since such ancient versions of Buildbot are no longer in use, this functionality is largely vestigial.
-
slaveVersionIsOlderThan
(command, minversion)¶ Parameters: - command (string) -- command to examine
- minversion -- minimum version
Returns: boolean
This method returns true if
command
is not implemented on the slave, or if it is older thanminversion
.
-
getSlaveName
()¶ Returns: string Get the name of the buildslave assigned to this step.
-
runCommand
(command)¶ Returns: Deferred This method connects the given command to the step's buildslave and runs it, returning the Deferred from
run
.
-
addURL
(name, url)¶ Parameters: - name -- URL name
- url -- the URL
Add a link to the given
url
, with the givenname
to displays of this step. This allows a step to provide links to data that is not available in the log files.
The
BuildStep
class provides minimal support for log handling, that is extended by theLoggingBuildStep
class. The following methods provide some useful behaviors. These methods can be called while the step is running, but not before.-
addLog
(name)¶ Parameters: name -- log name Returns: LogFile
instanceAdd a new logfile with the given name to the step, and return the log file instance.
-
getLog
(name)¶ Parameters: name -- log name Returns: LogFile
instanceRaises: KeyError
if the log is not foundGet an existing logfile by name.
-
addCompleteLog
(name, text)¶ Parameters: - name -- log name
- text -- content of the logfile
This method adds a new log and sets
text
as its content. This is often useful to add a short logfile describing activities performed on the master. The logfile is immediately closed, and no further data can be added.
-
addHTMLLog
(name, html)¶ Parameters: - name -- log name
- html -- content of the logfile
Similar to
addCompleteLog
, this adds a logfile containing pre-formatted HTML, allowing more expressiveness than the text format supported byaddCompleteLog
.
-
addLogObserver
(logname, observer)¶ Parameters: - logname -- log name
- observer -- log observer instance
Add a log observer for the named log. The named log need not have been added already: the observer will be connected when the log is added.
See Adding LogObservers for more information on log observers.
-
LoggingBuildStep¶
-
class
buildbot.process.buildstep.
LoggingBuildStep
(logfiles, lazylogfiles, log_eval_func, name, locks, haltOnFailure, flunkOnWarnings, flunkOnFailure, warnOnWarnings, warnOnFailure, alwaysRun, progressMetrics, useProgress, doStepIf, hideStepIf)¶ Parameters: - logfiles -- see
ShellCommand
- lazylogfiles -- see
ShellCommand
- log_eval_func -- see
ShellCommand
The remaining arguments are passed to the
BuildStep
constructor.This subclass of
BuildStep
is designed to help its subclasses run remote commands that produce standard I/O logfiles. It:- tracks progress using the length of the stdout logfile
- provides hooks for summarizing and evaluating the command's result
- supports lazy logfiles
- handles the mechanics of starting, interrupting, and finishing remote commands
- detects lost slaves and finishes with a status of
RETRY
-
logfiles
¶ The logfiles to track, as described for
ShellCommand
. The contents of the class-levellogfiles
attribute are combined with those passed to the constructor, so subclasses may add log files with a class attribute:class MyStep(LoggingBuildStep): logfiles = dict(debug='debug.log')
Note that lazy logfiles cannot be specified using this method; they must be provided as constructor arguments.
-
startCommand
(command)¶ Parameters: command -- the RemoteCommand
instance to startNote
This method permits an optionalerrorMessages
parameter, allowing errors detected early in the command process to be logged. It will be removed, and its use is deprecated.Handle all of the mechanics of running the given command. This sets up all required logfiles, keeps status text up to date, and calls the utility hooks described below. When the command is finished, the step is finished as well, making this class is unsuitable for steps that run more than one command in sequence.
Subclasses should override
start
and, after setting up an appropriate command, call this method.def start(self): cmd = RemoteShellCommand(..) self.startCommand(cmd, warnings)
To refine the status output, override one or more of the following methods. The
LoggingBuildStep
implementations are stubs, so there is no need to call the parent method.-
commandComplete
(command)¶ Parameters: command -- the just-completed remote command This is a general-purpose hook method for subclasses. It will be called after the remote command has finished, but before any of the other hook functions are called.
-
createSummary
(stdio)¶ Parameters: stdio -- stdio LogFile
This hook is designed to perform any summarization of the step, based either on the contents of the stdio logfile, or on instance attributes set earlier in the step processing. Implementations of this method often call e.g.,
addURL
.
-
evaluateCommand
(command)¶ Parameters: command -- the just-completed remote command Returns: step result from buildbot.status.results
This hook should decide what result the step should have. The default implementation invokes
log_eval_func
if it exists, and looks atrc
to distinguishSUCCESS
fromFAILURE
.
The remaining methods provide an embarrassment of ways to set the summary of the step that appears in the various status interfaces. The easiest way to affect this output is to override
describe
. If that is not flexible enough, overridegetText
and/orgetText2
.-
getText
(command, results)¶ Parameters: - command -- the just-completed remote command
- results -- step result from
evaluateCommand
Returns: a list of short strings
This method is the primary means of describing the step. The default implementation calls
describe
, which is usually the easiest method to override, and then appends a string describing the step status if it was not successful.
-
getText2
(command, results)¶ Parameters: - command -- the just-completed remote command
- results -- step result from
evaluateCommand
Returns: a list of short strings
Like
getText
, this method summarizes the step's result, but it is only called when that result affects the build, either by making it halt, flunk, or end with warnings.
- logfiles -- see
Exceptions¶
-
exception
buildbot.process.buildstep.
BuildStepFailed
¶ This exception indicates that the buildstep has failed. It is useful as a way to skip all subsequent processing when a step goes wrong. It is handled by
BuildStep.failed
.