Automation for the people: Continuous feedback

When I describe CI to technologists that haven't used the practice
yet, I tend to focus on one of its major benefits: a decrease in the
time between when a defect
is introduced and when it is fixed. This benefit is a direct result of
a CI server's ability to provide rapid feedback of build statuses. In
my opinion, rapid feedback of build events is the central tenet of CI
and one that causes many to describe a CI system as a continuous feedback
mechanism.

When a build is broken, I want to know immediately so I can take the proper action to fix it -- whether that
be a compilation error, test failure, or even an increase in overall complexity. The sooner I know, the more relevant
the information and the better chance I have to fix the problem.

Feedback, however, is useless without action. In the case of CI,
this action must be prompt because a broken build affects everyone;
therefore, the feedback mechanism employed by a CI server must be
timely. The beauty is that most, if not all, CI servers available today
provide an e-mail feedback mechanism; however, there is a cornucopia of
other mediums to choose from ranging from Really Simple Syndication
(RSS) feeds to SMS text
messages to X10 devices. And these various feedback mechanisms quickly
facilitate jumpstarting the necessary people into action.

About this series

As developers, we
work to automate processes for users; yet, many of us overlook
opportunities to automate our own development processes. To that end, Automation for the people
is a series of articles dedicated to exploring
the practical uses of automating software development processes and
teaching you when
and how
to apply automation successfully.

Execution a la e-mail

The ubiquitous nature of e-mail makes it a simple and useful means of
feedback -- every CI server I'm familiar with provides some type of
e-mail feedback mechanism. In fact, most CI servers provide the
capability for configuring who
receives which
message, based on a desired build event. For instance, a technical
lead may always receive an e-mail regardless
of build status, but the project manager only receives an e-mail when
the build fails. Additionally, the developer or developers who checked
any code into a CM system (and thus triggered a CI build) always
receive an e-mail indicating the build's status.

Listing 1 is an example of a CruiseControl main configuration file (config.xml), which is
configured to send an e-mail, in HTML format, to the project's "build master" for both successful and failed builds:

Listing 1. Sending an e-mail with CruiseControl

<publishers> <htmlemail mailhost="localhost" defaultsuffix="localhost" username="bfranklin" password="G0Fly@Kite" returnname="Buildmaster" returnaddress="buildmaster@localhost" subjectprefix="[CC]" xsldir="webapps/cruisecontrol/xsl" css="webapps/cruisecontrol/css/cruisecontrol.css"> <always address="buildmaster@localhost"/> </htmlemail> <publishers>

You can use the

defaultsuffix

attribute of CruiseControl's e-mail task to also send e-mails to the
developer(s) who committed the most recent changes, as well.

Figure 1 shows an example of the e-mail a user will receive based on the configuration in Listing 1:

Figure 1. A sample build status e-mail a la CruiseControl



While e-mail is arguably the de facto standard feedback mechanism for
projects that use CI, if team members are flooded with too many
e-mails, they may begin to ignore them, which, of course, defeats the
purpose of a feedback mechanism in the first place. I find it more
productive to configure CI systems to send e-mails only to the
developer or developers who applied the most recent change to a code
base and to project leads who don't mind receiving a flood of build
status e-mails every day.

What is Continuous Integration?

Continuous integration (CI) is a practice that enables teams to receive feedback and make improvements on a
continual basis
rather than waiting until later in the
development cycle to find and fix defects. Tools (like CruiseControl)
run in the background polling a version control repository looking for
changes. When a change does occur, the tool executes a predefined build
script through Ant, for example. Continuous Inspection is facilitated
by the practice of continuous integration

RSS to the rescue

RSS
is both a passive and active form of feedback, which can make this
feedback mechanism more appealing because of a reduction in the amount
of e-mail messages you'd receive otherwise. An RSS feed is a generated
XML file (which conforms to a standard) that is updated based on
certain events. An RSS reader then picks up the change and creates a
passive message indicating new content. So, if you'd rather not get
bombarded with e-mail, try this approach instead.

Within a CI system, this RSS feed can be updated based on the latest build status.
For instance, Figure 2 is an example of the CruiseControl RSS XML that, by default, is generated for any build event:

Figure 2. RSS XML from CruiseControl



Simply
point your handy RSS reader to this feed and when a new build event
occurs, and the RSS feed is updated, bingo, you'll be notified. Figure
3 shows the RSS feed as displayed in an RSS reader:

Figure 3. RSS indicates things went wrong!



Back to top

Stir it up with SMS

What if you're away from your computer and the build fails? If you
believe that keeping an integration build successful is one of the
highest priorities on your project, you'll want to know the build
status at any point in time. One way to constantly keep abreast of
build statuses is through the Short Message Service (SMS), which are
short text messages sent to mobile phones.

Fortunately, the same mechanism used to send e-mail can also be used to
send SMS text messages. For example, Listing 2 shows an example of
using the CruiseControl e-mail task to send an SMS message:

Listing 2. SMS sample code

<publishers> <email mailhost="localhost" returnaddress="buildmaster@localhost" returnname="Buildmaster" reportsuccess="fixes" subjectprefix="brewery" buildresultsurl=""> <failure address="7035551212@vtext.com"/> </email> </publishers>

Why so many feedback mechanisms?

When using CI, the criticality of a broken build is paramount and many
teams do not move
to other tasks until the integration build is successful again.
Accordingly, it's often effective to choose multiple mechanisms that
serve different purposes, such as SMS for those who aren't at their
computers often and X10 devices that enable people to know a build's
status in a glance. It is probably ineffective to use all
the feedback
mechanisms I cover here on a single
project-- choose what works best for you and change it up every now and again to add some spice to a project, so to speak.

I have configured the e-mail task in Listing 2 in such
a manner that I will receive text messages only when there is a build
failure, which is an effective means to reducing SMS messages.
Furthermore, the

reportsuccess="fixes"

attribute of the

e-mail

task indicates that you will receive a follow-up text message when the build has been fixed.

Figure 4 shows an example of the message sent from the CruiseControl server when a build fails:

Figure 4. Text message on your phone -- has your build failed?



Using SMS to send text messages can be an effective form of feedback
for urgent build status messages. Generally, it is not recommended that
you send text
messages every
time a build occurs (especially if you pay per
message!). Remember, too much information can become "noise" that
reduces the effectiveness of feedback, so choose wisely.

Back to top

Excite things with X10

E-mails work well and SMS is great when you're away from the office, but what if you just want to glance at
something
to determine a build's status? Luckily, there is an
entire cottage industry around programmable devices, dubbed X10
devices, which can provide a more passive form of feedback when hooked
up to CI systems. With X10-enabled devices, you can control most any
electrical device that can be enabled using special hardware that can
be manipulated through the X10 protocol. This means
that every day devices such as lights can be used to notify us of a
build failure.

For instance, Listing 3 offers an example of the code required
to plug an X10 device into CruiseControl's publisher mechanism. This
example demonstrates turning on a red warning light if the build fails.
The light is connected to an X10 controller (several "X10 kits" are
commercially available). CruiseControl includes the Java™ COMM API so
that you can use CruiseControl's config.xml to communicate with the
device and turn it on or off. The X10 controller will connect to a
communications port on your build machine; the

port

attribute indicates the communications port you are using. The

houseCode

and

deviceCode

attributes refer to the letter and number you set on the X10 controller for the device. The

onWhenBroken

attribute indicates that the device will turn on when the build fails (and off when successful). Finally, the

interfaceModel

attribute refers to X10 computer interface. The default is

CM11A

.

Listing 3. X10 configuration code for CruiseControl.

<publishers> <x10 port="COM1" houseCode="A" deviceCode="3" onWhenBroken="true" interfaceModel="CM17A"/> </publishers>

Now, when this publisher is utilized, a device, such as the one shown in Figure 5, will turn on when the build is broken:

Figure 5. an X10 red warning light



There
are a variety of X10 devices -- LavaLamps, sirens, you name it -- that
can really enliven a project's environment. Adding these devices to a
room is bound to create excitement, along with a clear indication of a
build's status.

Back to top

Don't forget about monitors and IM

If you'd rather receive feedback instantly
,
have a look at client monitors and instant messaging. A client monitor
is an application that is always running in the background and
periodically polls your build server for the latest build status. Of
course, everyone knows what instant messaging is, and a host of popular
platforms, from AIM to Yahoo to MSN, are available for this purpose.

Monitoring status with monitors

CruiseControl.NET
(a Continuous Integration server for the .NET platform) provides a
monitoring mechanism called CCTray that works with both
CruiseControl.NET and the traditional Java built CruiseControl and is
run from the Windows taskbar (which means it works only on Windows
machines). An example of CCTray in action is shown in Figure 6:

Figure 6. CCTray monitor from the Windows taskbar



You can use another client monitor called Nag on non-Windows machines and even on other CI servers, such as Apache Continuum.

Instant feedback with IM

You
can use instant messages (or IMs) to notify project members of the
status of the build too. Just like using a monitor, it's another way to
receive feedback quickly because of IM's ubiquity among developers.

CI servers like Apache Continuum provide a simple way to send IMs regarding a build's status. For example,
Figure 7 displays an example of a failed build message, from Continuum, which is sent to a Google Talk instant
message client using Jabber. CruiseControl also provides a Jabber publisher to send messages.

Figure 7. An IM signifying a failed build



Monitors
and IMs are some of the quickest ways to send build status feedback,
and I'm familiar with project teams that
exclusively use these mechanisms. As long as this is coupled with
another way to inform
project members when they are away from their desk (such as with e-mail
or SMS), this can be a highly effective form of feedback.

Back to top

A party of feedback mechanisms

As
you can see thus far, there are many types of feedback mechanisms
available for you to utilize in a CI environment. Many other feedback
mechanisms, which I did not cover, are available, including the Ambient
Orb, Web browser plug-ins, sounds, and widgets. Table 1 provides a
summary of the feedback mechanisms I covered in this article, plus a
few others I didn't:

Table 1. Continuous Feedback Mechanisms

Feedback Mechanism	Description
E-mail	Provides build status at discrete points in time
RSS	Pushes alerts regarding build status to an RSS reader
SMS	Alerts you to build status with text messages sent to your cell phone
X10	Provides feedback through visual means; popular examples include lava lamps and red flashing lights
Ambient Orb	Provides another visual means of assessing build status; can be customized for alerts based on specific information
Sounds	Alerts to build status through sound; especially useful for co-located teams
Displays	Provides feedback through an LCD monitor
Widgets	Displays the build status on the user's desktop; Yahoo! and MAC OS X provide Widgets for this purpose, including CruiseControl Dashboard for Java
Monitor	Alerts you to the build status from the Windows taskbar
Instant Message	Allows for instantaneous notification
Browser plug-ins	Notifies you to build status through the browser; similar to CCTray, a Firefox plug-in for CruiseControl displays a greed or red icon for success or failure, respectively

The most effective
way to proactively generate action is to use multiple feedback
mechanisms in concert. Choose the combination that's most appropriate
for you.

Back to top

Have fun with it!

Remember,
the purpose of build feedback is to initiate rapid action so that teams
can fix a broken build. Typically, you'll need a combination of these
feedback mechanisms to elicit this action. For instance,
you may provide e-mails for all build events, SMS for build failures
(and follow-up fixes), and an X10 device for visual notification. As
your team matures in using these devices, you'll want
to change it up every so often to add variety and keep people
interested in build statuses. Be creative and have fun with it -- it's
amazing how adding a few feedback gimmicks can enliven the mood and
generate interest in the information they are providing!

Back to top

Download

Description	Name	Size	Download method
CruiseControl configuration for e-mail and SMS	j-ap11146.zip	1800KB	HTTP

Information about download methods

Resources

Learn

Automation for the people

(Paul Duvall, developerWorks): Read the complete series.

"Bubble, Bubble, Build's In Trouble"
: Mike Clark, author of Pragmatic Project Automation
, demonstrates a detailed example of configuring an X10 device for CruiseControl.

"Jabber"
(Gerhard Poul, developerWorks, May 2002): Gerhard Poul shows how
XML-based Jabber fits into today's e-business infrastructure,
highlighting instant messaging in a whole new way.

"eXtreme Feedback for Software Development
"
(Alberto Savoia, DeveloperTesting, April 2004): Other examples of
Continuous Feedback Mechanisms -- or as the author describes them:
eXtreme Feeback Devices.

"Pragmatic Automation"
: Mike Clark, author of Pragmatic Project Automation
, covers various feedback devices to monitor build status.

"Continuous Integration
" (Martin Fowler, martinfowler.com, May 2006): A description of the purpose and practices of Continuous Integration.

"Configuring Continuum for CI
" (Testearly.com): An article that describes the basic setup and configuration of Continuum.

Ambient Orb Ant task example
(Testearly.com): Examples on using the Ambient Orb Ant task.

The Java technology zone
: Hundreds of articles about every aspect of Java programming.

Get products and technologies

CruiseControl
: An open source server for Continuous Integration.

4-Pack Firecracker Automation System
: Hardware to help configure
your build to interface with X10 devices.

Nag
: A Continuous Integration montior that works with multiple CI servers and platforms.

Yahoo! Widgets
: Search for CruiseControl widgets for feedback to your desktop.

Ambient Orb Ant task
: A freely available open source Ant task for changing Ambient Orb colors based on build status.