The Facts behind TCP Offload Engine (TOE)

Thursday December 25, 2008

The Facts behind TCP Offload Engine (TOE)

Monday, 06 August 2007 05:42

Editor

By Larry Boucher

TCP Offload Engine (TOE), is gaining increased attention among users
seeking to get the most performance and scalability out of their
servers. Microsoft’s release of TCP Chimney Offload1 has
helped fuel this awareness. To date, the industry response to Chimney
TOE compatible products has been robust. As TOE enters the mainstream,
there is a desire to better understand what it is and its capabilities
from a basic framework perspective. First, we will look at what TOE is
and the factors that shaped its development. We will look at the
anatomy of a server and why TOE is of benefit in shaping performance
and scalability even in today’s multi-processor, multi-core server
architectures. And, with iSCSI’s
growing popularity and anticipated growth, we will discuss how TOE is
used today in iSCSI deployments and its significance going forward. We
will close with a discussion on bandwidth, specifically just how much
can be achieved with gigabit Ethernet and contrasting this with 10
gigabit Ethernet.

Introduction to TCP/IP Offload

TCP/IP (Transmission Control Protocol/Internet Protocol) is clearly
the defacto standard protocol used by computers throughout the world to
communicate. IP contains the necessary information to connect two
computers much like a phone number is the information necessary to
connect two phones. The network infrastructure uses IP to route data
from one computer to another and TCP guarantees the integrity of that
data. TCP breaks the data into segments for transmission, and
reassembles the data at the receiving end, making sure that the data is
in order and error free. TCP requires significant processing to
perform these tasks and because of this, by way of background, UDP
(User Datagram Protocol) was often used in the local area. Unlike TCP,
UDP does not guarantee delivery of the data, but it is much more
efficient than TCP. For that reason, many data intensive applications
made use of UDP, resulting in a much more efficient transfer of data in
noise-free environments. However, UDP failed miserably in noisy
error-prone environments. As the Internet grew, it became increasingly
difficult to differentiate between data that could be isolated to the
local area and data destined for the wide area network. As the
processors became more powerful, TCP became the standard method for
transporting data for both the local and wide area network.

The availability of TOE has largely negated the need for lightweight
protocol alternatives to TCP. TOE offloads TCP processing from the CPU,
freeing it to do more of the tasks that actually generate the data to
be transferred. TOE is frequently considered to be valuable only when
the CPU is not sufficiently powerful to saturate the network
connection. While TOE is certainly useful in this regard, there are
many other reasons one might want to specify TOE for their servers. To
better understand this, we first need to delve a bit further into the
anatomy of the server.

The Anatomy of Today’s Server: From TCP to TOE

The Internet has driven major changes in the appearance of the data
center. Prior to the Internet, most of the data stored on disk drives
was “data processing” data. This data had no use until it was further
processed by the CPU. This might include word processing data,
database information, etc. All of this data was sent to the CPU,
processed (sometimes including interaction with the user) and then
returned to the disk drive. With the advent of the Internet and HTTP,
the amount of data stored on disk drives has grown by leaps and bounds.
All of this data is designed exclusively to be viewed, that is it is
not data that requires processing by a CPU. In addition to driving this
huge growth in data, the Internet has driven equivalent growth in the
data delivery infrastructure. Today, we move orders of magnitude more
data than we did prior to the Internet. TCP protocol processing has
grown proportionally with data growth as TCP is responsible for
transporting this data. So while CPUs have gotten faster, we now have
such a large amount of TCP to process that a more efficient method than
a general purpose processor is needed. TOE is a method that can save
significant cost and energy, both in power and human resource.

A generally understood rule of thumb is that it takes one CPU
instruction per bit to process TCP. Stated differently, a one
gigahertz CPU would be saturated as it tried to process a one gigabit
Ethernet connection. Or, a three gigahertz processor would use 33
percent of its capability to support the same Ethernet connection. If
this capability could be returned to the CPU, it could support 33
percent more users before it ran out of gas. TOE can allow the entire
computer, including the processor, associated storage, etc. to support
significantly more users.

A different way to look at the value of TOE is to consider the
function the server is performing and how the application running on
the server does its job. For instance, an efficient file server
application will spend much more time processing metadata than it will
to process TCP protocol. By utilizing TOE, the processor can process
twice as much metadata, and, as long as sufficient network bandwidth
exists, the server can support twice as many clients. Backup can be
made even more efficient than a file server application, due to the
orderly processing of data and the efficient metadata processing that backup allows.

With the introduction of multi-processor and multi-core
architectures, there is a common misperception that the need for TOE in
gigabit environments has passed. After all, if one has spare processor
capacity to throw at TCP/IP processing, why invest in TOE? It is
certainly true that if a server has substantially more capacity than is
required to provide the needed functionality, adding more capability
would be unnecessary. However, for all applications where multiple
servers are needed to provide the required function, TOE can
significantly reduce the number of servers required. Many large I/O
intensive jobs fall in this category including file serving, web
serving and backup applications. For these applications, TOE can
provide significant power, space and capital expense savings. TOE
provides value for any application that requires significant TCP/IP
processing. Many of these applications result in 50 percent or higher
total CPU utilization when allocated to TCP.

Implications for iSCSI and the Path to the Enterprise

As more and more users move to iSCSI for their block-level storage
requirements, the requirement for TCP processing is driven to a new
level. When iSCSI was first introduced, many in the storage community
dismissed it, largely based on the fact that the TCP processing burden
on the CPU would keep it from performing adequately. Today, in many
areas where iSCSI has been introduced, the processor itself is
sufficient to provide excellent performance. However, as iSCSI moves
to the enterprise level, TOE will be required.

Consider an enterprise-class file server head connected to a large
storage array via iSCSI. With each client request for data, if the
data is not cached, there is twice the TCP processing required compared
to that of a server connected via Fibre Channel or directly to a set of
disk drives. This file server head can easily double its ability to
supply data to clients by adding TOE capability. For a large data
center with significant file serving requirements, TOE can provide huge
savings.

While Microsoft started out providing a desktop operating system,
over time it has also provided powerful operating systems that are
becoming a significant factor in the enterprise server space. Just
witness the market share and growth that Microsoft has captured in the
database space with SQL. With the addition of TCP Chimney Offload to
Windows, Microsoft is providing levels of performance not available in
UNIX or Linux. This will accelerate the adoption ramp of Windows and
iSCSI in the enterprise. While it is unlikely that competing operating
systems will wait too long to add this functionality in order to
maintain their competitive position, Microsoft has already gained a
large lead and will undoubtedly be able to capitalize on it.

TOE Adoption Ramps Up

Today we are seeing servers with TOE capability built into the
server motherboards. Large server manufacturers are making it widely
available to customers as they clearly understand the benefits of TOE
and the enterprise performance benefits when combined with Windows TCP
Chimney Offload. While this currently requires a separate integrated
circuit on the board to provide the function, this capability will soon
be included in the Southbridge chip2, which will further
significantly reduce the cost to the end user. Systems designers are
seeing the benefit of TOE and its importance to server scalability and
performance, and recognize its ability to improve the overall
efficiency of the server.

Thoughts on Bandwidth

As servers become larger and processors become more powerful, 10
gigabit Ethernet (10GbE) will become increasingly important. Today,
however, most servers can only make use of a few gigabits per second of
bandwidth, even if the TCP processing requirement has been removed.
Through the TCP-provided port aggregation functionality, a few gigabit
Ethernet connections (which can be obtained on a single PCIe card) can
supply all the capability that can be achieved with a single 10 gigabit
Ethernet. For the next few years, until the price of 10GbE comes down,
this combination of aggregated gigabit connections and TOE will provide
the most cost-effective way of moving large amounts of data. We expect
that TOE and iSCSI will continue to be the fastest growing new
technologies in the networking space.

In summary, TOE can add tremendous value to users seeking to get the
most performance out of their servers. Driven by the growth of the
Internet, the load on the server has changed. With the addition of
TCP/IP processing and the adoption of TOE, valuable cycles from the
server’s CPU can be reduced, allowing the server to support other
functions (such as metadata processing in the file server) and enabling
it to scale and support more users. Even in today’s multi-processor,
multi-core server environments, TOE provides significant benefits. The
adoption of iSCSI will continue to increase, fueled in part by
Microsoft’s continued success in displacing UNIX servers in the data
center as well as the attractiveness that iSCSI offers in price,
performance, and simplicity. TOE will increasingly be integrated into
servers in the form of dedicated chip sets laid down on the motherboard
(LOM), and eventually integrated with the Southbridge itself;
supporting front-end, client-facing and back-end, iSCSI SAN-facing TCP
offload. Lastly, users do not need to immediately jump to 10GbE in
order to achieve higher bandwidth. While 10GbE will certainly
contribute to iSCSI adoption, users can benefit from high bandwidth
gigabit performance today through the use of multi-port PCI-e TOE
adapters that support link aggregation.

Larry Boucher is founder, president and chief executive officer of Alacritech.

www.Alacritech.com

1TCP
Chimney Offload is the process of offloading the TCP protocol stack in
Microsoft Windows to a network interface card (NIC) while preserving
customer investments in applications, manageability and security.

2
The Southbridge is a chip that implements the "slower" capabilities of
the motherboard in the construct of a Northbridge/Southbridge
architecture delineation. The Southbridge is generally distinguished
from the Northbridge by not being directly connected to the processor.
Rather, the Northbridge ties the Southbridge to the CPU.