LoadRunner 同开源性能测试工具 OpenSTA 的比较

关键字：LoadRunner 开源性能测试工具 OpenSTA 性能测试工具比较

前端时间做评估的时候找到的，有兴趣的朋友可以了解一下。

两篇，第一篇来自 testingreflections.com ——测试大师云集的地方 ^_^

Item	Description	LoadRunner	OpenSTA
Protocols	The communication protocols that can be captured, manipulated and replayed by the tool.	Many supported. Protocols are charged per item. Has a multi-protocol recording feature.	HTTP 1.0 / 1.1 / HTTPS (SSL) only.
Playback functions	Replaying of the script and script debugging facilities.	Extended logging supports view of parameter values and Server messages. Also view and comparison with ‘recorded’ version of web page view and client response messages. Debugging facilities in script generator, step and breakpoints.	Similar playback facilities, but no integrated comparison function. The debugging functions are in the controller, including set break points and single stepping.
Scripting language	The medium used to represent the captured protocol data and manipulate the data for play-back.	Called TSL, it uses standard syntax for “C” & allows C function libraries to be added. Has extensive customised functions for the different protocols supported by the tool.	Called SCL, it uses a proprietary, “BASIC” like language that has special automation scriptingfacilities. Is limited in available functions, such as string manipulation and supports direct DOM addressing.
Extensibility	The ability to increase the functionality of the tool.	Additional TSL or “C” function libraries, limited to functional capabilities of the tool.	SCL script modules can be defined in â��Includeâ�� files. Open Source therefore new tool functionality can be added using C++.
Scripting Interface	The interfaces supplied by the tool application for the purpose of script editing.	Captures in several modes, high level context based and low level HTTP view. Has both a graphical tree structure & a script view. Script view has function sensitive entry.	Has low-level HTTP protocol view and provides graphical tree representation of the �Document Object Model� (DOM) structure. Viewable captured HTML rendering and addressable server-header table. Some language sensitive, syntax colour coding functionality.
Correlation	The task of substituting values in dynamic data to enable successful playback.	Automated correlation faculties. Including during recording, after recording and comparing recordings with playback results. Not available for all modes of capture.	Manual correlation using graphical interactive �DOM� structure. Facility to automatically generate script code to aid variable substitution.
Cookie Management	Detection, recording and playback of HTTP cookies. Both tools need additional code to manage JavaScript generated cookies.	HTTP header cookies are managed automatically and can be manipulated manually if required.	HTTP header cookies are managed automatically and can be manipulated manually if required.
Parameterisation	Automatically changing dynamic data values for more accurate emulation of real users. Often essential for session management.	Extensive facilities for data entry including wizard interface to DB interrogation. No standard function to lock data sources and maintain uniqueness of concurrently accessed data across distributed tests.	Extensive facilities for data entry including wizard interface to automatically generate test data. Standard functions for sequential, random and pseudo-random data-file access. Has standard common locking facilities for maintaining uniqueness of parameters for an individual load injector or across all injectors on a distributed test.
Controller	Application that manages and conducts a test.	Facilities for real-time monitoring. Automatic scenario generation. Individual control of vusers, scripts and groups of scripts. Scheduling, percentage runs & iterations.	Facilities for real-time monitoring. Simple drag & drop multi-scenario test configuration supporting modular scripting and addition of new scenarios/Vus at run-time. No automatic scenario generation. Allows test-wide and user-specific HTTP monitoring and debugging while multi-user load in progress.
Monitoring	Resource usage information is captured during execution. It can be shown during execution and used to build performance reports.	Many supported. On-line graphics during execution. Supplied with Apache, Netscape & IIS, other monitors are charged per item. Results used for reporting. New facility to allow remote users to monitor real-time results via a browser based interface. Note: To monitor through a firewall requires TCPIP access through a specific port. Future versions of LoadRunner should use HTTP messaging & avoid this issue.	Supports Integrated real-time, graph view of Windows NT/2000 Performance (Perfmon) & SNMP collectors. Various measurements of test progress including VU-specific, custom status and activity information. Web-relay allows monitors to run on remote machines beyond firewall. On-line graphics during execution, & monitored results used for reporting.
Distributed tests	The ability to distribute the generation of load across multiple load-injector machines.	Supports multiple load-injectors managed by a single controller.	Supports multiple load-injectors managed by a single controller. Uses TCP/IP if on the same network or the Web-relay feature uses HTTP to control load-injectors located within remote DMZs.
IP Spoofing	The ability to emulate the behaviour of different IP addresses accessing a system. Particularly useful with Load balancing systems.	Supports IP spoofing including automated router updates for IP forwarding.	No inbuilt features to enable IP spoofing.
WAN/LAN emulation	The ability to emulate the behaviour of different network infrastructures during a test.	New feature added to version 7.6. Allows the emulation of latency, packet loss, link faults & dynamic routing effects over the LAN used in a test. Requires a special license.	No inbuilt features to enable WAN/LAN emulation.
Caching	The ability to emulate the caching of pages as performed by a web browser.	Can control browser cache emulation during playback and control setting for each individual vuser.	No specific faculties, although can be emulated with simple script code.
User connection speed emulation	The ability to emulate the different network speeds that can be used by real users.	Can emulate different network speeds during playback	No inbuilt features to emulate user-connection speed emulation.
Distributed/Remote load generation	To generate large loads additional load generator machines are required. These machines must be centrally controlled.	Can control multiple load generators and collect results.	Can control multiple load generators and collect results.
		Can control load generators located at remote networks (through a firewall) if required using HTTP ports.	Can control load generators located at remote networks (through a firewall) if required using HTTP ports.
Reporting & analysis	The facilities to examine and investigate the results of a test including timers and monitored resources and display the results in graphical format.	Sophisticated & large range of charts and graphs with overlay facilities. -Automatic report generation into MS-Word. Analyser is a separate application that can be distributed to users.	Simple charts and graphs sufficient for analysing key load related statistics and resource usage monitors. Resource usage monitors supports graph overlays. Can be exported to Microsoft Excel. No license restrictions on OpenSTA distribution thus stats can be viewed by any user with access. -Free tools and excel macros available through public user-forums.
Scalability	The ability of the tool to generate numbers of virtual users and the corresponding resource usage. Actual resource use depends on the number, size and complexity of the scripts.	Resource limitations are number of threads and RAM. Approx 1 vuser per ½ Mb RAM for NT/W2K. Windows 95, 98 & Unix are less efficient. Max. approx 1,500 VUs per PC.	Main resource used is RAM. Tested for simple ASP pages, reaching up to 3,000 users on load generator of 1Gb RAM on a single P4 processor & W2K. Unconfirmed report of a limit of 1664 Vus per Win2K machine for complex scripts. Suspected thread limitation. No licensing limitations.
Initial Costs	Purchase costs of software and licenses, excluding upgrades or support.	From £16,000 for basic package with no virtual users. Additional charge for each protocol and monitored resource and for virtual users.	FREE� download from SourceForge via www.OpenSTA.org. Downloads available: Previous versions; Automatic installer or current source code (with simple build instructions for MS C++ Visual Studio 6).
Costs of Virtual users	Most commercial tools charge on the basis of the number of virtual user available. Extra hardware is an additional cost.	Prices vary but for the purpose of expectation: additional vusers cost from £10K for 25 VUs to £66K for 1000 VUs. Also temporary VU days at £3.50 each per day (min 1000). This is not a quoted price.	FREE. No licensing limitations.
Support & consultancy	The support services available for the tool and some costs.	By M.I., approx 1/5 of initial cost per annum. Includes upgrades. MI & their partner companies also provide consultancy (including etest associates).	Various independent resources. etest associates charged from £50 per incident for remote tech support. Consultancy rates available on request.Numerous online resources including web and e-mail forums. Upgrades are free (approx every 3-6 months)
Training	The training services available for the tool.	MI has a range of courses from around £400 per day per person. Many partners also provide training.	Specialist companies provide tailored training, prices vary.
System requirements	The operating systems required to host the tool. (Not the OS of the system under test).	MS windows 2000, NT4 (sp6a),XP-Pro (also 95 & 98 cannot run multi-threaded) - Load generators also support limited generator functionality on: Unix: HP, Solaris, Linux	MS windows 2000, NT4 (sp5+), XP-Pro.
Hardware requirements.	The hardware requirements to host the tool. (Not the hardware of the system under test).	Min: Pentium 350 & 128M - Load generators: Pentium 1GHz & 1 MB per VU.	Min : Pentium 200 & 80MB RAM. Pref: Pentium 500MHz+ & 128MB+ RAM.
Access to source code	Availability of the source code of the tool itself.	Unavailable.	Open Source GNU public license - ‘C++’ Language.
A users opinion.	Opinion of Internet performance engineers who have experience with both tools.	Has a very user friendly user interface and fantastic monitoring & results analysis. Automatic correlation and improved script recording facilities can aid productivity. Very flexible scripting functions & great help documents. Complexity of options and layout of controller are drawbacks. The availability (at a cost) of numerous protocols for recording is a significant strength.	Easy to use interface and excellent scalability. Inbuilt results analysis is slightly limited in comparison to LoadRunner. The captured data is open and can also be exported to Excel in two clicks. Drag & drop scenario set-up and control is very intuitive and easy to interact with, simplifying creation of scenarios with modular scripts. Manual correlation can be a headache but is eased slightly by using third-party ‘Diff’ tools and the inbuilt GUI DOM addressing feature. A relative shortage of standard script language functions but enough to get most HTTP load testing jobs done. If not, with “Includes” support and the availability of the source code, it is very extensible

第二篇来自 AdventNet 公司的一份评估报告，宣传自家产品的意味比较浓。

Features	Comparisons
	QEngine	LoadRunner	OpenSTA	OpenLoad
Cost / License	Pricing starts at $599	Pricing starts at $38,000 for 25 virtual users. ($22,000 for Controller and $16,000 for 25 users.)	Free	Express - $4,995 Advanced - $37,995
Top
Platform Support
Platform Support	Windows, Linux	Windows	Windows & UNIX	Windows, Linux & Solaris
Top
PlayBack
Session Tracking (Cookies / URL redirection)	Yes	Yes(Additional code for JS)	Yes(Additional code for JS)	Yes
Page Validation (String/RegEx)	Yes	Yes	no integrated comparison function	Yes
IP Spoofing	Yes	Yes, with automated router updates for IP forwarding.	No inbuilt features	Yes
Server Monitoring (WMI/SNMP/RSTAT)	Yes	Yes	Yes	Yes
Simulate Multiple Browsers	Yes	Yes	Yes	Yes
Scalability	1000VU on a P4, 512 MB RAM 1.6 Ghz CPU	Approx 1 VU / ½Mb RAM for NT/W2K. Windows 95, 98 & Unix are less efficient. Max. approx 1,500 VUs / PC.	Simple pages - up to 3,000 VUs 1Gb RAM, P4, W2K. Complex scripts - 1664 VUs W2K	large workloads- 1000 VUs
Distributed tests	Yes	Multiple load-injectors managed by a single controller.	Multiple load-injectors managed by a single controller. TCP/IP - same network or Web-relay feature HTTP - Remote DMZ	---
Distributed/Remote load generation	Yes	Yes, can control multiple load generators and collect results.	Yes, can control multiple load generators and collect results.	---
Dynamic Load (Virtual User Rampup)	Yes	Yes	Yes	Yes
Simulate Multiple Connection Speeds	Yes	Can emulate different network speeds during playback	No inbuilt features	Yes
Correlation (substituting values in dynamic data while playback)	Yes (DB, CSV)	Partially supported	Yes	Yes
Caching	Partially supported	Yes	Partially handled	---
Reporting & analysis	Yes - Graphs and Charts	Yes - Graphs and Charts, MS-Word	Yes - Graphs and Charts, MS-Excel	Yes - Graphs and Charts, Text/Spreadsheet/PDF
Basic/Digest/NTLM Authentication Support	Yes	Yes	---	Yes
Recording
Protocol	HTTP,HTTPS	Many. Protocols are charged per item. Has a multi-protocol recording feature.	HTTP, HTTPS (SSL)	HTTP, HTTPS (SSL)
Scripting language	Jython	TSL, uses standard syntax for "C" & allows C function libraries. Customised functions for different protocols	SCL, "BASIC" like language, has special automation scriptingfacilities. limited functions available	Script Free recording and playback
Scripting Interface	Yes	Yes	Yes	No
Parameterisation	Yes	Yes	Yes	Yes
Ajax request recording	Yes	?	?	?