未加星标

Open Source Load Testing Tool Review

字体大小 | |
[开发(python) 所属分类 开发(python) | 发布者 店小二04 | 时间 2017 | 作者 红领巾 ] 0人收藏点击收藏

(Version 1.0)

Benchmarking & Comparing Open-Source Load Testing Tools

Ragnar Lnn


There are tons of load testing tools, both open- and closed-source. Open-source tools are growing in popularity, and we use mainly open-source software (OSS) at Load Impact, so we thought it might be useful to take a deep look at the available OSS load testing options.

OSS load testing options represent kind of a jungle, and the difference between one tool and the next in terms of usability, performance, feature set or reliability can be enormous. Thus, we’d like to help people find the right tool for their use case.

We have chosen to look at what we consider to be the most popular, open-source load testing tools out there today. This list includes:

Jmeter Gatling Locust The Grinder Apachebench Artillery Tsung Vegeta Siege Boom Wrk
Open Source Load Testing Tool Review

This review contains both my own personal views on the good and bad sides of these load testing tools, and my thoughts based on a round of benchmark testing that gives a sense about relative tool performance.

If you are mainly interested in the results from the benchmarking, they will be posted in a followup article very soon :)

So ― what's the setup?


Open Source Load Testing Tool Review
Setting up the Open Source Load Testing Review

We installed, configured and ran these tools from the command line, plus spent a lot of time trying to extract results from them using shell script and standard Unix tools. Then we have tried to figure out the relative performance of the tools. First, by manually trying to squeeze the best performance out of each one, optimizing configuration parameters for the tool in question, and then by running a benchmark test on all the tools, trying to run them with as similar configuration as possible.

The benchmark numbers are one thing, but the opinions on usability that I give will be colored by my use case. For example, a tool that is hard to run from the command line, or that is hard to get useful results from when you run it that way, will make me frustrated. And when I’m frustrated, I’ll have to whine and complain about it. So that is what this article is all about.

One big caveat: I have not been looking much at the data visualization options included with each tool, so you may want to see this comparison as most useful in an automation setting, or if you are planning to push results into some external data storage and/or visualization system anyway.


Open Source Load Testing Tool Review

Try it yourself: run the Docker image

To make things easy for people, I’ve created a public Docker image that can be used to easily repeat all the tests we have made (or to just run one of the load testing tools without having to install it yourself). Try this:

<span style="font-family: 'courier new', courier;"><i><span style="font-weight: 400;">&nbsp; &nbsp;docker run loadimpact/loadgentest</span></i></span>

Or, if you want to be able to simulate extra network delay you should do:

docker run --cap-add=NET_ADMIN loadimpact/loadgentest


Open Source Load Testing Tool Review

You can also build the Docker image yourself, if you clone our Github repo:

git clone https:// github.com/loadimpact/loadgentest

At https://github.com/loadimpact/loadgentest you will also find some documentation on how to use the setup, by the way!

So ... Which tool is best?

To go against all common wisdom regarding how to keep visitors reading your article, I’ll go ahead and just say: Gatling! Which is funny, because a month ago I would never have said that, given that Gatling is kind of a modern Jmeter, and I usually hate Java applications.

So Gatling is number one, the best, overall winner, etc. Of course... if you’re into python you may be better off with Locust. Oh yeah, and if you wanthigh performance, Wrk would be a better option. However, Wrk is scriptable in Lua, not Python, and has a callback-based API that is not as nice as Locust’s.

If you just need to hit a single URL and nothing else, Apachebench may be the way to go. With Apachebench, you won’t get distracted by all its features because it can literally not do anything but hit one specific URL.

Actually, I think I’ll withdraw that single recommendation. Instead I will say which tools I think have something going for them, and why, and then I’ll show you some benchmarking figures and let you decide for yourself, because one size really doesn’t fit all here.

The top 3open source load testing tools
Open Source Load Testing Tool Review
Gatling ( http://gatling.io )

Apart from maybe its text output to stdout (which is as messy as Locust’s), Gatling is a very nice load testing tool. Its performance is not fantastic, but good enough. It has a consistent design where things actually make a little bit of sense, and the documentation is very good.

It has a DSL (domain specific language) more or less equal to what Jmeter and Tsung offer. However, while Jmeter and Tsung use XML with their specific tags to implement stuff like loops, Gatling lets you define Scala classes that offer similar functionality but that are a lot more readable.

Yes, Scala . Apparently everyone reacts the same way upon hearing that. Gatling’s documentation specifically tells you not to panic, which of course made me panic, but once I realized I couldn’t get out of this and bothered to read the docs and look at the examples I saw that it’s not tricky at all.

My initial assumption was that Gatling would be as clunky to use as Jmeter since it’s a Java app and with DSL functionality similar to Jmeter. But I have now seen the error of my ways and while I still very much prefer to use a “real,” dynamic scripting language to define what happens in a load test, Gatling’s DSL is perhaps the second best (to a real, dynamic language) thing around. Here is how it can look:

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._

class GatlingSimulation extends Simulation {

val httpConf = http

.baseURL("http://myhost.mydomain.com")

.disableCaching

.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")

.userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8

本文开发(python)相关术语:python基础教程 python多线程 web开发工程师 软件开发工程师 软件开发流程

主题: DockerScalaJavaGitXMLLuaPythonADMHead
分页:12
转载请注明
本文标题:Open Source Load Testing Tool Review
本站链接:http://www.codesec.net/view/529755.html
分享请点击:


1.凡CodeSecTeam转载的文章,均出自其它媒体或其他官网介绍,目的在于传递更多的信息,并不代表本站赞同其观点和其真实性负责;
2.转载的文章仅代表原创作者观点,与本站无关。其原创性以及文中陈述文字和内容未经本站证实,本站对该文以及其中全部或者部分内容、文字的真实性、完整性、及时性,不作出任何保证或承若;
3.如本站转载稿涉及版权等问题,请作者及时联系本站,我们会及时处理。
登录后可拥有收藏文章、关注作者等权限...
技术大类 技术大类 | 开发(python) | 评论(0) | 阅读(14)