16

I looked at Tools for load-testing HTTP servers? but I couldn't see how to replay my own existing logs in any of those tools. I have a bug that only occurs under certain load operations which my existing JMeter and AB testing stuff can't reproduce.

I want to simply give the tool access_logs and have it play them back, either faster or at the same speed.

Community
  • 1
Stewart Robinson
  • 1,155
  • 4
  • 12
  • 24

5 Answers5

17

You can use Jmeter's Access Log Sampler component.

There's a short tutorial on it's use in this PDF.

Dan Carley
  • 25,797
  • 3
    I tried this and obviously I must be a moron because I couldn't get it to work well. I ended up using the python script at http://insom.me.uk/z/2009/02/10m-hack-replay-an-apache-log-file-and-graph-the-response-times.html – Stewart Robinson Nov 19 '09 at 10:12
  • Since the original link is dead, here is the link in archive.org: https://web.archive.org/web/20090305084634/https://www.insom.me.uk/hacks/hit-it.txt – Walty Yeung Dec 26 '17 at 05:54
  • Ok, it in case some others found this, this script has a problem with threading. One may need to replace thread lib with threading lib (https://stackoverflow.com/questions/19558401/python-join-all-threads-after-start-new-thread) – Walty Yeung Dec 26 '17 at 06:19
2

Assuming all you've got in your access log are GET requests, and you don't mind the requests bunching up at the limit of resolution of the timestamps in the logs, about 10 lines of $SCRIPTING_LANGUAGE should do the trick. POSTs, cookies, HTTP auth, and more subtle timing are a far more interesting exercise.

womble
  • 97,049
-1

That would require a program, such as a load-testing app, which supports http log replay. One such app is HTTPerf (https://github.com/httperf/httperf).

A how-to article is at https://www.igvita.com/2008/09/30/load-testing-with-log-replay/

Barry G
  • 1
  • 1
  • 1
    This would be a much better answer if it included an actual explanation. Simply linking to an external web site is discouraged here, as web sites can and often do disappear. – Michael Hampton Mar 27 '15 at 18:07
-2

Why don't you develop your own? Get the log; parse it. Get the URI. Make curl call. You can write this in php and run it in apache for concurrency.

If your are logs are in gz format, do a zcat, use readlog facility. That will give URL. Now use phpCurl to hit the URL. For higher throughput run it in apache (use ab to load it).

Deer Hunter
  • 1,080
Prashant R Kumar
  • 1
-2

I would do something a little differently. I do understand your question, but if your looking at loading up your server you might want to look into the 'ab' tool. It comes with most installations of apache. Running:

ab -c 15 -n 1000 http://site.name/

Will perform 1000 requests doing 15 requests at a time. I know this isn't exactly what your looking for, and this will only query the one address you give it. If you need load this is a quick way and simple way to do it, and it will give you some potentially very useful statistics for debugging.

TrueDuality
  • 1,894
  • 5
  • 27
  • 37
  • If the question says "I have a bug that [...] AB testing [...] can't reproduce", why would you suggest using AB? – womble Nov 12 '09 at 14:40
  • I missed AB listed in the tools. :p I'd love to say it was an edit but I'm honestly not sure myself. – TrueDuality Nov 13 '09 at 18:31