Comments on: Performance Testing – A Simple Plan http://bopl.samharris.us/2009/02/performance-testing-a-simple-plan/ It's not all caviar and baby wipes, mate Sun, 19 Dec 2010 05:28:30 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Colonel Nikolai http://bopl.samharris.us/2009/02/performance-testing-a-simple-plan/comment-page-1/#comment-6566 Colonel Nikolai Sun, 22 Feb 2009 21:00:55 +0000 http://bopl.samharris.us/?p=1115#comment-6566 A former employer sent me to an MIT-run performance boot camp a few years ago where we learned how to do performance optimization. The most important thing we learned was how to make an M/M/1 model of the system you want to test, which would allow you to literally clock a single path through the code (i.e. a single request) and arrive at how many of those requests you could do per second with the given hardware using a calculator. It was so accurate that a lot of other testing can be forgone if you start here. You still have to test how your app will react when it's "overloaded" and some other tests, too. But you can save a lot of time with M/M/1 modeling because you'll do less of them. A former employer sent me to an MIT-run performance boot camp a few years ago where we learned how to do performance optimization. The most important thing we learned was how to make an M/M/1 model of the system you want to test, which would allow you to literally clock a single path through the code (i.e. a single request) and arrive at how many of those requests you could do per second with the given hardware using a calculator. It was so accurate that a lot of other testing can be forgone if you start here.

You still have to test how your app will react when it’s “overloaded” and some other tests, too. But you can save a lot of time with M/M/1 modeling because you’ll do less of them.

]]> By: Paul http://bopl.samharris.us/2009/02/performance-testing-a-simple-plan/comment-page-1/#comment-6565 Paul Wed, 11 Feb 2009 06:20:34 +0000 http://bopl.samharris.us/?p=1115#comment-6565 Phil - excellent points, especially the first one. I wrote this up in part to prompt the person to realize that he couldn't just 'do' a bit of performance testing, so the mention of sizing was there as much to scare him as anything :) Phil – excellent points, especially the first one. I wrote this up in part to prompt the person to realize that he couldn’t just ‘do’ a bit of performance testing, so the mention of sizing was there as much to scare him as anything :)

]]> By: Phil http://bopl.samharris.us/2009/02/performance-testing-a-simple-plan/comment-page-1/#comment-6564 Phil Wed, 11 Feb 2009 02:48:25 +0000 http://bopl.samharris.us/?p=1115#comment-6564 Both bursty and spikey, that's very thorough and fits nicely with the boilerplate verbosity :-) In all seriousness: approach it a bit like a scientific paper, that doesn't mean you have to write in a dry manner though. Assume you are writing in order that a peer could reproduce your results. I'd add: - How you will produce the load? - What will you measure in order to determine the constraints? - What could mitigate the constraints, what are the trade-offs? Sizing, that's a slippery slope when a customer's workload could be vastly different from your application. But you already know that ;-) Both bursty and spikey, that’s very thorough and fits nicely with the boilerplate verbosity :-)

In all seriousness: approach it a bit like a scientific paper, that doesn’t mean you have to write in a dry manner though. Assume you are writing in order that a peer could reproduce your results. I’d add:

- How you will produce the load?
- What will you measure in order to determine the constraints?
- What could mitigate the constraints, what are the trade-offs?

Sizing, that’s a slippery slope when a customer’s workload could be vastly different from your application. But you already know that ;-)

]]> By: Paul http://bopl.samharris.us/2009/02/performance-testing-a-simple-plan/comment-page-1/#comment-6563 Paul Tue, 10 Feb 2009 17:08:57 +0000 http://bopl.samharris.us/?p=1115#comment-6563 It's good it calls that out - too many people confuse burstiness with spikiness, and we all know what happens then... It’s good it calls that out – too many people confuse burstiness with spikiness, and we all know what happens then…

]]> By: Andrew http://bopl.samharris.us/2009/02/performance-testing-a-simple-plan/comment-page-1/#comment-6562 Andrew Tue, 10 Feb 2009 17:05:45 +0000 http://bopl.samharris.us/?p=1115#comment-6562 It is an unnecessarily long doc. However I like it because it covers the all-important "burstiness test". It is an unnecessarily long doc. However I like it because it covers the all-important “burstiness test”.

]]>