NuGet Perf–Results and Source Code
Join the DZone community and get the full member experience.
Join For Freethe test was run locally (no network involved ) on a lenovo w520 laptop with 8 cores & 8 gb ram with an ssd card. the storage engine we used was esent, safe transactions. default ravendb configuration, running in console, with logging disabled.
we took the most obvious approach both in the code we wrote and the test approach. i am pretty sure that i’ll get a lot of helpful suggestions about the load testing. the code is available here, and you are more than welcome to take it for a spin and get your own results. what is important for me to note is that we have done exactly zero performance tuning . that is relevant to both the index we use, to the code that we wrote, everything. i just wrote things down, and didn’t worry about performance, even though this code is going to go through a load test.
why don’t i worry about it? because ravendb is setup to do the right thing. it will self optimize itself without you need to take care of that.
with that said, here are the test results:
you can see that the red line is the number of users we have, and we have this worrying green line that seems to go crazy…
except that this is actually the number of page served. the part that we care about is actually the avg. page time, and that is the blue line.
this line , however, is basically flat no matter the load.
here are the test results in details
load test name | loadtest1 |
description | |
start time | 04/09/12 14:16:38 |
end time | 04/09/12 14:21:38 |
warm-up duration | 00:00:20 |
duration | 00:05:00 |
controller | local run |
number of agents | 1 |
run settings used | run settings1 |
max user load | 300 |
tests/sec | 20.0 |
tests failed | 0 |
avg. test time (sec) | 12.5 |
transactions/sec | 0 |
avg. transaction time (sec) | 0 |
pages/sec | 77.1 |
avg. page time (sec) | 0.0062 |
requests/sec | 77.1 |
requests failed | 0 |
requests cached percentage | 0 |
avg. response time (sec) | 0.0062 |
avg. content length (bytes) | 3,042 |
url (link to more details) | 95% page time (sec) |
---|---|
0.018 | |
0.018 | |
0.014 | |
0.014 | |
0.014 |
name | 95% test time (sec) |
---|---|
browsing | 19.3 |
browseandsearch | 17.6 |
10.6 |
name | scenario | total tests | failed tests (% of total) | avg. test time (sec) |
---|---|---|---|---|
browsing | load | 1,533 | 0 (0) | 16.0 |
browseandsearch | load | 1,685 | 0 (0) | 15.0 |
load | 2,770 | 0 (0) | 9.00 |
url (link to more details) | scenario | test | avg. page time (sec) | count |
---|---|---|---|---|
load | browsing | 0.0072 | 1,629 | |
load | browseandsearch | 0.0071 | 1,783 | |
load | browseandsearch | 0.0064 | 3,443 | |
load | searching | 0.0064 | 2,798 | |
load | browsing | 0.0063 | 1,617 | |
load | browsing | 0.0063 | 1,580 | |
load | browseandsearch | 0.0063 | 1,760 | |
load | searching | 0.0055 | 2,810 | |
load | searching | 0.0055 | 2,839 | |
load | searching | 0.0054 | 2,866 |
name | scenario | test | response time (sec) | elapsed time (sec) | count |
---|
machine name | % processor time | available memory at test completion (mb) |
---|
machine name | % processor time | available memory at test completion (mb) |
---|---|---|
13.0 | 1,356 |
type | subtype | count | last message |
---|
you can dig in and look at the data, it is quite interesting. under the load of 300 users, the average page response time was… 0.0062 seconds.
and ravendb was using just 13% of the cpu, and that include running the agents running the tests.
after seeing how well ravendb does in perf testing, i decided to take it up a notch.
- starting from 10 users, with a step duration of 1 sec, add 50 users for each step, all the way to 3,000.
- start with a warm up period of 20 seconds, then run the test for 10 minutes.
let us see what happens, okay?
just to be clear, this is a ravendb application running with three thousands concurrent users , on an off the shelve laptop while i was busy doing other stuff.
one word of warning before hand, because i run everything on a single machine, just running so many users on the machine significantly slowed down how ravendb is reacting. basically, the code for managing the perf test took so many resources that ravendb had to fight to get some to actually answer the queries.
scared yet, because here are the results in graph form.
now you can actually see that we have some fluctuations in the graphs, the number of users grows and grows until we get to 3,000 and we have 0.37 seconds response times.
again, i remind you, we have done zero optimizations and this is idiomatic ravendb code. and we were able to serve requests at a frankly pretty amazing rate of speed.
and here are they in their full details:
load test name | loadtest1 |
description | |
start time | 04/09/12 15:28:48 |
end time | 04/09/12 15:38:48 |
warm-up duration | 00:00:20 |
duration | 00:10:00 |
controller | local run |
number of agents | 1 |
run settings used | load |
max user load | 3,000 |
tests/sec | 196 |
tests failed | 0 |
avg. test time (sec) | 14.3 |
transactions/sec | 0 |
avg. transaction time (sec) | 0 |
pages/sec | 741 |
avg. page time (sec) | 0.37 |
requests/sec | 741 |
requests failed | 0 |
requests cached percentage | 0 |
avg. response time (sec) | 0.37 |
avg. content length (bytes) | 3,080 |
url (link to more details) | 95% page time (sec) |
---|---|
0.83 | |
0.82 | |
0.82 | |
0.82 | |
0.81 |
name | 95% test time (sec) |
---|---|
browsing | 20.8 |
browseandsearch | 19.8 |
12.9 |
name | scenario | total tests | failed tests (% of total) | avg. test time (sec) |
---|---|---|---|---|
browsing | load | 31,843 | 0 (0) | 17.4 |
browseandsearch | load | 33,989 | 0 (0) | 16.8 |
load | 51,650 | 0 (0) | 10.8 |
url (link to more details) | scenario | test | avg. page time (sec) | count |
---|---|---|---|---|
load | browsing | 0.40 | 32,338 | |
load | searching | 0.39 | 52,597 | |
load | browsing | 0.39 | 32,627 | |
load | browseandsearch | 0.39 | 68,576 | |
load | browsing | 0.38 | 32,803 | |
load | searching | 0.38 | 52,283 | |
load | browseandsearch | 0.37 | 34,766 | |
load | browseandsearch | 0.36 | 34,982 | |
load | searching | 0.35 | 51,991 | |
load | searching | 0.33 | 51,846 |
name | scenario | test | response time (sec) | elapsed time (sec) | count |
---|
machine name | % processor time | available memory at test completion (mb) |
---|
machine name | % processor time | available memory at test completion (mb) |
---|---|---|
85.4 | 1,203 |
type | subtype | count | last message |
---|
note that the reason fro the high cpu usage is that the tests and ravendb were running on the same machine.
the code for the entire series can be found here: https://github.com/ayende/nuget.perf
no, i’ll not do a similar sql version, if you want to, i would be very interested in seeing one, but that isn’t something that i intend to do.
yes, it is a simple and trivial implementation, but that was pretty much the whole point. being able to get to that scale without actually doing anything special is what we strive for in ravendb.
Published at DZone with permission of Oren Eini, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments