jmeter-user

[Prev] Thread [Next]  |  [Prev] Date [Next]

Re: Variances between automated and manual tests Andrew Melnyk Tue Jun 23 09:01:34 2009

Hi Kevin,

There are several reasons why you can see such picture:

1.) Your jmeter scripts don't do what they intended to do. Make sure that
you have implemented in your tests assertions which guarantee that tests
doing right things.
2.) Jmeter not a browser so if there is a rendering of content on the client
side it does not measured by Jmeter.

Regards,
Andriy

2009/6/23 Badeau, Kevin (DPH) <[EMAIL PROTECTED]>

> Hello folks,
>
>
>
> We are using jMeter to capacity test an application we are considering
> purchasing.
>
>
>
> When we ramp up to 500 concurrent users jMeter is reporting response times
> under 5 seconds and it appears it is stepping through all the functionality
> we are asking it to do.
>
>
>
> This is very acceptable for us.
>
>
>
> However, while the test is running we try to hit the application manually
> and we find it is unresponsive.
>
>
>
> Manual testing is quick outside of a concurrent jMeter test running.
>
> Manual testing performance degrades as we in range from 100 to 200
> concurrent users.
>
> Manual testing is unresponsive when we run 500 concurrent users.
>
>
>
> jMeter reports response times only degrade by about a ½ second for each
> level of concurrent users we try.
>
>
>
> There seems to be some wide performance variance from what jMeter is seeing
> vs. what we see manually.
>
>
>
> I'm wondering if anyone has any general suggestions as to why this might be
> or how we might go about isolating this anomaly.
>
>
>
> I can provide more specific details if needed but I feel the question is
> pretty basic in terms what we understand the tool is supposed to be
> accomplishing in simulating real world scenarios and benchmarking them.
>
>
>
> Thanks in advance.
>
>
>
> Kevin
>
>