Load Testing The WinFrame Server
Using LoadRunner & WinRunner
This is my second take on the subject of load testing the Citrix WinFrame server
using Mercury Interactive's LoadRunner and WinRunner tools. For a brief background
and strategy in performing load testing on WinFrame server, please read my page
on Stress Testing the WinFrame Server. A lot of the
strategy and methods explained in that page will be applicable in performing
capacity testing as well.
Capacity vs. Stress Testing
Although both stress and capacity testing are considered load tests, they each
have different objectives. Stress testing is useful in determine if the server
can handle peak usages, which will be (rather unrealistically, some may say)
simulated by having several users perform the same action at the same time.
On the other hand, capacity testing is more designed to test the system's ability
to handle the average day to day transactions. In real life, there is very little
probability that all of the users will be performing the same action at the
same time. The more likely scenario will be where different users performing
different actions, or no actions at all (i.e. when reading the generated result).
So capacity testing has less peaks, but generally has a lot of transactions
taking place continuously.
The stress testing module that I have created can be used in performing capacity
testing, if so desired. However, you will be restricted in having all of the
WinFrame sessions in one host performing the same actions. You can use the _delayed
functions to delay the action between each WinFrame sessions, but you will still
have 10 users doing the same thing one after the other. Doable, but we can do
The capacity testing module (modules, actually) is designed to allow you to
perform a variant of the old timeslicing method of the old operating systems.
It allows the testers to have each WinFrame session's actions be independent
of the other sessions on the same host. This will allow for a more realistic
simulation of multiple users on the system, each perfoming independent actions
at independent times.
Features of the Capacity Module
Here are the highlights of the capacity module:
- Allows for control of multiple WinFrame sessions. The limit of sessions
per host is limited by the CPU, memory, and how sensitive the tester is with
regard to lags between actions. Ten sessions seem to be the ideal for a Pentium
II 266 machine with 64MB of physical RAM.
- Allow for each controlled WinFrame session in a host to have it's own separate
script, independent of what the other WinFrame sessions.
- New feature allows transactions in each WinFrame session to be timed independently
of the other windows. The module achieves this by appending a unique identifier
at the end of the transaction name, "[#]", where # is a number identifying
the index of the WinFrame session (starts at 0). At the end, if you want to
combine all of the transactions together, you can modify the result file by
eliminating the "[#]" strings.
- A special rendezvous function allows for several WinFrame sessions within
a host to synchronize to a rendezvous. However, you can not set a rendezvous
between a subset of a host's WinFrame sessions with another subset of another
host's WinFrame sessions. It only works within a host. If all of the WinFrame
sessions in a host meets at a particular rendezvous, then the function will
call an actual rendezvous to the LoadRunner controller. This is the only way
that you will be able to synchronize rendezvous between hosts: rendezvous
all of the sessions.
Are There any Drawbacks?
So far, so good. But there are several drawbacks that you should be aware of;
some of which is inherent to the way that the module controls multiple windows:
- There will always be a time lag between the time that a window/WinFrame
session receives the focus, and the time that it receives the next focus.
This is due to the fact that WinRunner has to cycle through all of the WinFrame
sessions, sequentially. The module does not have any sophisticated priority
management that a modern operating system does, remember that.
- Rendezvous between a subset of a host's WinFrame sessions with a subset
of another host's WinFrame sessions is not allowed (see above).
- Memory requirement might be slightly larger if complex scenarios are used.
Read this in more detail later on. However, in general it would help to have
as much memory as possible when running multiple sessions, since memory swaps
will noticably slow the window manipulations.
- Harder to debug and program (yes, I said program). I pretty much had to
create a mini TSL (Test Script Language) derivative, and the corresponding
interpreter to run it and manage the timeslicing. Unfortunately, there is
no parser to validate the script before it is ran. Fortunately, there is a
way to debug "most" of the script. Also, loops are a little harder
to implement, and I had to resort to using "goto" like statements
(I could hear some gasps). Conditional statements are also a little kludgey
(if there is such a word) to implement, but very doable.
However, once you get used to the custom functions (very few, and easy to pick),
and the way that the module executes the script, then you'll find that this
is a very flexible and powerful module to work with, and will help make your
testing task easier.
You should follow the first part of the stress testing section's Tips
Tech Page @