Load Testing The WinFrame Server
Using LoadRunner & WinRunner


This is my second take on the subject of load testing the Citrix WinFrame server using Mercury Interactive's LoadRunner and WinRunner tools. For a brief background and strategy in performing load testing on WinFrame server, please read my page on Stress Testing the WinFrame Server. A lot of the strategy and methods explained in that page will be applicable in performing capacity testing as well.

Capacity vs. Stress Testing

Although both stress and capacity testing are considered load tests, they each have different objectives. Stress testing is useful in determine if the server can handle peak usages, which will be (rather unrealistically, some may say) simulated by having several users perform the same action at the same time. On the other hand, capacity testing is more designed to test the system's ability to handle the average day to day transactions. In real life, there is very little probability that all of the users will be performing the same action at the same time. The more likely scenario will be where different users performing different actions, or no actions at all (i.e. when reading the generated result). So capacity testing has less peaks, but generally has a lot of transactions taking place continuously.

The stress testing module that I have created can be used in performing capacity testing, if so desired. However, you will be restricted in having all of the WinFrame sessions in one host performing the same actions. You can use the _delayed functions to delay the action between each WinFrame sessions, but you will still have 10 users doing the same thing one after the other. Doable, but we can do better.

The capacity testing module (modules, actually) is designed to allow you to perform a variant of the old timeslicing method of the old operating systems. It allows the testers to have each WinFrame session's actions be independent of the other sessions on the same host. This will allow for a more realistic simulation of multiple users on the system, each perfoming independent actions at independent times.

Features of the Capacity Module

Here are the highlights of the capacity module:

  1. Allows for control of multiple WinFrame sessions. The limit of sessions per host is limited by the CPU, memory, and how sensitive the tester is with regard to lags between actions. Ten sessions seem to be the ideal for a Pentium II 266 machine with 64MB of physical RAM.
  2. Allow for each controlled WinFrame session in a host to have it's own separate script, independent of what the other WinFrame sessions.
  3. New feature allows transactions in each WinFrame session to be timed independently of the other windows. The module achieves this by appending a unique identifier at the end of the transaction name, "[#]", where # is a number identifying the index of the WinFrame session (starts at 0). At the end, if you want to combine all of the transactions together, you can modify the result file by eliminating the "[#]" strings.
  4. A special rendezvous function allows for several WinFrame sessions within a host to synchronize to a rendezvous. However, you can not set a rendezvous between a subset of a host's WinFrame sessions with another subset of another host's WinFrame sessions. It only works within a host. If all of the WinFrame sessions in a host meets at a particular rendezvous, then the function will call an actual rendezvous to the LoadRunner controller. This is the only way that you will be able to synchronize rendezvous between hosts: rendezvous all of the sessions.

Are There any Drawbacks?

So far, so good. But there are several drawbacks that you should be aware of; some of which is inherent to the way that the module controls multiple windows:

  1. There will always be a time lag between the time that a window/WinFrame session receives the focus, and the time that it receives the next focus. This is due to the fact that WinRunner has to cycle through all of the WinFrame sessions, sequentially. The module does not have any sophisticated priority management that a modern operating system does, remember that.
  2. Rendezvous between a subset of a host's WinFrame sessions with a subset of another host's WinFrame sessions is not allowed (see above).
  3. Memory requirement might be slightly larger if complex scenarios are used. Read this in more detail later on. However, in general it would help to have as much memory as possible when running multiple sessions, since memory swaps will noticably slow the window manipulations.
  4. Harder to debug and program (yes, I said program). I pretty much had to create a mini TSL (Test Script Language) derivative, and the corresponding interpreter to run it and manage the timeslicing. Unfortunately, there is no parser to validate the script before it is ran. Fortunately, there is a way to debug "most" of the script. Also, loops are a little harder to implement, and I had to resort to using "goto" like statements (I could hear some gasps). Conditional statements are also a little kludgey (if there is such a word) to implement, but very doable.

However, once you get used to the custom functions (very few, and easy to pick), and the way that the module executes the script, then you'll find that this is a very flexible and powerful module to work with, and will help make your testing task easier.


You should follow the first part of the stress testing section's Tips for Success.



Tech Page @ itechnologist.com