Stress Testing The WinFrame Server
Using LoadRunner & WinRunner

Citrix's WinFrame server is probably one of the more popular thin client product for the Windows environment. Unfortunately, as many testers find out, there is no testing tool out there that fully supports load testing the WinFrame server. That is what stumped me when my job required me to simulate a sizable load on our WinFrame server. Fortunately, Mercury Interactive's LoadRunner along with WinRunner's TSL is flexible enough to make this possible.

Note that this page is not meant to provide you with information on performing functionality/baseline testing for applications being deployed using WinFrame. For functionality testing, your two choices are either: test a locally installed copy of the application, or install WinRunner on the WinFrame server and run/create the scripts from within the WinFrame window. As far as I know, the WinFrame server will crash if you try the second option, but someone has claimed that you can do this with version 1.7 of the WinFrame server (check the WinRunner discussion group at Mercury's web site).

First of all, it is very useful to read Nate Muller's technical notes on "Using LoadRunner to Test Citrix WinFrame". You can get it from WinRunner's technical support department. To sum up, it is pretty much impossible to use LoadRunner's db virtual users, and your best bet would be using GUI virtual users (i.e. WinRunner). Unfortunately, WinRunner cannot recognize any of the GUI objects inside the WinFrame (WF) sessions's window. The whole WinFrame window is only recognized as a large bitmap image by WinRunner. However, instead of using the analog tool as suggested in the paper, use the context sensitive win_ commands, which is performed relative to the windows.

The trick as suggested by by the technical note, is to use the analogopen multiple WF sessions on the client's computer, and use WinRunner to control all of the WF sessions. Unfortunately, Mercury does not provide the functions to do this in the package, so you'll have to create it yourself. That's what I ended up doing, and learned quite a few lessons at the same time.

These modules were created and tested on WinRunner 4.04 and LoadRunner 4.51. However, all of these modules use WinRunner's TSL language, which means that they will work with WinRunner 5 and any future releases of WinRunner (as long as Mercury does not modify their core TSL language).

Some Tips for Success

Here are several things that makes this process a lot easier:

1. Try to rely on text recognition as opposed to bitmap checks to for one reason: it's a lot faster!!
2. Try to use keystrokes for navigation: keypads and mnemonics (it's a lot faster).
3. Install the WinFrame client software on a network location that is accessible to all the host computers. This will make it easier to launch the WinFrame sessions, as well as keeping the WinFrame entries centralized in one repository. You'll appreciate it when you have to change your WinFrame configuration. Although you do not have to map the network drive (I believe WinRunner can execute application with UNC paths [although WinRunner itself has to be ran from a mapped network drive -- go figure], and WinFrame clients can be executed from UNC paths), it might be more convenient and make your code more readable.
4. Set up multiple entries in the WinFrame manager with the same configuration (same window size, number of colors, server address/published application), but differentiate each window entry with an incremental numerical index (e.g. "App0", "App1", "App2", "App3", "App4", and so on). This makes it a lot easier to assign the windows based on the number of session you want to control.
5. Set up multiple WF accounts with incremental numbers, or clusters of incremental numbers ("Acct0", "Acct1", "Acct2", ..., "User0", "User1", "User2", ...). This will again simplify assigning the user for each WF session. Again, try to cluster similarly configured user in groups.
6. Get used to programming in TSL. I do most of my scripts by programming, and to perform this efficiently, you'll need to do both recording and programming extensively. Typically, you'll write the scripts by programming, perform some recordings (mainly to get coordinates), and then modify the recorded scripts.

The method that I'm laying out here is useful for simulating multiple WF users on one machine, performing similar actions. All actions are very sequential, and all of the windows will pretty much progress at the same rate. For example, in a client computer, to test how fast a table load:

1. The scripts would set up all of the WF sessions just before the data is loaded.
2. Set up a rendezvous point.
3. Starts a transaction.
4. Performs the action on all the WF sessions that will cause the table's data to be loaded in all of the WF session.
5. Wait for all of the WF sessions to complete loading the table.
6. End a transaction.

As you can see, the transaction data recorded for the client computer with multiple WF sessions will not be accurate, as the transaction time that is recorded will be the transaction of the slowest session. Instead, you'd want to set up a client computer with only 1 WF session to more accurately record the transaction time.

If you're interested in using WinRunner to drive multiple WF sessions with multiple (different) scripts/scenarios for load testing purposes, then you'll want to see my second take on this topic (Load Testing the WinFrame Server with WinRunner). Warning though, that you'll have to do a lot more programming than recording (way more). However, I've had a great success with it to really better simulate simultaneous WF sessions on a single computer.

WinRunner's Performance

So how well does WinRunner control the WF sessions? Pretty good.

On a decently fast computer (Pentium II 266Mhz, 64MB Ram, 64MB pagefile, running Windows NT Workstation 4), it takes between 3 to 5 seconds to cycle through 10 WinFrame sessions for text recognition or keystrokes, and up to 10 seconds for mouse actions (clicks, double clicks, drags). One second per session of time lag. The faster the processor and the more physical memory available, the better your performance will be (with 10 sessions, plus WinRunner, you'll be using over 100MB of memory. Yikes!!).

I have noticed that some computers perform the window flipping, get_text(), and other GUI actions a lot faster than other similarly configured (hardware) computers. I am not sure of the source of the "boost", but I think IE 4 and other updated Windows system DLL's might play a role.

There are ways to improve your GUI user's (WinRunner script) performance, which I will explain later.

Tech Page @