PhOSCo GUI Test Robot

Version 1.01 - Initial release 25th September 2000

The PhOSCo GUI Test Robot is a very easy to use tool for testing the Graphical User Interface (GUI) behaviour of Java applications. It can also be used to create automatic demonstrations of Java applications.

Capture is done by using the program under test as normal, invoked from the Capture tool. the Capture tool adds one dialog which can be used to add screen captures, comments, and other tests to the test script.

Replay and testing is under control of a humanly readable (and editable) test script file; the replay tool can invoke processes and compare data in any type of file; together these make this a pretty open-ended and configurable tool.

It is available as source and executable code.

It can only be used on Java v1.3 or later programs for which you have the source code of the main Frame class and to which you can add the helper class and recompile (this requires two lines of code to be added to the application under test).

This code is Copyright (C) 2000 Mike Calder-Smith, mike@gmot.demon.co.uk This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. You must include this notice in its entirety with any redistribution. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

"PhOSCo" and "Pharma Open Source Community" are trade marks of Guillemot Design Ltd. Guillemot Design Ltd Reg No 3803598. PhOSCo has its home at www.phosco.com

Please note that this program is currently not up to full PhOSCo coding standards. It is a work in process, internal use interim release tool. In particular, it currently does not support intenationalisation and is an English language version. Guillemot Design Ltd and Mike Calder do not commit to produce or release a full standard PhOSCo version by any specified date. If a full PhSOCo version is provided in future we will either ensure that data files (test script files) to the format specified in this document will work with any full PhOSCo version or we will provide a conversion utility.

This document describes the two programs:

• The PhOSCo GUI Capture tool PGUICapture,
• The PhOSCo GUI Robot program PGUIRobot,

• The PhOSCo GUI Robot Slave helper class PGUIRobotSlave.class.

There is also here a description of the Test Script files and a table Test Commands which documents all the possible commands in those files.

The classes require Java 1.3 to compile, and the availability of the com.sun.image.codec.jpeg extensions from the Sun JDK 1.3 or JRE.

To create a test case, the program to be tested has to be instrumented for PGUIRobot. This means that two lines of code have to be added to the program to be tested, to include the PhOSCo GUI Robot Slave helper class PGUIRobotSlave.class in the program, and the program has then to be recompiled.

It will become the standard for all PhOSCo GUI programs to be instrumented as they are written, and existing GUI programs will have this fitted shortly.

The test case designer decides on the files and databases to be used by the program, and saves copies of these to the appropriate test case subdirectory.

Creating the test case itself is a simple matter of running the PhOSCo GUI Capture tool PGUICapture with the program to be tested, and using the mouse and keyboard to drive the application under test. PGUICapture will save the mouse movements and keyboard input to a test script file which PGUIRobot can use to run the test.

The test case script files are humanly readable, and can be amended if necessary using a text editor. Some additional test commands can only be added manually in this way.

The test is normally passed or failed depending on whether on replay under PGUIRobot the screen window of the application looks the same as it did at the time when a screen was captured by user command when setting up the test with PGUICapture.

Additional tests can be added to the test case script file by manual editing to add File Compare commands, for example. For a File Compare command, the test case designer specifies two file names. The contents of these files are then compared at the point in the script where the compare command was inserted when the test case is run, and if the files have matching contents at that time, the test case passes, otherwise it fails. These files are normally pairs of files, one of which is standard data set up by the designer, and the other is either a file produced by the application being tested, or a program run against files or databases used by the tested application.

At any time during the test script, the designer can specify, either by using PGUICapture or by editing the test case script file, that a subsidiary program or batch script should be run. This could, for example, be a simple query in a batch script against a database to get the data values of interest and put them into a file. This file could then be compared with expected values in a file comparison test later in the test case.

PGUIRobot can run either a single test case or can be given a file of a list of test case names to run a whole series of tests. In this way it is possible to run whole sets of tests automatically.

The PhOSCo GUI Capture tool PGUICapture

When this program is run it is given a runtime parameter of the name of the file to which it will save mouse, keyboard, and other commands; for example:

  java -classpath .;d:\jdk1.2\lib\jdbcdrv.zip PGUICapture test1.gut

When it runs, it prompts the user for the invocation string which starts up the application to be tested:

This dialog should be given the full command string needed to start up the application to be tested, with any run-time parameters. Under Windows, it should be remembered that the file extension of command scripts is a necessary part of the invocation string; so if you have a "batch file" called "jr.bat" to start a particular application, the full invocation string would be

  jr.bat arg1 arg2 arg3

The application to be tested then starts up, and can be driven normally using the mouse and keyboard. All the input from the mouse and keyboard is saved to the test script file.

The only difference to a normal run of the application being tested is when the F1 key is pressed. Normally this will be set to a function such as to display Help.

Under PGUICapture, the F1 key is used to bring up the PGUICapture Command Dialog. This lets you place one of several special commands into the test script file at the current position. You can:

• Just pass the F1 key forward to be used normally by PGUIRobot during the test replay,
• Add a comment to the script file to mark a stage in the testing,
• Save the current image of the application screen window to a file,
• Copy a File selected from a dialog to a location you specify, or
• Run a program or batch script.

The dialog itself is a simple drop-down list from which you select the option you want, and then press the "OK" key.

If the screen window is large or complex, or you have a slow machine, give the program sufficient time to save the image before continuing with the next action.

If you press the "Cancel" key, nothing is saved to the test case script file.

As well as doing the function specified, the command is placed in the test case script file together with the filenames and other options you supply. PGUIRobot will do the same function at the same position when it runs the test case script file, and in the case of saving an image of the window, will compare its results with those gained here.

When you close the application being tested, PGUICapture will save the test script file and close itself down. If the application brings up any dialogs after the terminate call, your responses to these will not be saved to the test case script file, and PGUIRobot will hold waiting for this final input. In such cases, you should add keyboard or mouse events manually to the test script file to satisfy such dialogs in order for PGUIRobot to be able to move on to the next test case or to close.

The PhOSCo GUI Robot Slave helper class PGUIRobotSlave.class

Most testers can ignore this section, as this will normally be done for you.

The comment lines in the examples below are of course ignored by the Java compiler, but some such comments should be included to document for later programmers why these lines are included.

The class specifying the primary Frame of the application under test must be an extension of JFrame or JInternalFrame.

The following line of code should be added to the class field definitions of the primary Frame class of the application to be tested.

    /**
    PGUI Robot Slave.

    Instrumentation for PGUICapture and PGUIRobot: Block 1; definition.

    Include after class definition outside class code.

    This code is necessary for the capture of key and mouse data by
    PGUICapture to automatically test this program using PGUIRobot.

    */
    PGUIRobotSlave minion;

and the following line of code should be added at some point inside the class constructor for the primary Frame of the application to be tested.

      // *************************************************************
      //
      // Instrumentation for PGUICapture and PGUIRobot: Block 2 Start
      //
      // Include in class constructor code, but see also note on the
      // addWindowListener call at the end of this block.
      //
      // Code in this block is necessary for the capture of key and
      // mouse data by PGUICapture to automatically test this program
      // using PGUIRobot.
      //
      // *************************************************************

      minion = new PGUIRobotSlave( this );

      // *************************************************************
      //
      // Instrumentation for PGUICapture and PGUIRobot: Block 2 End
      //
      // Code in this block is necessary for the capture of key and
      // mouse data by PGUICapture to automatically test this program
      // using PGUIRobot.
      //
      // *************************************************************

If the primary frame creates a WindowListener for window closing WindowEvents, as most do, such as:

      addWindowListener(new WindowAdapter() {
          public void windowClosing(WindowEvent event) {
              dispose();
              System.exit(0);
          }
      });

the line of code to be added above should be added before this WindowListener or other AWTEventListeners are created, so that the PGUIRobotSlave can tell when the application is terminated.

If this is not done correctly, the "TERMINATE" command will not be inserted in the Test Script file, and should be added manually. Also, the PGUICapture program will probably not close automatically when the application under test is closed, and must then be closed from its console using the Control-C key combination.

The creation of the PGUIRobotSlave object in an application will cause a wait for reception of a UDP Datagram of approximately 3 seconds to occur during the application startup if the PGUICapture program is not live on the same machine. A UDP conversation takes place between the Slave and the Capture program to ensure that if the Capture program is not listening no time is wasted creating instrumentation datagram packets and sending them to the non-existent Capture listener. It may be more appropriate to only create the Slave object when instrumentation is required, based perhaps on a boolean set by a run-time parameter, thus:

      if (captureOn) minion = new PGUIRobotSlave( this );

Note, however, that the Slave must be active while PGUIRobot is working as well as during Capture.

The PhOSCo GUI Robot program PGUIRobot

This program is run by giving it a runtime parameter of the name of a test script file or a test suite file.

  java -classpath .;d:\jdk1.2\lib\jdbcdrv.zip PGUICapture test1.gut

A "Test Script" file is a file of commands created by PGUICapture, perhaps amended manually using a text editor. A "Test Suite" file is an ASCII file which is a set of lines, each of which contains the path and name of a Test Script file, and no other information. It is recommended that specific file extensions be used to distinguish between the two in any installation; those delivered by PhOSCo will be ".gut" for GUI Test Script files, and ".gts" for Test Suite files.

If the file given to PGUIRobot is a test suite file, it will run each test script within the test suite in turn.

Each test script file specified to PGUIRobot, whether by direct call or from a test suite, will be processed by reading the commands in the file line by line, and replaying them. Completion of each command is noted in a log file.

An early command in the file will be to invoke the application to be tested. Before that, commands (mainly added manually) may be processed to set up the environment for the test; to copy files, for example. Some commands, such as mouse movement and keyboard commands, are only valid while an application is being processed.

Note that PGUIRobot will pause for 10 seconds after first invoking an application to be tested to allow the application to set itself up properly and display its screen window. If this time is insufficient in any particular case, the test designer should add an appropriate Delay command to the Test Script file immediately after the INVOKED command.

If commands to take a screen snapshot, or to compare files, are found, these will affect the pass/fail status of the test case. Initially, the program is assumed to pass the test case. If a file compare is found where the files do not have equal contents, the application will fail the test case. If a screen snapshot is found which is not the same as the snapshot saved by PGUICapture, again the application will fail the test case.

Note that PGUIRobot pauses for a period of 3 seconds prior to taking any screen snapshot to allow any current repainting to occur. If this time is insufficient in any case, the test designer should add an appropriate Delay command to the Test Script file immediately prior to the Screen Capture command.

Pass/failure status of each test case is written to the log file, and summaries presented to the screen and to the log at the end of the program.

It should be noted that screen snapshots are very dependent on the current machine environment; screen resolution, display capabilities, fonts, and so on. When running a test case or suite on a new machine for the first time, it may be necessary to delete the old comparison snapshots, to run a test script to recreate all the snapshots, and rename them to the original names, and to compare the new ones with the originals for functional equivalence. (PGUIRobot names its snapshots "Testimage1.JPG", "Testimage2.JPG", and so on.

It is recommended that a paper copy of test case snapshots be kept for this purpose of functional equivalence comparison.

In some cases, where complex processing or extended disk access takes place, the application under test will be unable to complete the appropriate function (and perhaps repaint) before the next event is replayed. This will usually cause the test to fail, and subsequent action will in most cases not be what is required. It will usually be obvious from watching the replay that this is happening. The test designer should be aware of this possibility and should edit the test script file manually, adding delay commands where appropriate so as to allow the tested application to complete its functions.

There are two delay commands possible: "Delay" (DLY n) and "Set Autodelay" (AD n). In both of these the time is in milliseconds. "Set Autodelay" makes the Robot wait the specified time after every replayed event, where "Delay" just delays the specified time once at the point in the script where it is found. The autodelay can be reset to different times at as many points within the script as is necessary.

The default is for an autodelay of 300 milliseconds.

Test Case Script Files

A very simple test case script file is:

    INVOKED jr.bat BLog
    USING 0 0 800 460
    AD 300
    MM 388 432
    CM 1
    KEY 65 A
    KEY 66 B
    MM 306 190
    CM 1
    SC 0 0 800 460 E:\JCTPROD\Robot\test.jpg
    TERMINATE

This file tells us that PGUICapture was run against a procedure called "jr.bat", which was passed a command line argument od "BLog". The application came up in a window at top left co-ordinate 0, 0 on the screen, 800 pels wide and 460 pels deep.

The mouse moved to a position in that window, was clicked once, and the keyboard keys 'a' and 'b' pressed, the mouse moved away and clicked again, and a screen snapshot taken and saved as a particular file.

If PGUIRobot were run using this, it would run that application again, replay the mouse and keyboard commands, take a new screen snapshot, and compare that new snapshot with the saved file. If they were the same, the test would pass, otherwise it would fail.

Test Case Commands

A Test Script file for a Test Case is a line-based ASCII file. Each line contains a single Test Case command or a comment. If the first non-whitespace token in a line is not a recognised command keystring as defined in the table below, it is assumed to be a comment and ignored.

The Source given in the table is the normal source of the command; any command whose source is "Capture" may also be added or amended by manual editing of the file.

Keystring	Parameters	Source	Description
AD	time in milliseconds	Capture	Set Autodelay period (automatic delay added after executing every Robot command). A default value of 300 milliseconds is added to each test script file to allow for repaint activities. This can be amended if necessary.
CM	Mouse Button Mask (1 = Left Mouse Button, 2 = Right Mouse Button)	Capture	Click Mouse once. The click is at the current position on the screen as defined by the most recent Mouse Move command. If you are adding these by manual editing you must ensure that this position is given correctly by a prior command.
COMPARE	File name 1, File name 2	Manual	File Compare. Compare contents of two files for equality. If not equal, status of test case is set to "fail".
DCM	Mouse Button Mask (1 = Left Mouse Button, 2 = Right Mouse Button)	Capture	Double-click Mouse. The click is at the current position on the screen as defined by the most recent Mouse Move command. If you are adding these by manual editing you must ensure that this position is given correctly by a prior command.
DLY	time in milliseconds	Manual	Delay at this stage in the script for a defined time.
FC	Source file name, Target file name	Manual	File Copy. Copy the source file to the target file name.
INVOKED	Invocation String	Capture	Application under test startup.
KEY	Key code, or character	Capture	Keyboard character keyed. PGUICapture gives the key code followed by a space followed by the key character. For ease of manual editing, the key character alone is also acceptable. Note that the character denotes the key alone; it does not denote the shift state. That must be set by SET or reset by UNS commands; by default at the start of a session no shift is set.
MM	x, y co-ordinate in pels	Capture	Mouse Move.
PI	Invocation String	Capture	Process Invocation. Run a program or batch script. PGUIRobot will spawn a thread to run this Process in, which will then run asynchronously. If the test depends on the results of this process (for later File Compare, for example), a suitable Delay command must be inserted manually after the Process Invocation command to ensure that the spawned process has sufficient time to complete before the dependent command is executed. This may or may not be successful depending on the scheduling algorithm used by the current environment; it may be more appropriate to make any dependent test run in a later test script file in the same suite.
SC	x, y origin of top left; width, and height of screen window, name of file saved	Capture	Screen Capture. The portion of the screen given by the co-ordinate, width, and height is saved to a JPEG format file. PGUIRobot will save the same screen area to a temporary file (called Testimage1.jpg, Testimage2.jpg, ... ; restarting the numbering within each test script) and compare the saved file with the temporary file. If the two files do not have identical contents the status of the test case will be set to "fail".
SET	Key code, or key name.	Capture	Set Shift or meta mode. PGUICapture gives the key code of the mode type. For ease of manual editing, the mode key name ("Alt", "Ctrl", "Shift") is also acceptable. The SET command sets the mode on; to reset it or set it off the UNS command must be used.
TERMINATE	none	Capture	Close the application under test. This is normally the last command in a test script.
UNS	Key code, or key name.	Capture	Unset Shift or meta mode. PGUICapture gives the key code of the mode type. For ease of manual editing, the mode key name ("Alt", "Ctrl", "Shift") is also acceptable. The UNS command sets the mode off; to set it the SET command must be used.
USING	x, y origin of top left; width, and height of screen window.	Capture	Notification from the application under test of the screen window it is using. This command is repeated each time the application main Frame is resized or moved.