1 Testing tools
Testing tools measures specific user performance in human – computer interaction. Each test is designed to measure one, exactly defined, type of interaction. During the test, several samples are taken, the average value is calculated, and then this average value is recorded in the database as a result of the test for specific person.
Some tests measure two or more values. For example, in the typing speed test the number of typed characters per second and the number of errors the user made during the test are measured. All these values are recorded into the database.
Below is a detailed description of the available tests.
1.1 Time for left mouse button click
This test measures how many times a user can click the left mouse button repeatedly at a given time interval.
Testee should click the "Click Me" button to start the test and in the same time to start time measurement. The button will turn green. Testee should press the same button repeatedly for 10 seconds. After the 10 seconds, the button will automatically become disabled which means the test is completed.
Testee has two option then:
- To select the "Save" button to save data for a later analysis.
- To select the "Quit test" button to discard this test result.
|
Measured values |
label |
Units |
|
Number of mouse clicks |
|
|
|
Elapsed time |
|
|
|
Values recorded in database tables tests and mouse_click |
DB Atribut |
Type |
|
User ID |
cad_user_id (table tests) |
int |
|
Date and time of test |
test_date (table tests) |
Datetime |
|
Number of clicks in given interval |
cclick (table mouse_click) |
int |
|
Elapsed time |
elapsed_time |
ms |
|
Calculated values |
||
|
Values (test results) presented on the screen |
||
|
User ID |
||
|
Username |
||
|
Date and time of test |
||
|
|
||
|
Previous results of the testee for mouse clicks |
Clicks per second |
|
|
Histogram for average speed of mouse click |
||
|
Sample median for average speed of mouse click |
||
|
Sample mean for average speed of mouse click |
||
|
Standard deviation for average speed of mouse click |
1.2 Mouse movement test
This is compound test that discover testee’s ability to efficiently move the mouse (i.e. cursor). Starting position of the cursor (X0, Y0) is random (actually, position where cursor was positioned last time) and the values of X0 and Y0 are recorded in database. The screen displays the small rectangle which needs to be selected with the mouse. Time of appearance of the first rectangle T0 is recorded. Rectangle is selected only if testee click on some pixel inside rectangle. After a point inside the rectangle is selected, current time Tc is recorded, new rectangle is drawn on some other random position of the screen and the user needs to re-select it. The procedure is repeated 10 times.
As the size of the distance expressed in pixels can be affected by the resolution that is set on the user's screen, the data is normalized in comparison to the full HD resolution (1920 x 1080) before storing it in the database. Normalized coordinates are calculated using the following forms:
Xn = X * 1920/w
Yn = Y * 1080/h
where w and h represents the width and height of the user's screen.
|
Measured values |
label |
Units |
|
Time when new rectangle appears on the screen |
T0 |
second |
|
Time when rectangle is selected – current time |
Tc |
second |
|
X,Y coordinates of starting point |
X0 , Y0 |
pixels |
|
X,Y coordinates of the rectangle |
Xr ,Yr |
pixels |
|
Consecutive X and Y coordinates over time |
Xn ,Yn |
pixels |
|
Values recorded in database tables tests and mouse_speed |
DB Atribut |
Type |
|
User ID |
cad_user_id (table tests) |
int |
|
Date and time of test |
test_date (table tests) |
Datetime |
|
Elapsed time from moment when rectangle is drawn until is selected |
elapsed_time (table mouse_speed) |
int (in ms) |
|
Real distance |
distance (table mouse_speed) |
int |
|
Short distance |
shorter_distance (table mouse_speed) |
int |
|
Time spent to select new rectangle (Elapsed time Tr - T0) |
Te |
second |
|
Distance between the starting point and the new rectangle |
Di |
pixels |
|
Speed of mouse movement to the particular rectangle |
Vi |
pixels/s |
|
Calculated values |
||
|
Total distance between all rectangles |
Dt |
pixels |
|
Averaged speed of mouse movement |
Va |
pixels/s |
|
Total length of the actual path of the mouse |
Lt |
pixels |
|
Mouse movement distortion ratio Mdr = Lt / Dt |
Mdr |
|
|
Mouse movement efficiency Me = Dt / Lt |
Me |
|
|
Values (test results) presented on the screen |
||
|
User ID |
||
|
Username |
||
|
Date and time of test |
||
|
Averaged speed of mouse movement |
||
|
Mouse movement distortion ratio |
||
|
Previous results of the testee for averaged speed of mouse movement |
||
|
Previous results of the testee for mouse movement distortion ratio |
||
|
Histogram for averaged speed of mouse movement |
||
|
Sample median for averaged speed of mouse movement |
||
|
Sample mean for averaged speed of mouse movement |
||
|
Standard deviation for averaged speed of mouse movement |
||
|
Histogram for mouse movement distortion ratio |
||
|
Sample median for mouse movement distortion ratio |
||
|
Sample mean for mouse movement distortion ratio |
1.3 Time for left, right, middle and double mouse button clicks
The goal of this application module is to determine the mean time required for one left, right, middle or double mouse click. Double click is performed with the left mouse button. After the test has been initiated, the message is shown containing the name of the mouse button to be pressed. The mouse button to be clicked is determined by random selection. If the user selects the correct button, a message about the selection of a new button is displayed. In total, during the test testee has to click 20 times.
|
Measured values |
label |
Units |
|
Time when mouse button name appears on the screen |
T0 |
ms |
|
Time when mouse button is pressed |
Tc |
ms |
|
Number of errors before exact button is clicked |
En |
|
|
|
||
|
Values recorded in database table tests and mouse_change_taster |
DB Atribut |
Type |
|
User ID |
cad_user_id (table tests) |
int |
|
Date and time of test |
test_date (table tests) |
Datetime |
|
Mouse button name which should be pressed |
taster |
varchar |
|
Previously pressed mouse button name |
previous_taster |
varchar |
|
Elapsed time |
time |
int |
|
Number of errors for given attempt |
number_of_errors |
|
|
Calculated values |
||
|
Average time for LEFT, RIGHT, MIDDLE and DOUBLE button clicks (per user and per test) |
||
|
Average number of errors for LEFT, RIGHT, MIDDLE and DOUBLE button clicks (per user and per test) |
||
|
|
||
|
|
||
|
|
||
|
Values (test results) presented on the screen |
||
|
User ID |
||
|
Username |
||
|
Date and time of test |
||
|
|
||
|
Previous results of the testee for |
||
|
Previous results of the testee for averaged time of mouse button clicks (LEFT, RIGHT, MIDDLE and DOUBLE CLICK) |
||
|
Previous results of the testee for number of errors |
||
|
Histogram for averaged time of mouse buttons clicks |
||
|
Sample median for averaged time of mouse button clicks |
||
|
Sample mean for averaged time of mouse buttons clicks |
||
|
Standard deviation for averaged time of mouse buttons clicks |
Those data that deviated from the average value by 500% were considered to be irregular. Also, users who have incorrectly reported the year (for example year: 5) and users who did not perform the test for 30 seconds were excluded from calculations. The extreme values of time usually appear at the start of the test. This is probably due to the fact that the user need some time to understand how the application works. Also, another cause of extreme values is the lack of user concentration, most frequently caused by: message on the phone, conversation with another person, not watching the command display or continuous clicking on the left mouse button.
1.4 Time for mouse wheel scroll
This test implies scrolling of the text that is displayed in a separate text box. When scrolling the mouse, the number of rotating steps of the mouse wheel per second is measured. The text should be scrolled in both directions. The test lasts 10 seconds.
|
Measured values |
label |
Units |
|
Number of mouse wheel rotating steps |
|
|
|
Elapsed time |
|
|
|
|
||
|
Values recorded in database tables tests and mouse_scroll |
DB Atribut |
Type |
|
User ID |
cad_user_id (table tests) |
int |
|
Date and time of test |
test_date (table tests) |
Datetime |
|
Number of rotating steps in given interval |
count (table mouse_scroll) |
int |
|
Elapsed time |
elapsed_time |
ms |
|
|
||
|
Calculated values |
||
|
Average number of rotating steps (per user and per test) |
||
|
Values (test results) presented on the screen |
||
|
User ID |
||
|
Username |
||
|
Date and time of test |
||
|
|
||
|
Previous results of the testee for mouse scrolling |
||
|
Histogram for average time of mouse scrolling |
||
|
Sample median for average time of mouse scrolling |
||
|
Sample mean for average time of mouse scrolling |
||
|
Standard deviation for average time of mouse scrolling |
This test needs to be improved because it does not correspond to the real use of the mouse wheel during the scroll. In a revised test, the user should scroll up to strictly specified lines of text. Now the scrolling goes back and forth, without paying attention to cursor positioning precision.
1.5 Time for left hand keystrokes
This test is used for determining average time needed for pressing one keystroke on the keyboard with left hand. Keys that are used are: Alt, Ctrl, Shift, Esc. By random principle the program displays the key that has to be pressed. When the test starts, program displays the first key, and only when user presses a correct key, the next key is shown. The test is 30 seconds long.
|
Measured values |
label |
Units |
|
Time when key name appears on the screen |
|
|
|
Time when key is pressed |
|
|
|
|
||
|
Values recorded in database table tests and mouse_change_taster |
DB Atribut |
Type |
|
User ID |
cad_user_id (table tests) |
int |
|
Date and time of test |
test_date (table tests) |
Datetime |
|
Time elapsed since the display of the key name, until it has been pressed |
elapsed_time (mili seconds) |
int |
|
Previous key that needed to be pressed |
previous_taster |
String |
|
Key that should have been pressed |
taster |
String |
|
Calculated values |
||
|
Average input speed for all keys |
Keys per second |
|
|
Average input speed for ESC, ALT, SHIFT and CTRL keys |
Keys per second |
|
|
Values (test results) presented on the screen |
||
|
User ID |
||
|
Username |
||
|
Date and time of test |
||
|
|
||
|
Previous results of the testee for |
||
|
Previous results of the testee for averaged speed of keyboard keys selecting (ALT, SHIFT, CTRL, ESC) |
||
|
Histogram for averaged speed of keys selecting |
||
|
Total median value of all users |
||
|
Total average value of all users |
1.6 Time for combination of keystrokes
Within this test, users were asked to simultaneously press one of the auxiliary keys (Ctrl, Shift or Alt) in combination with a key representing a letter (for example, Ctrl+X, Alt+O, etc.). Upon starting the test, a combination of keys (also known as keyboard shortcuts) that are meant to be selected by the subject is displayed in a specified space. If the subject inputs the designated combination successfully another combination is displayed and so on. As in previous cases, the combinations are chosen randomly, and the time required to enter the corresponding combinations is measured. The test lasts for 30 seconds.
|
Measured values |
label |
Units |
|
Time when key combination appears on the screen |
|
|
|
Time when key combination is pressed |
|
|
|
|
||
|
Values recorded in database tables test and keyboard_two_tasters |
DB Atribut |
Type |
|
User ID |
cad_user_id (table tests) |
int |
|
Date and time of test |
test_date (table tests) |
Datetime |
|
Time elapsed since the display of the key combination, until it has been pressed |
elapsed_time (mili seconds) |
int |
|
First key in combination |
first_taster |
String |
|
Second key in combination |
second_taster |
Sring |
|
Calculated values |
||
|
Values (test results) presented on the screen |
||
|
User ID |
||
|
Username |
||
|
Date and time of test |
||
|
|
||
|
Previous results of the testee for averaged speed of keyboard key combination selecting (ALT, SHIFT, CTRL, ESC) |
||
|
Histogram for averaged speed of key combination selecting |
||
|
Total median value of all users |
||
|
Total average value of all users |
||
During the analysis of this test, all recorded instances of keystroke combinations with a duration time greater than a fivefold value of the average time were discarded. We believe that these occurrences of longer time intervals (which were discarded) were caused due to subjects being distracted by various unforeseen elements, such as: phone calls, communication with other users, etc.
1.7 Time required for typing regular text
Within this test, users were asked to type a pre-prepared specific text using both hands (for example, file name or part name). This test accepts all data typed in text box, with all possible mistakes, which makes this test imperfect. The data obtained are stored in the database for further processing. Before processing, two samples with too many errors were discarded. Data processing consists of calculating average times for one keystroke for every user and then average values for all users.
In addition, the number of errors per ten typed chars entered for each user is counted. The Damerau–Levenshtein algorithm is used to calculate the number of errors. The Damerau–Levenshtein distance between two words is the minimum number of operations (consisting of insertions, deletions or substitutions of a single character, or transposition two adjacent characters) required to change one word into the other. Since this number is usually too small, we counted the number of errors per 10 characters.
|
Measured values |
label |
Units |
|
Time when text which should be entered appears on the screen |
|
|
|
Time when enter key is pressed, after entering the text |
|
|
|
Damerau-Levenshtein distance between entered and given text (this is number of errors) |
||
|
|
||
|
Values recorded in database table tests and keyboard_text |
DB Atribut |
Type |
|
User ID |
cad_user_id (table tests) |
int |
|
Date and time of test |
test_date (table tests) |
Datetime |
|
Time elapsed since the display of the text, until it has been entered |
elapsed_time (mili seconds) |
int |
|
Text which should be entered |
given_text |
String |
|
Text which is actually entered |
entered_text |
String |
|
Number of errors for each entered text |
number_of_errors |
int |
|
Calculated values |
||
|
Values (test results) presented on the screen |
||
|
User ID |
||
|
Username |
||
|
Date and time of test |
||
|
|
||
|
Previous results of the testee for averaged speed of entering text from the keyboard |
||
|
Histogram for averaged speed of text entering |
||
|
Total median value of all users |
||
|
Total average value of all users |
||
1.8 Time required for typing numbers
The goal of this test is to see how much time it is needed for one keystroke of a number on the keyboard. By random principle the program displays a number made of three whole digits and one or two decimals separated by comma. Upon starting the test program displays the first number. When the user finishes pressing all number keys (and a comma that separates whole from decimal digits), user has to press Enter so the next number can be displayed. The process is repeated for 5 different numbers.
Data processing consists of calculating average times for one keystroke for every user and then average values for all users.
In addition, the number of errors per ten typed numbers entered for each user is counted. The Damerau–Levenshtein algorithm is used to calculate the number of errors. The Damerau–Levenshtein distance between two words is the minimum number of operations (consisting of insertions, deletions or substitutions of a single character, or transposition two adjacent characters) required to change one word into the other. Since this number is usually too small, we counted the number of errors per 10 typed numbers.
|
Measured values |
label |
Units |
|
Time when number which should be entered appears on the screen |
|
|
|
Time when enter key is pressed, after entering the number |
|
|
|
Damerau-Levenshtein distance between entered and given number (this is number of errors) |
||
|
|
||
|
Values recorded in database table tests and keyboard_numbers |
DB Atribut |
Type |
|
User ID |
cad_user_id (table tests) |
int |
|
Date and time of test |
test_date (table tests) |
Datetime |
|
Time elapsed since the display of the number, until it has been entered |
elapsed_period (mili seconds) |
int |
|
Number which should be entered |
given_number |
String |
|
Text which is actually entered |
entered_number |
String |
|
Number of errors for each entered number |
error_number |
int |
|
Calculated values |
||
|
Values (test results) presented on the screen |
||
|
User ID |
||
|
Username |
||
|
Date and time of test |
||
|
|
||
|
Previous results of the testee for averaged speed of entering numbers from the keyboard |
Numbers per second |
|
|
Histogram for averaged speed of numbers entering |
Numbers per second |
|
|
Total median value of all users |
Numbers per second |
|
|
Total average value of all users |
Numbers per second |
|
1.9 Time for transfer the right hand from the mouse to the keyboard and vice versa
Within this test, the speed of moving the hand from the mouse to the keyboard and vice versa is determined. The user is shown the rectangle to be selected by the mouse. After choosing the rectangle, he is asked to enter a number in the text input field. After entering the number, the rectangle to be selected is displayed again, and so on. The procedure is repeated 10 times. The time follows from the selection of the rectangle to the entry of the number, as well as the time elapsed from entering the number to the rectangle selection. The user is also required to use only the numeric keypad.
|
Measured values |
label |
Units |
|
Time when number which should be entered appears on the screen |
|
|
|
Time when enter key is pressed, after entering the number |
|
|
|
Time when left mouse button is pressed |
||
|
|
||
|
Values recorded in database table tests and mouse_keyboard_movement |
DB Atribut |
Type |
|
User ID |
cad_user_id (table tests) |
int |
|
Date and time of test |
test_date (table tests) |
Datetime |
|
Time elapsed since the display of the number, until it has been entered |
mouse_to_keyboard_time (mili seconds) |
int |
|
The time elapsed since the user presses the enter until the moment he presses the left mouse button |
keyboard_to_mouse_time (mili seconds) |
int |
|
Value which should be entered |
entered_value |
String |
|
|
||
|
Calculated values |
||
|
Values (test results) presented on the screen |
||
|
User ID |
||
|
Username |
||
|
Date and time of test |
||
|
|
||
|
Previous results of the testee for averaged times of moving hand from mouse to keyboard |
seconds |
|
|
Previous results of the testee for averaged times of moving hand from keyboard to mouse |
seconds |
|
|
Histogram for averaged speed of numbers entering |
Seconds |
|
|
Total median value of all users |
seconds |
|
|
Total average value of all users |
Numbers per second |
|