Getting the best out of online assessment

There has been extensive debate in the HR and testing press about the pros and cons of the internet for test delivery, but the only thing we can be sure about is that online testing is here to stay.

This growth in the use of the internet has lead to new definitions of best practice in testing. As one of the companies at the forefront of developments in online testing, PSL would like to share some of the knowledge we have gained from working with clients.

Here are some of the hints we have for organisations who would like to introduce an online testing process.
Have a clear strategy for online testing and define your expected benefits It is not enough to want to use the internet for testing just because it is a new way of doing things.

You need to consider how you want to assess people. For instance, will you be screening large numbers of candidates as part of a high volume recruitment campaign, or will you want to assess shortlisted candidates or individuals participating in a development programme in more detail?

There are many different types of online testing system on the market. Some can be used as stand-alone test sites, others can be integrated with other HR software. Do you need an all-singing, all-dancing end-to-end solution if you are only testing a few people? Likewise, a stand-alone system that works fine for small numbers of candidates may prove very cumbersome for a large campaign.

You also need to define what the expected benefits will be. Do you want to reduce costs, to speed up recruitment, to improve the efficiency of your selection process or to make test completion a more convenient process? This will form the basis of your business case for an online testing process.
Choose your online tests with care
All tests are not created equal, and some are definitely more suitable for online assessment than others. Choose carefully.

Firstly, what do you want to achieve? Tests for screening will be different to tests for in-depth assessment.

Is the test reliable and valid? The rules for selecting good tests are the same whether they are offline or online. An unreliable or invalid test is worthless, no matter how it is delivered. The same is true for tests from other countries. Always make sure that they are fit for purpose and legally defensible before using them.

Is the test suitable for online delivery? If the test contains a lot of small text or complex graphs it may be difficult for candidates to read on a computer screen. Good online tests are developed for the internet, and not just copies of paper versions.
Make sure your technology is up to the job
There is no excuse these days for using unreliable internet technology. Poor test delivery software can ruin your whole process. Pick a supplier who can give you confidence in their systems.

Make sure the software can handle your expected volume of users. What happens if everyone tries to complete the test at the same time? The test should be consistent and standardised irrespective of the number of users.

Is the data secure? And what happens if the supplierís server fails? Are there recovery and back up plans in place? Does the system comply with data protection legislation?

Make sure the tests can be accessed by all your desired test takers, particularly relevant in volume recruitment. Does the software support different internet browsers? Do you need to download software in order to run the test? Candidates are put off very quickly if they cannot easily take the test.
Practical considerations
Consider the average test taker, who when at home will be using a PC that is 2-3 years old with a 56k modem. A test with lots of graphics that takes a long time to download may look impressive but will exclude a lot of candidates. Timed tests in particular need to be quick to download.

In a traditional test session you can supervise the person and make sure that nothing goes wrong. You cannot do this with online testing. This means that you need to make the process as user-friendly as possible. Instructions should be clear and unambiguous, there should be ëhelpí buttons if possible, and there should be the means for test takers to ask for assistance if required.

You should always make sure that test takers are kept informed if there are any technical problems. Rather than just crashing, the test system should return meaningful error messages or provide suggestions.
The issue of cheating
Online tests can be completed remotely in an unsupervised environment. Remote testing carries issues, but these are usually outweighed by the considerable cost and resource benefits of an online testing process.

The first issue is test exposure. If a test is in the public domain, then sooner or later it will be copied. Candidates can copy a test and work out the answers at their leisure. This may sound cynical, but if it is possible, someone will do it. In practice this means that online tests will have a much shorter ëshelf lifeí than paper tests, and will need changing more frequently.

Most online test suppliers have introduced security measures to try and prevent this, ranging from password protected access to disabling the print buttons on a browser, but none of these offers a complete solution. The most sophisticated method of reducing the effects of exposure and cheating is by using item-banked tests.
Item-banking and randomised tests
An item-banked test will deliver a random series of test questions (or items) from a bank of available questions. Because the test is randomised, every test will be different, so there is no advantage in trying to learn the answers. Item-banking also reduces practice effects and allows you to re-test candidates easily.

The other issue is that you have no real control over who is actually completes a remote test. A candidate can ask their clever friend to take the test for them, and you will never know. No current technological security measures can prevent this, but it can be minimised by a good selection process. By re-testing candidates under controlled conditions you can identify cheating candidates. A clear statement that all candidates will be re-tested establishes an ëhonesty contractí with the test taker and removes much of the incentive to cheat. Alternatively, some employers are arranging local test centres around the country where candidates complete online tests after having had their ID checked. This removes the need for retesting at the subsequent assessment centre.

In summary, the internet does offer significant benefits over traditional methods of test delivery, but this does not mean that the principles of best practice in testing should be abandoned. Instead, new principles are evolving as the field matures, so that new users can evaluate test delivery systems and implement online assessment processes with confidence.