To Get the Best, Make Sure You Test
03/05/07
Here’s a true story: Two managers for a publishing company were sitting around congratulating each other on finding a gem for a new editorial/sales position.
“Should I call him now to offer the position?” asked one.
The other, suddenly apprehensive, said, “I wonder whether we should test him to make sure he has the skills we think he does.”
“Come on,” said the first. “He just got his master’s degree in English. And we know he has a good personality for selling.”
“I’m sure he’s fine,” said the second.
“Let’s just make doubly sure.”
They arranged for the recruit to come in to take simple tests in writing, proofreading, and arithmetic all skills directly related to the job.
What a shock! The job candidate, in spite of his advanced degree and excellent references, couldn’t write. He couldn’t spell. And he couldn’t add or subtract.
Eyes now opened, the company began giving every potential hire for that position the same test.
Why Test?
As this story and the graph show, you can’t beat testing to ensure that your shiny new recruit can actually do what the job requires. And according to industrial psychologist and test developer Jesse M. Llobet, Ph.D. Who has created all of G. Neil’s tests “All jobs, entry level through management, lend themselves well to testing.”
But you can’t just make up any old test and offer it haphazardly. Testing is carefully governed by various laws and regulations, including the federal government’s Uniform Guidelines on Employee Selection Procedures (see the box on page 4). If you run afoul of these guidelines, which govern screening, interviewing, testing, and more, you can find yourself in deep trouble.
The following guidelines will help you stay on the right side of the law and recruit a better class of employee.
Analyze the Job
Rule one: If you test, you must test for skills or characteristics that are directly related to the job and that are truly necessary for carrying it out. In other words, you need to know what you want to measure, and why you want to measure it.
For instance, say you need a secretary. Half the person’s time will be spent typing, so requiring a typing test makes perfect sense. But before you require your prospective employee to be able to type seventy words per minute, make sure a speed that high is really required.
To figure out what to test:
Identify the job’s tasks and responsibilities;
Rank tasks and responsibilities in importance;
Assess how much time will be spent on each;
While you are at it, reassess the job’s initial requirements. Do you really need someone with a college degree if she can carry out a job’s important tasks? Adverse impact is often the result of arbitrary standards with no real effect on job performance.
Once you’ve analyzed the job in depth, isolate specific, necessary skills you can test for.
Choose a Test
You know what skills you want to assess. Should you create your own test? In most cases, that’s not a good idea. The reason: If your selection process shows adverse impact, you’ll have to prove that each element, including your test, is valid. Validity means that there is a strong correlation between those who do well on the test, and those who do well on the job. Showing validity is not easy to do.Besides, there’s no need. Says Jesse Llobet, “Generally speaking, there are valid tests out there to cover most job skills and abilities. And the costs associated with developing a customized test make the purchase of off-the-shelf-tests much more attractive than customized development.”
It’s important to understand the concept of validity, because it will help you make a better buying decision. It also helps you understand what the federal government expects in a test.
While the Uniform Guidelines discuss three types of validation, only two are of concern to users of business-related tests:
Criterion-related validation: The test publisher demonstrates a statistical relationship between test performance and job performance. This kind of validity is, according to Llobet, “the most important in a business setting because it is the only validation strategy that compares test scores with actual job performance.”
A criterion-validated test will also reveal whether or not it is “fair.” Llobet elaborates: “Fair means the test is a valid predictor of job performance regardless of race, color, religion, sex, and national origin, and that it’s in compliance with the ADA.” In creating tests, for example, Llobet avoids questions and content that appear unfair, stereotypical, or biased toward any group that might take the test.
Content-related validation: The test publisher demonstrates that test items measure important requirements and qualifications for the job directly. “Content validation,” says Llobet, “is usually conducted when there aren’t enough study participants for a criterion-related study.” Your safest bet: Purchase a criterion-validated test.
Use the Test
Administer the test to job seekers (or anyone else about whom you need to make an employment decision such as those hoping for a promotion). You’ll be pleased at how it helps you improve the quality of your hires. Test publishers usually make scoring easy—it may even be automatic if the test is taken on a computer.
And they usually provide a scale that shows scores and a percentile ranking based on the groups used to validate the test. (A ranking in the 75th percentile, for example, means the test taker scored higher than 75 percent of all the test takers in the research group upon which the test was developed.) Obviously, the higher the score and ranking, the better predictor of success on the job.
But be careful how you interpret scores. Remember, a job may not require superb skills in any one area average may be good enough. Don’t create cutoffs unless a particular skill level is necessary for success on the job.
Some employers rank applicants by score results. That can be useful, but heed well the next section.
Consider the Whole Person
Never forget that testing is only one part of hiring or of making any other employment decision. Others include interviewing, reference and background checking, and performance appraisals.
Also, while tests are pretty good indicators of success on the job, they are not perfect. How can they be? After all, a person may score at the very top of a skills test, yet may be a bear to work with. Or the person may have a record of jumping from job to job every twelve months.
Jesse Llobet’s advice: “Avoid making hiring decisions based solely on test results. Hiring decisions should be based on all the elements of the selection process.” Testing, then, is only one piece of the hiring puzzle. Look at the whole person when making any employment decision.
Validate Your Test
Using a valid test is essential. But don’t rely completely on the test publisher’s validity study—start validating the test yourself. Why? It puts you in a stronger position should someone challenge your selection process, for one thing. For another, says Llobet, an employer’s “geographic location, type of job, industry, etc., may be different from the test publisher’s study sample. That might affect the test’s validity within that organization.”
Start by doing two things:
Compare test results with on-job-performance once a person has been hired. Compare the test results of job applicants with those of people already holding the same jobs. Your goal, says Llobet, “is to establish and document that, in general, high scorers on specific tests are also your better performing employees.”