Back to all articles

How to Tell if a Pre-Employment Assessment is Valid

Angela Leffler
Published January 26, 2017

Reducing bias in the hiring processAn organization’s success is driven by acquiring top talent. Over the past 10 years, significant changes in technology have affected the way that companies are recruiting and hiring candidates. SHRM recently indicated that the process most employers now use to find and select the best possible candidate includes using a preliminary assessment to screen out those who lack the desired level of skills and competencies for a job.

Choosing the right pre-employment assessment can be daunting. With a variety of options available, you need to sort through the ‘bells and whistles’ and understand whether the test is valid and reliable. Implementing a pre-employment assessment that is not valid can lead to future legal issues and ineffective hiring decisions.

If a test is reliable doesn’t that mean it’s valid?

No! Validity will tell you how good a test is for a particular situation; reliability will tell you how trustworthy a score on that test will be.

A document by the U.S. Department of Labor and Relations outlines some things you need to consider when determining the reliability and validity of a pre-employment assessment:

The test measures what it claims to measure consistently or reliably. This means that if a person were to take the test again, the person would get a similar test score (aka test-retest reliability).
Find a test that deploys forced-choice structured questions. The science behind the “forced-choice” methodology has been firmly established where applicants cannot successfully game the test.  Research has consistently shown that forced-choice inventories maintain their validity even when given to the most motivated job applicants, whereas commercial inventories using rating scales or true/false do not (Bartram, 2007; Christiansen, Burns, & Montgomery, 2005; Hirsh & Peterson; 2008). The personality sections of the Plum assessment are specifically designed to prevent applicants from misrepresenting their behavioral tendencies and claiming to have only positive dispositions at work.

The test measures what it claims to measure
For example, a test of mental ability would be considered valid if it does, in fact, measure mental ability, and not some other characteristic. There are multiple types of validity, a few of which we will briefly review. Construct-related validity refers to how well scores from an assessment correlate with scores from other established tests that measure the same characteristics. For example, Plum has been tested for construct-related validity against three other assessments. Content validity refers to how accurately an assessment measures the various aspects of the construct under question. Criterion validity considers the correlation between assessment results and a criterion variable. In this case, said variable would be a measure of the candidate’s actual job performance.

Is the assessment multi-dimensional?
Your pre-employment assessment should have questions structured around assessing more than just personality.  Most commercial psychometric assessments focus either on personality traits or cognitive abilities related to intelligence.  The appropriate strategy is to ensure that the job domain is defined through job analysis by identifying the important behaviors, tasks, or knowledge and the assessment is a representative sample of the behaviors, tasks, and/or knowledge drawn from that domain. The Plum assessment includes both personality AND cognitive factors to provide a complete assessment of the talent that applicants will show on the job. Research has shown that combining the results of multidimensional assessments of personality and intelligence will typically have twice the ability to predict job success over either type of assessment alone (Schmidt & Hunter, 1998; Tett & Christiansen, 2007).

The test is job-relevant. In other words, the test measures one or more characteristics that are important to the job.
Plum Match Scores provide an overall estimate of how well the traits and abilities of each applicant fit with what is required for a given role or job. A key component to the Plum Match Scores is to narrow the number of traits and abilities down to just those that are most relevant to the role or job. For any given position, at least half of the dimensions of a psychometric assessment may not actually predict success; the trick is to identify those that will. Research has shown that scores on dimensions identified as relevant by job experts predict performance much better than those that were not (O’Neill, Go n, & Rothstein, 2013).

By using the test, more effective employment decisions can be made about individuals. For example, an arithmetic test may help you to select qualified workers for a job that requires knowledge of arithmetic operations.
Plum Match Scores customize the scoring of the assessment to focus on those dimensions that are important for success. Typically, this is based on the expert judgments of hiring managers and top performers in the organization that are collected through a structured survey process designed to obtain job information from those that know the position best. Because of this, the profile of what is important for success as a bank teller, for example, will look very different than that of a customer service representative, and a candidate could have a high Plum Match Score for one position but not the other.

The process of merging the results of the psychometric assessments with the judgments of job experts has proven to be the most robust method for identifying top candidates (Tett, Jackson, & Rothstein, 1991; Tett, Jackson, Rothstein & Reddon, 1999).

Plum customers have reported a 93% success rate: 93% of employers would choose their Plum hire again.

Check out our science page to dig into the Plum Talent Assessment Science a little further.

This entry was posted in blog and tagged , ,  . Bookmark the permalink.