Written by
James Meachin

Published
07 Apr 2017

Situational judgement tests: The future of recruitment in a digital world

07 Apr 2017 • by James Meachin

There are numerous apps and websites all designed to help people find work, and most communication takes place through an internet connection. The world as we know it is shifting towards an all-digital revolution, and recruitment is following. 

In comparison to more traditional methods, online recruitment is driving higher number of applicants for each vacancy. Thus, as the way we recruit changes, so must the way we examine candidates during the recruitment process. At Pearn Kandola, we have been utilising situational judgement tests over other forms of testing – as we believe they hold many key benefits which methods like traditional psychometric tests will not deliver. 

What are situational judgement tests?

Situational judgement tests (SJTs) present candidates with realistic scenarios that job-holders face on a day-to-day basis. The scenarios can be presented as text, video or even audio recordings (for example, with 999 call handlers). Alongside each scenario are a number of response options – actions that the role holder could take in response to the scenario. For each scenario, candidates rate the effectiveness of different response options. This provides a clear measure of how well a candidate’s judgement fits with the role requirements and the culture of the organisation. Does a candidate recognise that one response is more appropriate than the others? Can the candidate see that another response option would be very ineffective? We know that candidates who show better judgement in completing SJTs are also likely to show better judgement – and therefore performance – when in role.
 
There are three key advantages that SJTs have over traditional psychometrics such as verbal and numerical reasoning tests when using for screening:

Diversity

Achieving diversity targets has been identified as the top priority for graduate recruiters in a survey by the High Fliers Research Centre, with 69% of organisations citing this as an important challenge – more so than any other factor. In screening candidates, SJTs are an important tool that help improve diversity outcomes in comparison to traditional psychometric ability tests. Research clearly shows that SJTs lead to better diversity outcomes than traditional psychometric ability tests. 

Our own experience bears this out. When working with a large graduate recruiter to support their assessment process, an SJT we developed was being used in conjunction with verbal and numerical reasoning tests from a test publisher. When working with the client and test publisher to set cut-off scores, we found that we needed to give the SJT a much higher weighting than the verbal and numerical reasoning tests in order to avoid adverse impact, which would reduce the diversity of the candidate pool. 

Building on this, for organisations that really want to make progress with diversity, SJTs can be used in place of traditional psychometric ability tests.

Breadth

Screening tools that can measure a wide range of qualities are important because they allow for more efficient and meaningful assessments. One of the outstanding features of SJTs is how versatile and broad they are in measuring candidates’ suitability.

We worked with a major retailer to develop an SJT for selecting shop-floor staff. In validating the tool, we explored which aspects of candidate performance it tapped into. Here, researchers identify three categories of job performance, and we explored all three:

•    Task performance – how effective someone is at delivering the core requirements of a role. This includes the behaviours typically found within a competency framework. In our research, we used performance ratings from line managers to measure task performance.

•    Citizenship behaviour – the extent to which someone is a ‘good organisational citizen’ by going above and beyond what is normally expected, such as helping and advising others, and going out of one’s way to get the right result. We used ‘special recognition’ awards that were given by store managers on a discretionary basis to measure citizenship behaviour.

•    Counter-productive behaviour – this is the ‘dark side’ of performance at work and it includes theft, disruptive behaviour, and unauthorised absences. We used unauthorised absences to measure counter-productive behaviour. 

We found, to our surprise, that the SJT scores were able to predict all three performance domains very well. In essence, using SJTs to screen candidates is advantageous because the results tell us about a very broad range of performance outcomes. This is probably because well-designed SJTs reflect the diverse range of scenarios that people face when in-role. As such, they help to select the most effective, most well-rounded candidates. In contrast, traditional psychometric tests tend to predict task-based performance. 

Attraction

In a competitive recruitment environment, many organisations are seeking to differentiate themselves around their brand and their culture. A survey by the High Fliers Research Centre shows that ‘improving students’ perceptions of the organisation’ is the second highest priority for graduate recruiters, behind only ‘achieving diversity targets’ in importance. 

Whilst traditional psychometric tests are generic and therefore communicate nothing about the employer to applicants, SJTs offer candidates rich insights into the role they are applying for, as well as the wider organisation. In essence, SJTs offers candidates a ‘realistic preview’ that candidates use to better understand what they will experience and therefore how attractive the organisation is. 

Providing a realistic preview early on in the recruitment process is important for two reasons. First, it helps organisations to stand out from the crowd by showing what makes them/the role unique. Does the job involve meeting a wide range of new people each day? Does it require quick thinking and inventiveness? Does it have a strong focus on personal development through stretch opportunities? All of these features, and more, can be highlighted in the SJT scenarios. The second, related benefit, is that SJTs reduce turnover for successful candidates. This is because candidates who do join have clearer and more realistic expectations.

The bottom line

Overall, SJTs are able to examine candidates in a way that tests before them have not been able to. By creating scenarios where the candidate is brought into the world of the business and assessed on their cultural fit, among other factors which would not have previously been possible to uncover, SJTs can create better pools of candidates from a diverse background who are highly suited to the job which they’re apply for.