What happens when you bring together two veterans of the psychometric assessment industry? 

The headlines: some very strongly aligned views and some opposing ones. 

This week, Robert is speaking to Ben Williams –– a Chartered Psychologist and Associate Fellow of the British Psychological Society with almost two and a half decades of experience in the assessment industry, and the Founder and Managing Director of Sten10 ––a specialist assessment consultancy working with the likes of The Telegraph, Deloitte.

Join Robert and Ben for the latest episode of the TA Disruptors podcast as they debate…

💡​The difference between skills, personality, behaviour, knowledge, competence, and motivation… and how to evaluate each effectively in the selection process 

💥Why so many skills-based hiring processes open up a pandora’s box and unleash a web of complexity so challenging to unpick that implementing skills-based hiring feels impossible… and how to simplify your thinking around what good looks like 

🔥What skill-enablers are and why evaluating them is mission critical if you want to guarantee you’re hiring fresh talent who will be able to continuously adapt, grow, and acquire new skills 

​📜 The best alternatives to CV-based hiring, and how to ensure that you don’t accidentally equate skills for experience or re-introduce bias to your process having worked so hard to root it out 

🤖 Two sides of the argument on whether self-assessment is problematic in the era of the AI-enabled candidate, the challenge of avoiding addressing the issue, and why emerging and easily accessible tech stands to exacerbate it

✨How to embrace new technology –– depending on whether your organisation is full of pioneers or you have work to do in convincing them to do something differently 

Listen below 👇

Podcast Transcript:

Robert: Welcome to the TA Disruptors podcast. I'm Robert Newry, CEO and co-founder of Arctic Shores, the task-based psychometric assessment company that helps organisations uncover potential and see more in people.

We live in a time of great change and TA disruptors who survive and thrive will be the ones who learn to adapt and iterate and to help them on that journey. In this podcast, I am speaking with some of the best thought leaders and pioneers who are leading that change.

Robert: This is our second series, and we'll be focusing on skills-based hiring. And I am very excited to be welcoming an expert in psychometrics and an old friend of mine, Ben Williams, who is founder and managing director of Sten 10. And Ben is one of the leading business psychologists in the UK, possibly in Europe as well, and somebody who I have the highest respect for in terms of expertise around understanding people, assessing people and being able to develop people. 

And so Ben and I, which is why I'm particularly excited to have you on the podcast today, go back actually right to the beginnings of Arctic Shores. And when fresh-faced out of the IT world coming into the world of psychometrics had absolutely no idea what this was all about, I was fortunate enough to be introduced to you. And to your great credit, you gave me a Psychometrics 101 on the principles of how you design and create a gold standard psychometric assessment. And I've forever been grateful for that education and the continued advice that you've given me over the years. So Ben, welcome to the podcast.

Ben: Yeah, thank you so much, Robert. Yeah, real pleasure to be here. And I've got very fond memories of those early days too. So, yeah, look forward to getting stuck in to talk about skills. 

Robert: Yes. And you are one of the people sort of best placed to really be talking about this. And there's obviously so much discussion at the moment around skills-based hiring. And everybody has got hugely excited about it. And we're seeing playbooks, webinars, podcasts, all around skills-based hiring. But the one challenge that I have around this, and for me it's the elephant in the room, is what is a skill? So can we just start with that, because there seems to be a lot of confusion around all of this. And from a business psychologist's perspective, maybe from your own personal research perspective on this, but what's your understanding and definition of a skill? 

Ben: Well, I think it's a really good question. I think...There are certain terms in business psychology that are universally understood. So what is a psychometric test? Well, it's a test that has properties that can be measured and it has certain properties like reliability, validity, norm groups, et cetera. And there's a universal agreement about what that is. There are other terms, which things like maybe capabilities, even strengths, and I would also put skills in that category that don't have a textbook definition. So I think...

Where I would go with it is say, well, look, what's the common consensus about what a skill is from the practitioners and the consultancies that are offering services around this? And try to unpick, is that any different to competency, a strength, or a personality trait indeed? Yes. I think, in my view, a skill is...

So in my view, the common consensus in how a skill is defined is a real combination of personality, of behaviour, of knowledge, of, I guess, behavioural competence in certain techniques, for example, negotiation skills or persuasion skills or presentation skills. So it's a real patchwork quilt of qualities that people have used to talk about skills. I think also, and maybe we'll talk about this in a moment, but

it's what is a skill as contrasted to what. So some people talk about a skill in comparison to experience-based hiring. Some people talk about a skill as being distinct from a competency. Some people say it's the same thing. So I mean, I guess that was a really long way of saying, I don't think there is a university agreed upon definition, but I think the way in which I've seen it used is that it's a real patchwork quilt of psychological qualities and knowledge that go to making a skill. 

Robert: Yes. So a very broad and well-thought-through explanation there. I will go back to our alma mater on this one and the Oxford Dictionary definition of a skill, which is something that you're good at. So if we take it back down to its language definition point of view and its simplicity around this, something that you're good at, then we can think of plenty of things around that. And that's what you, in many cases, you think of a skill as something, a sport. You might have a skill in tennis, for example, and we can all agree, you know, if somebody is good or bad, has a skill in that. 

And so then when we apply it to the world of work, it starts to get a little bit more complicated. So if it is just something that you are good at, then how do we start defining that? And then certainly how do we start measuring it? But let's just go back to some of the things that you said. If it is something you could, so we talked about strengths. Is strengths then another term? You said there's not a, you know, a sort of greed academic definition of it, but it is a word that we throw around a bit like a skill. So something that you are good at, you know, you have a skill in something, you have a strength in something. So is it just something that you are good at? 

Ben: For me, I think something you're good at would actually preclude for me something like measuring motivation as part of that, or even necessarily a strength as part of that. It's purely the output of what you do. I think the problem for me also comes down to how finally you dissect a skill. So in your tennis example, you can be good at tennis or bad at tennis. Then obviously you can break that down into, are you good at serving aces? Are you good at returning a backhanded volley from the baseline? I think that's the right term. So you could actually subdivide and subdivide and subdivide. That's again, where there's no universally agreed upon level at which you stop.

So actually you could say the way in which you rotate the ball in your hand before you throw it up in the air It's a certain skill to get a certain degree of spin on it Yes Now in the workplace if you take something like a very broad skill like being good at teamwork Then you can start subdividing that say okay. Well part of that might be Emotional intelligence towards the behavior of others might be emotional intelligence about yourself And then you can subdivide that and say well, okay emotional intelligence about myself. So am I aware of my own emotions?mand can I manage them effectively? So those could be two more sub skills and then you can subdivide. So it goes on and on and on until you actually end up with almost an unmanageable number of skills to assess by any objective assessment method. So that's another thing I think we're grappling with at the moment.

Robert: Now, I love that. And there are two really important points that you highlight there. One is because we don't seem to have a clear definition, or even if we agree on the simple definition of it, it opens up a Pandora's box of sub-sectors within all of that. And I've seen everybody's running around at the moment trying to do skills taxonomies. And as you rightly point out, they then start going, oh, we can't just have leadership skills or teamwork skills on this, and they then subdivide it. And then there is no way or framework that's saying how many levels do you go down before you get to a core universally agreed skill. So that's problem number one. And it slightly raises the question of, of yes, it's worth I suppose having an understanding in overall perspectives of what your skills are. 

But actually, do we need to go down to the nth level of understanding on this? But the second thing I just want to pick up to that you mentioned as well is that people confuse skill with other things that make you competent in that skill. And so that's what you're also highlighting, I think, in the tennis one in there. So it could be motivation, it could be knowledge, it could be an inherent personality trait that you have, resilience, for example, that enables you to be good at tennis in our definition of it are sort of generic one. And those, so you've got two challenges around here. One is at what level do you stop digging to find the core skill? And then the other, what about all these other things that actually make you competent at it? 

Ben: Yeah, you're right. It's, what's the enabler of the skill? So again, in the tennis example, some people under the spotlights and under pressure and Cliff Richard leading a chorus of whatever it was, what was it, what songs do we sing? Anyway, celebration or whatever. Some people, that would be a huge motivator and it will actually bring their skill to the fore and allow them to fully realise it in a way that perhaps they wouldn't if it was a damp weekend and they just had one person in a scarf clapping along to them. So skill enablers, I think, are a really important thing to consider because I think they allow the skills to blossom and to come to life.

Robert: As you know, I and Arctic Shores are very keen on this concept of a skill enabler. And maybe this obsession with skills-based hiring is slightly missing the point about trying to think, oh, we should be hiring for skills, as opposed to perhaps we should be hiring for somebody's ability to acquire a skill. And the analogy we like to use is, you know, looking at the roots of a tree rather than the leaves. If you think of skills as leaves, they're things that you're going to refresh. And I think Boston Consulting Group said you've got two and a half years is the typical lifespan of a skill now with all the changes that are going on. So is that where you think we should be focusing rather than on the skill and the hard skill that somebody might actually require or already have and actually because we're going to need to refresh them the whole time maybe you need to be looking at skill enablers. 

Ben: Yeah, I mean I think it comes down to a combination of what's your purpose for assessing these skills. So if it's to identify training and development needs then actually getting an understanding of how good people are currently and where they might need to be in the future is actually a good thing to do to have a bit of a benchmark.

If it's something where you're looking to recruit, especially around early years recruitment and saying, right, what's their level of skill at political sensitivity and their communications, then obviously that's not gonna be fully developed yet, but actually looking at the potential for that. So whether it's around some kind of measure of personality traits around their propensity to analyse the needs and motives of others, perhaps a little bit of interpersonal adaptability.

Those are the building blocks that will lead on to the skill of political sensitivity, which might not be fully developed yet. I think also that it comes down to this, whether we're defining skills as or skills-based assessment as looking at someone's CV and saying, have they, can we put a tick into each of these boxes? Because I think that approach is the one that's fraught with problems from an inclusion and diversity perspective, and lots of other perspectives in terms of like socioeconomic status and what school you went to and everything is going to supercharge your ability to complete that CV well as your use of generative AI. 

So I think that the yeah if we're contrasting it against the use of a CV then I think people are kind of kind of missing the point of it is actually we should be looking at those building blocks the raw material that we can shape into the skilled employee of the future. 

Robert: Yeah and that's such an interesting point because you're right that you look at the reason why people have got very excited about skills-based hiring, and they say, oh, it's because we don't want to rely on education or past experience and job titles as defining whether somebody could be a good fit for our organisation, which I completely get. The worry is that you add the frying pan into the fire on this one if you're not really thinking it through.

So if we talk about skills-based hiring as being the problem we're trying to solve around this is that we don't want to use education or experience, then what's the replacement for it? And if the replacement for it is, oh, you need to hire for a bunch of skills that we have done some analysis on in our organisation, then that throws up another challenge around all this about, okay, well, what are those skills? Are they universally agreed? And, and how, and at the moment, how are you going to assess for them? Because at the moment, what people are doing is looking for a word match, a word association. 

So I've got, I need leadership skills. Okay, well, maybe you go down a couple of levels from that, but I've even seen it just at leadership skills. So then you go out to market to go and hire people who put leadership skills in there. Does that raise any concerns for you two? 

Ben: It does. I mean, I'm just hearing you talk that I'd be interested in your view on this, Robert, but there's there's part of me that wonders the extent to which skills-based assessment is a bit of a marketing tool. So a lot of the time, people who offer services in this space need to have something new to talk about. So whether it's, you can't just talk about competencies year after year, decade after decade, you have to find a new spin on it. Actually, I can see taken to the nth degree, a skills-based organisation where you have a pretty comprehensive understanding of every employee's different skill levels across multiple different skills really does allow people perhaps to facilitate horizontal career progression. 

Maybe if you're working in a consultancy firm, then knowing the pattern of skills and knowledge of your consultant's team, you can assign them to three, six-month projects really easily. And that's where I think, that's where I would say specifically a skills-based organisation is fully realised. But that's actually, most organisations are a long way from that. And actually it would be a hugely transformative change project to be able to get to that stage you wind it back and say, well, if we're not going to embrace skills fully across the whole organisation, what are we really saying here? We're actually saying, let's hire people, let's not hire people against their number of years of experience and their qualifications. And actually, we've been saying this for decades. As psychologists. So yeah, so it's, to what extent is it truly a new way of doing things if you're not going to embrace it as an entire organisation? 

Now, the other thing that has surprised me in my 24 long years in the business is that when clients ask me what's the latest in the world of assessment, for some, actually introducing a structured interview with a set of questions is a leap forward. For others, it's behaviour-based assessment. For others, it's eyeball tracking and galvanic skin response and things. So I think in that sense… the publicity and the push around skills-based assessment could only be a good thing because it's saying, look everyone, this is what we should be focusing upon. Then, I guess the job relevant aptitudes rather than perhaps the CV assessment process that some organisations might be still focusing upon. But going back to our earlier discussion around skill enablers versus actual skills themselves, I think skill enablers is a more sensible approach because then it says, well look, once you've got that knowledge you can then plan their onboarding and their upskilling in a way that gets them those skills in the quickest possible timeframe. 

Robert: Yes, you raise a couple of interesting thoughts around that. One is that competencies and somebody's therefore capability to learn something or pick up a quality in the role is something we've been talking about for a long time. So there does feel an element of the emperor's new clothes about all of this, that it's more of a marketing spin around this. But I suppose the reason it's got some traction around this is because of the way that you framed it, the logic of understanding what skills that you've got today, how they're going to change and what skills you're gonna need for the future and how you might cross-train people from that, absolutely strategically makes huge sense. I can see why organisations are doing that. 

But that's internal, how do we develop our organisation's strategy? It doesn't necessarily translate itself straight into, oh, we need to hire, therefore, for skills. And I think that's where people have got confused around all of this, that the skills-led organisation as a strategy makes absolute sense. And then somehow this has been translated down to talent acquisition, which is, right, go out and hire for skills. And they're sitting there going, well, I've been trying to do that for the last three or four years, and we've got skills crisis.

So how's that gonna work? Oh, it's okay. We've got AI now, which we didn't have before. And now that we've mapped out these 12,000 skills that we've got across the organisation, it's okay. We've got a piece of AI that's now gonna go out and find all these people, either passively or through inference, who have got similar types of skills and you can go and source them. And they're sitting there going, this is total madness because we've been trying to do this. And where does...

Where does AI come in all of this that's been trained and tested and validated? Because that's the other thing. If it's just a word match on all of this, then all the stuff that you and I share as a principle of how you should do selection on this is out the window in terms of reliability, test, retest, all the things that we were talking about when we first met 10 years ago, saying, Robert, these are the things you've got to make sure you get right.

Ben: And actually hearing you talk there, it makes me think that skills-based assessment is less a revolution in selection methods. It's almost more of a revolution in job analysis, which often is, when I do training course in job analysis, I describe it as the most boring area of occupational psychology because it hasn't changed in decades. It's still the same technique. But I've heard that some, I know Deloitte talk about skills-based organisations and they say, actually, dispensing with job descriptions altogether and instead having skills matrices is what the pure skills-based organisation looks like. So almost you don't recruit for jobs, you recruit for tasks or skills, which again, I can see as a bit of a revolution in terms of how you look at analysing jobs and structuring organisations. But at the end of the day, you come down the funnel to say, well, how are we gonna get people into those positions, whatever they might look like? Then it comes down to, well, do we need a completely revolutionary approach in how we're assessing them? I mean, yeah, potentially not. It's like, actually, we just need to apply good, tried and tested, valid, reliable, fair tools to solve that conundrum and to place people. 

Robert: It is, and let's just double down a bit on that one, because if TA teams are being pushed now to go out and, oh, we're the skills-led organisation now, you need to go and do skills-based hiring.

There's a bunch of tech out there that's saying that we can match these words for you, so don't worry, you know, you don't have to get on top of all of this. You start getting into how are they going to assess and do this matching. And let me just share an example with you as to where I see some of the flaws on this, because this is so important, people need to understand the things that you and I worry about when we're building an assessment that an AI model doesn't worry about. They're just worrying about, can I come up with a set of rules and parameters that take an input and create an output? So let's say we need to go and find a match then between these are the skills that we need and these skills that are out there in the marketplace. So you require people to self-certify their skills. So my daughter was doing this on LinkedIn. 

LinkedIn making a big push about, you know, go and put all these skills in there and that'll enable you to be found by organisations. So as my daughter's going through and LinkedIn are prompting her, because she's now applying for jobs, about what skills that she's got, they're coming up with things that she wasn't necessarily familiar with that were a skill for her, so like project management. And her instinctive response was, okay, I don't really know much about project management, it's not something that I'm...

qualified in, so I'm not going to take that as a skill. Whereas I'm going, no, this is something that you've demonstrated as somebody who's doing a PhD, who's run a sustainability project. You have got all the skills, and whatever level we might be talking about that, that means that you'd be great at project management. But you're not taking that, because you are thinking your interpretation of this, because you're self-certifying, is that I'm not worthy, I'm not up to it.

And that seems to me a big risk on this is how on earth, if you're gonna do any of this matching, are you gonna decide whether somebody's got the skill and then what level, advanced, medium, beginner, who determines all of those? 

Ben: Yeah, I mean, if you think about, you gave the example earlier of an organisation which has, let's say, 12,000 skills that need to be assessed. I mean, this goes back to the reason why competency frameworks are written as they are, they have a clear top level definition, they have positive and negative behavioural indicators so you know what they look like. And if your daughter had seen what we mean by project management is seeing a project through to meeting deadlines, monitoring progress, adapting if needed along the way, she probably would have said, oh yeah, that sounds fine. But there's all sorts of things that come into your head when you see the word project management. Do you think I need to be Prince 2 trained? I need to have gone on some kind of formal accreditation?

and it will hold you back. And as I know you were talking about earlier, there are gender differences as well in people's propensity to say, well, I meet these job requirements and there is that slight split in that women will generally say, look, I need to be meeting 95 to 100% of the job description before I say, okay, I'm gonna go for it. Whereas in general, again, men will be far more inclined to go for a job if they meet 75% of the skill requirements and say, oh, I'm sure I can wing it. So it's not just a definitional problem but there are also psychological aspects at play that could mean self-assessment of skill level is problematic.

Robert: It is and we need to be aware of those and we need to be talking about them now before organisations suddenly start changing their recruitment practices to bring in technology that we won't know about for many years has brought about a societal adverse impact as opposed to a societal improvement, which would be dreadful and catastrophic with all the improvements we've been trying to make about DE and I over the last few years. So when we think about the skills-based hiring then and what we need to get right on all of this.

At worst from all of this, we are just challenging, or actually probably at best out of this rather than worst, but at best out of all this, we're challenging people to rethink their recruitment process on that. That is a good thing. We know that there can be improvements in all this, take bias out of it, and you gave the example of just a structured interview as an example. So if we take the starting point of this, which is, oh, we could just start improving the way that we look at it. And if we change the job description and flip it to being less experiential centric and more about competencies, strengths, motivations, values and then start thinking, well, how do we design a recruitment process that brings those things out? Then that could be a good thing from all of this. 

But we need to be very very clear as to what a good process looks like and then use tech to improve it rather than I've just been told to put in skills-based hiring there's a piece of tech that says they do it so I'll just follow that cue. 

Ben: Yeah absolutely I think it's the impetus to re-examine or to examine for the first time one's selection process is to make sure that they're fair and relevant and the job descriptions I mean rather than saying perhaps look just put together a role profile and a few bullet points for the person spec and a few bullet points for the role requirements, the essential and desirables, actually a move towards doing that in a more structured evidence-based way and maybe having some kind of taxonomy to choose from that's developed per area will help to come up with better selection methods. 

I think that's all well and good. I think then leaping to think we need to throw out all of our existing assessment methods. And I think that's what frightens people sometimes. They say, well, I've been charged to, we're becoming a skills-based hiring organisation. Does that mean everything that I'm using at the moment needs to be chucked out as a result of that? And actually, no, but you need to be mindful and do a proper review of it and think, right, well, what validity evidence either have we gained or can we get from our test publisher partners or from research that shows that this type of tool works well in this type of environment. And that's what I'd be advising to our clients is that, yes, well, let's revisit it from almost like a bit of a health check. 

So how do you go about identifying what skills are important for the job in the first place? That gives you the lowest form of validity, which is legally defensible, which is content validity, and then progress to saying, right, well, have we got the most efficient selection methods to assess those? And of course, it's not all about just the most brutally, coldly efficient assessment methods possible. You also want to factor in the candidate experience, you want to factor in whether that's gonna make you stand out versus your competing organisations that are also trying to attract your talent, length of the process, et cetera, et cetera. 

Robert: So, yeah. A lot of things you wanna get right. No, and you're right, and I think it's important when people are thinking about, we've got an opportunity for change on this that they're factoring in all the relevant pieces of information. Now you take the point, you make the point rather that we don't have to throw out everything that we've used up to date because in many cases that's improved to be useful and has helped us in the past. I get that. But we have generative AI now. So I'm just gonna prod you a bit on this. 

Well, I know you have, you know, you've been thinking about yourself about how you assess for generative AI skills. And we'll come to that in a second. But don't you think we have a problem at the moment with some of our skills, not to say our skills, but our recruitment methods using existing tools with generative AI? I mean, I'm hearing stories, even things like video interviews now where people are just reading from a script.

Ben: Or even live ones like scrolling down the screen as they're talking, just shown to the left of their webcam and they're just reading it off. 

Robert: You can hardly even see. 

Ben: So I think there is concern there. I think something that I've always been quite interested in looking at the evidence around is people's motivation to cheat. And I actually read through your wonderful playbook, Relations to Skills-Based Assessment, but I was actually surprised by the statistics about the low percentage of people that admitted to using chat GPT as part of their... I mean, I think it was around 16% of minority applicants who said they'd use chat GPT as part of their application process and maybe it was 14% of non-white. So I think it was a... Yes, it's a sizeable minority of people that are saying they're using it, but I would then say, right, well...

Of that, let's call it 20% if we're being a bit more kind of like worried about it. And we say, well, 20% of people use it as part of the application process. So what stage do they use it? So I would imagine the vast majority, and again, this is just conjecture, but the vast majority of that 20% is to help me write my CV and my cover letter. And to get that absolutely spot on. And again, decades of research has shown that CVs and cover letters are poor predictors of future performance. 

So I think the fact that chat GPT has come through with a scythe and rendered those even less important, I think is no one's gonna shed too many tears over that. I think then we think, well, of that 20%, what proportion of those might use it to cheat on other aspects of the assessment process? So can they be sitting there typing out verbal reasoning questions and then saying, give me the right answer, true, false, or cannot say? It's gonna be a small percentage. It's not to say that that's...right or acceptable and ideally it would be zero. 

And this is again where I'd say, well, perhaps that's an argument for retesting when they come in face to face to be sure. And maybe not giving them the test again, but just as part of an assessment centre saying, let's give them some applied critical reasoning tasks to do and see how they perform. So I would say generative AI is a concern, but I'd probably say that it's less of the death knell than it could be at the moment.

I could foresee a plugin, and this is where, if I was being very Machiavellian, I would say actually, as a side hustle, I will develop a plugin for Google Chrome that will screen read, and at a click of a button, it will solve any reasoning test with, and you're about to tell me it already exists. It already exists, exactly. 

Robert: You can go out and find it on Reddit. Right. And the way the generative AI and OpenAI is working is going down the Apple route - you can create your own little app and then market it through there as through the open AI marketplace and platform as something free to use for people and you can then use it for marketing or whatever. 

Ben: I wonder what the law of it's not unintended consequences but there will be a certain demographic of individuals that actually manage to find this at least at first. 

Robert: Yes. And you. Well, I'm going to give you an example. Yeah. So this is often been sort of mooted as a really is people's motivation to cheat on this any different now than it was years ago. And traditionally, we knew that some people were trying to cheat and manipulate it was less than sort of 10% of applications. But I was talking to somebody the other day about Glastonbury tickets, which are probably every bit as hard to get as a job, it seems at the moment and you have to sort of sit there, you know, right at the moment of time and getting ready to go and put your little application in to get Glastonbury tickets. 

And I then, and it sounded absolutely impossible. And so you sit there going, oh, well, it's just luck of the draw and it didn't work out for me. And then I caught up with somebody the other day, said, oh yeah, I came about somebody through a WhatsApp group and there's a back door way to get Glastonbury tickets now. I can't remember exactly what it is. And even if I did, probably wouldn't necessarily tell everybody.

But there's a backdoor way, there's a separate server where you can go and put the IP address in and you can get tickets. And, you know, my friend and I, and we shared it on the WhatsApp group. And then suddenly it turns out that there are these group of people that have found a way of getting a backdoor, knowing that lots of their friends haven't been able to do it, some of which they share, some of which they don't. But then over time, that backdoor just gets opened up because somebody tells somebody else, saying, oh, I'll just pass on this one little secret to you.

It doesn't feel like you're cheating. It just feels that you're not a mug. Yeah, yeah. And that is where I think it's different. Whereas in the past, you would have had to have paid for something to enable you to cheat and it felt wrong. Whereas this time it feels, well, it's a calculator out there. If there's 20% of people are using it and they're getting jobs, I'm just a mug for like the Glastonbury tickets of them. Oh, I'm just gonna do the ordinary application knowing that somebody else has found a back door. 

Ben: I do get that. And I...there's definitely an arms race going on between certain test publishers and, and chat GPT to try and, to try and combat it. I guess, um, I'd be, I mean, if I was using the Glastonbury analogy, when I went to Glastonbury, the one time I went to Glastonbury, my legs absolutely killed me trudging around for ages. I think I wanted to see the who, but by that time I was absolutely knackered and I was actually on the way out the door, rained on relentlessly, but I got a ticket and Am I the right kind of person to be at Glastonbury? I'm not what you'd probably call a free spirit in many ways. I'm 40 years old now, and I actually wanna be curled up in bed. So actually I could use this back door, end up getting into Glastonbury and being thoroughly miserable and wanting to leave early. And to what extent are people motivated to get jobs they're not suited for?

That's another question I'd ask now. It's a good question. Could you be, I mean, again, I think the reality is that actually a lot of people don't care because they said it's my first job. If I'm going to get 50 grand a year at a managing consultancy firm, I'll go there and I'll prepare to burn myself out. But I guess then you've got, yeah, it's tricky. I mean, again, some of the data I saw another firm that I won't mention the name of presenting in December about what they were interesting because if you design, so an SJT being a situational judgment test, so a little scenario, and what would you do in this scenario? And I think some of them can be designed in a way that's blatantly obvious for something like chat GPT to cheat exceptionally easily. So what's a good way to respond to interpersonal conflict in general, option A, B, C, or D, and it can do it? It's far harder when there isn't a universal right or wrong answer. 

Now it's not to say it's impossible because ChatGPT will be trawling the web and it may well get that there was a bank that I did some work for a while ago and one of their competencies was around straight talking, so being quite no-nonsense, delivering it like it is. If it upsets people it upsets people but it gets the job done quickly. In a situational judgment test that asked, well look, how would you go about delivering this difficult feedback to a colleague? And one of the distractor options was something like I would talk about all of their strengths and talk about how much I appreciated them, but then also mention the fact that they could potentially improve here versus the right answer which was to tell them this is unacceptable, it needs to change, otherwise xyz. I would have thought that ChatGPT would find that quite difficult to assess for that particular application that's the right answer, but for this application, it's this answer and maybe for this job it's the middle answer. So it's that kind of...

Robert: Yes, I have to challenge you a bit on that one. Yeah, go ahead. It's interesting because it's now moved on so much that you can feed, particularly chatGPT4, you can feed it the career website where it says, we like straight-talking people. So you say to chatGPT, right, look through the career website and give me a list of the type of things they're saying in terms of personality traits they're looking for because you've got to say it on your website because it's your way of being able to filter. You've got to allow people to see, well, what kind of people do you want? Say, oh, we want straight-talking people. Right. Here is your list of criteria. Now, answer these questions with that perspective. And we've tested on this. 

You can get ChatGPT to be, it's inherently conscientious because it's been trained to be that. But you can get it to dial down the conscientiousness and be a bit more narcissistic about it if you want to. And so you've got that sort of challenge of actually, you can configure ChatGPT to be able to answer it in a certain way. But I take your point about how much people are gonna be motivated to do all these things, but I suppose it's part of just being aware, I suppose, of what's going to come out. But I'd like just in the last sort of few minutes, just to talk about what sort of skills, and what do you think, and we talk about skill enablers. So ultimately, what skills-based hiring should be doing is to prepare the organisation for the future and what they're gonna need, and they're gonna need to be more adaptable on that.

And we talked a little bit about those sort of skill and ablest and foundational ones in there. Do you, you know, I suppose I'm just telling you what the sort of the Arctic shores perspective is on, we've got three of those, but it'd be interesting to get, you know, your perspective on whether you think there's more than three or less than three, because people talk about meta-skills and there doesn't seem to me to be any universal definition. But in my sort of 10 years of observing and talking to lots of different companies, there seem to be.

Three in the research that we've done backs this up. Three things that largely every organisation and job requires. So one is what we call your thinking style. So your ability to learn information, but how you might analyse information, how you might process information. And I think that's an interesting point about cognitive ability and all this because that gets missed out. But then the other one is interpersonal skills, so how do you interact with people or not liked interacting with people. So you've got to factor that in. And then the last is, is what we call self-management, which is how do you like to structure the way that you work and potentially work with other people. So is it, if you, if you have those three main buckets, that seems to me to cover the main things from which then people can learn whether they're going to be a project manager or a negotiator or whatever.

Ben: Yeah, I think that's comprehensive. I think that element of people's raw reasoning skills, so thinking style is one thing, thinking aptitude and that kind of raw reasoning skill, I think is often a predictor of people's ability to take on board concepts quite quickly and see the implications of those and then to learn. But I think you take a bit of that, sprinkle on a hefty dose of motivation to, so learning orientation, a little bit of, I mean, one of the big changes in personality theory in, well, a few years ago now, was moving from a five factor model to a six factor model of personality and introducing the trait of humility. So actually the ability to realize that you're not the fully formed article yet, actually you're willing and receptive to feedback. So I think that that would be part of the recipe, I would say.

And I think, I mean, almost like interpersonal skills, interpersonal, whether it's intelligence or interpersonal orientation or emotional intelligence, I think is really important too. As to whether, yeah, you'd have to think with each of those, how does it pertain to the acquisition of the skill as opposed to just being good at the skill? So interpersonal aptitude, should we say, well, that might help you to be a good team player but how does it help you to learn and acquire a skill? And maybe it's you're more likely to engage with a mentor, you're more likely to be at a table with other people and looking at how they're doing it and absorbing it into your own work style. 

So I think that the Arctic Shores way of doing it, yeah, it sounds really comprehensive, it sounds good. It would just be always about linking it to, how does that relate to acquiring a skill as opposed to just being good at the skill? 

Robert: Yes, I think such an important point, that difference between acquiring it and just being good at it. And just back to the earlier point that you said around that, you know, cognitive ability and intelligence because that we know has been, you know, one of the strongest predictors, although some people have, you know, questioned some of the earlier research around that. But generally cognitive ability has been a predictor of performance in the workplace. And yet we do test and have always test cognitive ability in quite specific in slightly narrow ways. 

And one of the things that I've always struggled with because we're so task-based at Art at Shores, it's all about how you do things and how you build up your answer rather than whether you get the right or the wrong answer. Why is it that the way that we measure intelligence traditionally in the psychometric world just it's right or wrong. It's, you know, here's your multiple choice. You've got to distract her and here is the right answer. And it's it's we've not moved like education. Is it just been a technology piece or just a actually we're very happy with a right or wrong. We don't really care about your workings out. 

Ben:Yeah, I guess it's the distinction from the educational space is is what's the purpose of the assessment of someone's reasoning skills? And in the workplace, often if your workings are faulty and actually, or part faulty, and you end up getting the wrong answer. Actually, what we're most interested in is do you get the right answer ultimately. So that, correlating that do you get the right answer or not with how do you perform in the job is, I guess, being seen as the ultimate question we're trying to answer. Whether we can develop you along the way, so in an educational context, it's critical to understand where did it break down there? Where did they do that?

So I would say that where organisations would be missing a trick with a more traditional assessment is if they said, look, we do acknowledge that we're going to take on people that maybe aren't the fully formed article yet, and actually we're taking the onus upon ourselves to help them on that development journey, then a tool that can break down the workings and say a little spike and a trough here to show where they're strong and where they're less strong would be really valuable. 

I think probably most organisations up to this point have been more focused on getting people who are, perhaps there's less emphasis for them on taking on board a development agenda to say, look, we really want to upskill these people. And it does relate to, I guess, a broader topic than I'm quite interested in, which is students' work readiness in general. And whose responsibility is it? Is it the university career services? Is it the employers? And I think a lot of the employers think it's the university career services, but a lot of the work that I'm doing sees employers taking on more responsibility for doing that and producing the fully formed graduate or apprentice.

Robert: Yes and it probably actually does have to be a combination of the two because things are shifting so fast on this but it comes back to what do you want out of somebody who's come from a school, the education system or from a university. You want somebody I suppose who has got the skill enablers on all of this, that then it's up to the employer to then mould and finish the article. But if they haven't got the skill enablers, they don't know how to interact or communicate in a public way with confidence. I mean, this is some of these things, we may talk about them as skills, but actually they're just the basics of being able to be work ready. Which is why I like thinking about things, if we think about skill enablers, what are the things that we need out of the education system that means that when somebody goes into the world of work, they are gonna be able to run with whatever training that they're given and the set up there. 

There is still, I think, quite a lot of work to be done around that, and there's a lot of rethinking that needs to be done by universities on that. And I know you're working, I think you've got a programme around that to help facilitate it, but it does need a rethinking because the future of work is different from what it was in the past. 

Ben: I think an interesting question would be, are skill enablers developable? So skills are developable, are skill enablers themselves developable? And looking at your definition of them, I would say some of them certainly are, but some of them are more kind of raw building blocks, raw world attitude. And that's, and again, that's where you could imagine universities helping people to say, well, look, actually, maybe you're not predisposed for these certain aspects of skills acquisition, but actually here's the behaviours that you could introduce to your working persona that would enable you to gather those skills more quickly. And if you're up for that, then that's how you can more quickly get up to speed and get recognised and yeah. 

Robert: Well, I think that that's brilliant. And, you know, one of the things that just sort of round everything off around all of this, the reason you and I do what we do is that we believe that self-awareness, that somebody understanding who they are and why they do what they do, is an incredibly powerful and important way for that person to then be able to contribute in the community, in the workplace, and wherever else it might be. And one of the bits that I've most enjoyed about psychology and sharing many discussions with you is that I wish I'd had that self-awareness much earlier on, you know, coming out of education and rather than having to sort of build it up in the workplace. 

And if there's one thing that, you know, we could really do to improve the lot of the next generation coming through the education system is that they just understand themselves a bit better and support them in the things that they might need to develop more in order to be successful in the workplace. And I know, you know, you and others are doing a lot to support that. So Ben, as always, it's been a fascinating discussion. You know, we've covered a lot of different things, but as always, you've been so eloquent about it, so thoughtful about it and really appreciate you coming on the podcast.

Ben: Thank you for the discussion. Thank you very much, Robert. Been a pleasure and likewise, yeah, been a really stimulating discussion.