What is the opposite of Advanced?

Posted on April 26, 2012 by Paul Kearns

The reason I ask is because I have just come across an “AIM Practice” document entitled “HRM and Innovation Assessment.” which claims it can help “assess the extent to which HRM in your organization is oriented towards innovation.” If you have never heard of AIM Practice it is the sister organisation of AIM – the Advanced Institute of Management (Research) – and if you have never heard of AIM it might be worth asking yourself what makes these academics think they are so advanced? Sometimes advanced is not good news – as when cancer reaches an advanced stage.

The Deputy Director of AIM is Professor Andy Neely and I first came across Andy in 1998 when I was invited to speak (about Organisational HR Maturity) at the ‘First International Conference on Performance Measurement’ at Cambridge University’s Judge Institute of Management Studies. One of the papers presented at that conference (by Andy himself and Mike Bourne) was entitled “Why do performance measurement initiatives succeed and fail” and I have been trying to convince Andy ever since that one of the main deciding factors will be whether or not performance management is an integral part of a well-conceived, complete HR strategy. I failed to get him or his academic colleagues interested. In fact HR strategy has never really featured much in AIM’s research; so whatever they think they mean by ‘Advanced’ it certainly has nothing to do with advanced HR thinking.

Evidence-based HR does not loom large at AIM either. Yet the very reason this blog/book exists at all is because the main area of management that still lacks evidence is, funnily enough, not operational management but human resource management. This is because producing performance data on the way organisations manage their human capital is so problematic. If anything, AIM should be concentrating on EB-HR more than any other aspect of management research. Instead it is just dumping more and more HR products into a market already awash with non-evidence-based gimmicks.

So how good are these products being peddled by AIM Practice? How about their “Hot Spots toolkit – Taken from Professor Lynda Gratton’s book ‘Hot Spots: Why Some Companies Buzz with Energy and Innovation – and Others Don’t”. Odd choice when one considers the hot spot that Gratton’s exemplar organisation, BP, managed to get itself into.

If losing $$$billions in the Gulf of Mexico is advanced management what should we expect from the ‘HRM Innovation Assessment’ toolkit? Apparently it only “takes up to 15 minutes to complete.” and “When all of your nominated participants have completed the questionnaire you will be notified, and your personalised report will be available online within 24 hours.” There is no evidence offered as to what impact this ‘report’ might have so any thinking manager is bound to ask which elements of these off-the-shelf products qualify as ‘advanced’? How arrogant are these academics and how advanced is their management ‘practice’ if they think they can produce a report about innovation without even bothering to come down from their ivory tower to visit the company concerned? They would do well to heed Toyota’s very old principle of genchi genbutsu – ‘go and see’ for yourself.

AIM is the very opposite of advanced and there are plenty of antonyms of what that means – going back, receding, regressive, unsophisticated, even uncivilised and any organisation that does not take its human capital into account is certainly that.

Those bloody “Meetings Bloody Meetings” videos

Posted on March 28, 2012 by Paul Kearns

The whole point of EBM is the facilitation of learning – both individually and organisationally. It requires a culture of learning from evidence of both success and failure, in equal measure. Yet many learning and development professionals are the worst culprits for ignoring evidence of training failure. They persist in putting people on courses where training is not the problem or where the ‘trainees’ just do not want to learn. It does not matter as long as they get paid for delivery, not outcome. This phenomenon is best exemplified by one of the oldest video training products on the market – ‘Meetings Bloody Meetings’.

I still remember, when I was a very young and naive training manager, watching the original John Cleese video for the first time. Cleese was better known as an actor who had a silly walk, complained about dead parrots and offered a dire customer experience to the unfortunate residents of a third-rate hotel in Torquay. Nevertheless, I thought his video was humorous and made some telling points about how meetings go wrong. So I hired it – once. Never again: not because it had solved my problem, but because it hadn’t.

So I cannot believe that the producers have the nerve to issue a 2012 version; when the evidence is so plain that meetings have not improved. They are more prevalent, more time wasting, more ill-conceived, more hastily convened and managed just as badly today as they were over 30 years ago. So if you were just about to run this video on a management course I thought I might suggest a few evidence-based questions before you inflict it on yet another, unsuspecting generation?

Question 1. Why do you think there is a need for this video?

Answer. Probably because you still hear the same refrain that I heard – ‘there’s too many bloody meetings in this place!’ or ‘when is anyone expected to get any bloody work done around here?!’ If you took that to mean your organisation has a problem with ‘meetings’ then I’m afraid your analysis and diagnosis might be inaccurate.

Question 2. Have you measured this apparent problem?

Answer. That was my mistake when I did not know any better. I reacted to an apparent problem rather than a real one. An evidence-based management problem is one that has been accurately identified and measured. So has anyone ever measured how many meetings take place in your organisation? If they have, did they also distinguish between the good meetings and the bad ones? I guess not; because that would mean you have to tell some very senior people they could not run a third-rate hotel – I mean meeting.

In fact, your job might be on the line if you so much as hint that maybe, just maybe, it might be a good idea to start assessing the performance of the people who run meetings? It is a lot easier for those in authority to presume they are skilled in managing meetings and not give a second thought to those whose time they waste or even consider whether they might be dragging others away from more relevant, pressing matters.

Question 3. Even if you identify and solve the problem – what could it be worth?

Answer. Obviously the first benefits would be fewer meetings, less wasted time, more efficiency and lower costs. Meetings should also be managed better with clear objectives and a minimum of unexpected AOB. They should only take as long as necessary and ensure the right actions are agreed and responsibilities allocated. They will also check progress.

Probably much more valuable though would be a different culture altogether. Really great meetings are those where you are allowed to voice your most honest thoughts without fear of retribution: where expertise is respected and evidence-based recommendations given much more weight than rank or gut feel. One of my own favourite indicators of effective meetings is simply to ask how many attendees would just walk out if given the freedom to choose?

You know what? Those bloody training videos are not solving problems they are bloody well perpetuating them.

“We’re alright as long as we’re all wrong.”

Posted on February 28, 2012 by Paul Kearns

The Emperor’s new clothes by Thorarinn Leifsson www.totil.com

Time and time again this silent mantra, this fatal assumption that there is safety in numbers, has been HR and Learning’s undoing. While you are all comparing management competence models no one will be stupid enough to break ranks and admit that they don’t work.

If you all talk about ROI and profess to use the same 4-level model of evaluation you can just about maintain the pretence that your training is a good investment.

Using 360° feedback might mean you all end up disappearing up your own orifices but hey, while you are all up there together, you can pretend that there is some illumination in the darkness.

You can all keep your bosses happy by sending them on expensive leadership programmes that help them to feel good about themselves while masking their inadequacies.

You dare anyone to challenge your diversity policies and ignore the very obvious evidence that there can be a serious and problematic downside.

Well, maybe your dirty little secrets were OK once, while everyone kept their mouths shut, but you were bound to be outed by evidence in the end weren’t you? Now what do you do?

You could plead ignorance perhaps; although that is no defence in the eyes of the law. Be a bit difficult to now say you did not realise that training without identified business needs is nonsense. You could throw yourself on the mercy of your ‘customers’ when they realise you have made a fool out of them as well? I don’t fancy your chances though, do you?

I suppose you could use the same tactic again of jumping on any passing bandwagon (HR analytics?) in the forlorn hope that it might either bamboozle people further or at least buy you a bit of time? You haven’t learned have you? That new fad is bound to suffer the same fate as all the others eventually.

Old HR habits die hard don’t they? If you are only comfortable being part of the crowd then at least recognise that the numbers game is going in the opposite direction now. The façade of HR convention has finally crumbled and no one will be rushing to join your shrinking minority. Also, you are up against much more powerful competition than ever before. The evidence-based can prick every one of your non-evidence-based bubbles with absolute ease and impunity; one at a time. That can be a very painful process.

There is only one way out. Come clean. Get it off your chest. Start working from evidence. It’s a lot simpler and much less painful than what you are doing now. The real beauty is that you don’t even have to explain yourself to your customers after all – they are no more evidence-based than you are – why do you think it’s called evidence-based management? – and you will be one page ahead.

Whatever criticism they might want to hurl at you is really just self-criticism. They always knew your competence dictionary was unadulterated gibberish so why did they pretend to support it? They know they still have serious problems with managing performance (that’s their dirty little secret) so they will welcome anything that makes their lives easier: evidence-based performance management will certainly do that.

Go on – you know you don’t like feeling naked – go and get some clothes on. Get some evidence.

How many Deloitte HR consultants have found their stapler in a jelly ……..?

Posted on September 21, 2011 by Paul Kearns

….. probably all of those who contributed to Deloitte’s “Human Capital Trends”.

Any fans of the TV sitcom ‘The Office’ (either the original UK or US versions) will know the key character is the office manager – David Brent/Michael Scott – (played by Ricky Gervais/Steve Carell) and if you had to sum up their characters in one word most Brits would probably say – complete pillocks – defined in the Oxford English Dictionary as “a stupid person”.

But this does not begin to capture the essence and richness of the word pillock, whose origin dates back to the ‘mid 16th century: variant of archaic pillicock – penis’. Someone who, while trying to stand proud, becomes a parody of leadership and management. Prime examples from real life include George W. Bush and Tony Blair; both well known for their nonsensical, vacuous use of language.

But the real genius of Ricky Gervais’ creations is that they are characters of such sublime pillockry that they are beyond parody. The authors of Deloitte’s “2011 Human Capital Trends” fall into the same category.

I recently had the misfortune of finding myself diverted by this report and my immediate reaction was to ridicule its pretensions and verbosity but words fail me – it is beyond ridicule. As someone who is well-used to the rhetoric of large consultancies I still find it difficult to conceive how an organisation like Deloitte can employ so many supposedly intelligent people, who take themselves so seriously, and yet are happy to discharge such large volumes of untreated sewage into the HR ‘sea’. Or perhaps I under-estimate Deloitte – maybe they know exactly what their HR clients like to wallow in?

Right from the opening paragraph Deloitte sets its course in a determined effort to pervert the use of the English language – saying nothing in as many words as possible -

“Events of the past few years have brought sweeping changes to business and new challenges for the Human Resources (HR) leaders who support them. Two broad themes — innovation and global markets — have taken a front-row seat in human capital organizations around the world. From evolving technologies and process breakthroughs to new organizational models, new markets, new customers, and new approaches to talent, the power of fresh thinking runs deep and strong.”

… before leaping to recommend a peculiarly un-strategic and oxymoronic HR ‘strategy’* -

“Forward-thinking organizations should consider developing an explicit strategy in each area, even if that strategy is to wait and see.”

Let me detain you no longer on this journey to nowhere. Suffice it to say that if you check the internal phone directory at Deloitte’s you may not find a David Brent or Michael Scott listed but if you drop in to see the authors of this report one day you should be able to spot them quite easily – they will be the ones with their staplers in jelly – or jello, for those of you who speak American.

*“Avoiding default mode

A strategy is a conscious decision to cope with whatever challenges you might face. Therefore by publicly declaring your strategy you are acknowledging your determination to do everything possible to avoid being a victim of circumstance or slipping into the default modes of complacency and inertia.”

From HR Strategy (2nd Edition) p.47

We can measure what we call ‘talent’ even if we can’t define it

Posted on August 12, 2011 by Paul Kearns

This is part one of a 2-part post.

One of the earliest pieces in this series looked at a truism in human performance – any population will form a normal distribution – the bell curve – the highest and lowest performers will be minorities and the bulk of people will be somewhere in between. As a truism it gives EB-HR managers great confidence when they use it to analyse organisational strengths and weaknesses.

It is also perfectly adaptable to any type of organisation. If we take Goldman Sachs as an example, we know they offer some of the highest rewards in banking, which might lead you to believe they only employ great performers (i.e. their performance curve is skewed to the right), but the performance curve is always relative, so even Goldman Sachs will always have some people who are deemed to be under-performers relative to what their best achieve. It is also worth adding that their ‘worst’ might be much better than the ‘best’ their competitors have to offer. In effect, other banks start from a lower base (as shown in the graphic).

Goldman Sachs performance curve is ahead of other banks

Let us apply this simple tool and insight to what has come to be called ‘talent management’.

It is a decade since ‘The war for talent’ was published by Harvard Business Press – regular readers of this blog book will know I hold HBP responsible for publishing some spectacularly vacuous management books – and history has proven the authors so wrong (McKinsey’s chose Enron as a paragon). From the very beginning the more astute management critics could see there was no substance behind McKinsey’s hype and even Wikipedia points out that the authors failed to define ‘talent’. Yet this has never deterred ‘talent managers’ from selling their non-evidence-based, indefinable wares with great enthusiasm.

In the age of evidence-based HR let us at least get back to some basics. Whether you can define talent or not we all know it is rare – that’s why we give it a name – and we also know you can only manage what you measure, even though the non-evidence-based refute this*. Ideally the measures will be objective (output, cost, revenue, quality) but a subjective measure is better than no measure at all.

One group we naturally associate with having innate talents are sportspeople; whether they be high jumpers, footballers or basketball stars. Their performance is measured every time they compete (height, goals, baskets) and we tend to read across to say how talented they are (e.g. the top goal scorer in the league must be one of the most talented) knowing that the context, and their team-mates, have a significant part to play in their success. We also recognise some people as being highly talented despite poor performance – they are ‘off form’. So there is no automatic correlation between talent and performance; we just have to keep working at making the best of whatever talents we have at our disposal.

It is an HR dictum that you should ‘hire for talent and attitude’ because we cannot endow people with attributes they do not possess. You cannot give someone a talent and trying to change a bad attitude is more trouble than it is worth but if we want to manage talent it is worth measuring it. Try it now by focusing on one particular talent – say negotiating. I picked this one because it encapsulates many of the common issues surrounding discussions about talent. You can teach negotiating skills (e.g. save your big guns for later, when you say ‘no’ mean no) and train in how to prepare for a negotiation (e.g. what information do you have about the other side, how high can you afford to go?) but we all know good negotiators who just have a passion for negotiating and a personality that enables them to get away with things that others cannot – they have a natural talent for it.

If you send managers on negotiation training it can become evidence-based (without perfect evidence) by giving each trainee a baseline score from 1 to 10 before they start (as explained here) but also consider a few people who are not automatically on the list – they might have a talent as yet untapped. For anyone getting a 3 or less I would recommend saving time and money by taking them off the list. Anyone between 4 and 7 will probably need some motivating to improve their negotiating ability (they don’t have a natural talent and/or it is something they do not welcome) and with the 8 and over’s it might be worth considering what successes they have actually had before sitting down with them to try and work out what makes them so good. If you are clever enough you might even help to make some of it rub off on the others.

As with everything in HR though it is never entirely straightforward is it? Talent is not a one-way street; it can bring with it all sorts of problems and unexpected consequences. Recognising talent can create primadonnas, prompt them to demand too much, encourage them to move elsewhere or just disrupt the team. Defining talent does not help much and calling it a ‘competence’ just confuses the issue.

The only thing that matters is that you at least have some measures that tell you two things – you believe you are getting better at ‘it’ and ‘it’ is improving your performance. Alternatively if you think talent cannot be measured, because it is ‘intangible’, don’t be surprised if it never produces anything tangible.

*Those who say you cannot measure ‘intangibles’ like talent and leadership seem to have missed the obvious point that they have already attached some relative importance to it – a subjective measure that says it is more important than the other things they could be doing.

All you need to know about training evaluation in about 700 words

Posted on June 28, 2011 by Paul Kearns

When asked by the editor for a title for my CIPD book on evaluation I suggested – ‘The Last Word on Evaluation’ – not out of arrogance but because the mountain of literature and endless debate around the subject had never reached a conclusion. Evaluation is essentially a simple subject made unnecessarily complicated by vested interests. Probably the biggest and most influential vested interest is the American Society for Training and Development (ASTD) who, whenever they tried to offer their members advice on this thorny subject, managed to get it so completely and so obviously wrong.

From the 1970’s the ASTD regarded Kirkpatrick’s 4-Levels as their standard:

Reaction to training (happy or smile sheets)
Testing learning
Applying learning in the workplace
Business impact

Let’s try this out by thinking of one salesperson going on a sales training course at a cost of $1000. When they return you take them through the 4 levels, starting with how they felt about the course… – WRONG. That way you end up at Level 4 with no way of assessing impact.

Let’s start again. Before you send them on the course you ask them how much they sell now ($1.2 million) and what the profit margin is (10%)? Then afterwards, when you get to Level 4, you can ask again how much they are selling and you have a basis for gauging impact. The critical, pre-learning questions form the BASELINE and produce the simplest, most obvious and effective evaluation model with just 2 levels:

Baseline evidence
Business impact measured against the baseline data

- but you will quickly find in practice that this 1st, Baseline level will usually suffice because it is this one that adds all the value in what is now an enhanced, learning process. Individual learning starts when each individual knows their own Baseline.

The 4-Levels never captured this so, just as corporations started asking what the financial return (ROI) was on their training investment, Kirkpatrick’s obsolescence was becoming apparent. So the ASTD decided to back a different horse in the 1990’s, but could not admit Kirkpatrick* was wrong, so adopted Jack Phillips’ model which just added another level – level 5 for the ROI calculation – which also made it look like an innovation – WRONG again.

Let’s go back to the beginning. You ask the sales trainee the BASELINE questions. You do the training and then at Level 4 you get an answer as to how much sales have increased (1%) and you know the cost ($1000) so you can do the net ROI calculation immediately – it’s 20%. There is no need for any level 5 – it doesn’t add anything and anyone with a calculator can work it out. Under Phillips though you have to spend even more time and money making the numbers up and converting to $ because he doesn’t establish the relevant $ sign at the beginning. Level 5 was always a figment of the ASTD’s collective imagination and that’s why the figures Phillips produced never convinced anyone who had a business head on their shoulders.

So the ASTD decided it needed more credibility and drafted in a labour economist – Laurie Bassi – in a vain attempt to garner a more academic and quasi-scientific level of respectability. Laurie tried to show the business impact from the billions of $’s ASTD members were spending on training. Laurie did not know any more about evaluation than her predecessors though and, as an academic, used the only analytical tools she had, regression analysis to produce correlations, using retrospective data. I guess she also did not have the benefit of first-hand experience of what it feels like to work in a training department in a large corporation; where trainers are often under pressure from managers to produce all sorts of stupid, knee-jerk programmes to cover up their deficiencies in people management and development. Laurie is now heading up efforts to establish international standards by the other large American professional body, SHRM ( the Society for HRM) and the ASTD still makes it mandatory for anyone wanting to be their ‘partner’ to attend a Jack Phillips training programme.

Forty years should have been long enough for the Americans to get it right but so far, despite being an SBO (statement of the bleeding obvious), no ‘expert’ in the US has ever fully understood or acknowledged the crucial importance of the BASELINE level in evaluation and learning; whether it be sales training, management development, leadership or OD. Starting with a Baseline makes learning evidence-based, it is a perfect application of evidence-based management. Now, if you are looking for an indication of which corporations waste the most money on ‘training’ just ask them which model they use and where they start from.

For personal development linked to this topic visit the Consummate Professional Series or attend a workshop.

* The Kirkpatricks still refer to their model as ‘the 4-levels’, although son Jim now tries to make up for the absence of ROI with something called ROE (return on expectations). This is intended as a sort of ROI-lite, except ‘expectations’ are not necessarily couched in $ terms and no ‘return’ can actually be calculated because ROE is not a mathematical or financial formula. In short, ROE can mean whatever you want it to mean.

Evidence-Based HR managers know better than to rely on correlations

Posted on May 10, 2011 by Paul Kearns

Finding the causes of complex problems. is the single, most skilful part of the evidence-based manager’s job. Asking ‘Why?’ is also a prime motivator of human behaviour. Many of man’s greatest endeavours are driven by this most primal need – why are we here, what has caused me to feel this way? If we fail to identify the right causes, using evidence-based analysis, we fail to find the right solutions. When organisational demands for activity – ‘do anything but do something!’ – take precedence over intelligent thought and proper analysis then we are bound to see short cuts being taken, however erroneous.

Academia is not put under the same immediate, operational duress so we expect academics to go to the trouble of providing better evidence with more scientifically rigorous methods. The LSE (London School of Economics and Political Science) even takes its Latin motto ‘Rerum cognoscere causas’ – ‘to understand the causes of things’ – from Virgil’s dictum “happy is he who can discover the causes of things”.

Indeed, great happiness will stem from knowing the causes of what ails us and one LSE academic in particular has followed this line of enquiry – Professor Richard Layard – who is well known for his work on happiness and wellbeing. These are both matters of great interest to those who have devoted their entire careers to the search for the causes of employee happiness and fulfilment at work. So has Professor Layard followed the LSE’s creed in this endeavour?

Anyone dealing with people issues at work will be reminded, every day, that HR management is not a ‘science’ in any meaningful sense. It cannot be tested in laboratory conditions and neither does it respond well to conventional, statistical analysis. I am referring of course to correlations and regression analysis. This is an arcane, statistical technique used by social scientists to pretend that they have identified the causes of complex human problems when, in fact, they have just resorted to stacking one correlation on top of another – which is not the same thing at all. If this is all a bit too academic for you let me make my point much simpler.

Every HR department I know wants to believe that employee engagement causes organisational performance. This notion appears obvious and very valid until you consider that the buyer of your office furniture could equally make the same claim about their role. There is probably as much of a correlation between office furniture costs and profit as there is between engagement and profit but no one in their right mind (apart from the procurement department) would suggest that this denotes a significant, causal relationship with performance. Evidence-based managers do not rely on correlations that can so easily make them look stupid.

This logic does not stop highly respected academics passing off correlations as causation though. Take Professor Layard’s own paper – ‘Good Jobs and Bad Jobs’ where he tells us in “Annex A: Evidence on Happiness” that

“Happiness research has confirmed that happiness is a single dimension of all experience, measurable by psychometric or neurological measures (both highly correlated).”

They may well be highly correlated but that will never reveal what causes what. Or has the LSE dropped its motto now and replaced it with a much lower standard of evidence – “to find correlations between things”. Would they be just as happy to pass off a bottle of cheap Cava as Dom Perignon at receptions for their generous alumni? Probably, if they thought no one would notice.

Surreptitiously and knowingly substituting correlation for causation is a serious crime and extremely dangerous. There is a correlation between skin colour and prison population in the US and no doubt this is irresponsibly seized upon, by those with their own agendas, to infer that the colour of someone’s skin can actually cause crime.

This is the prime reason why evidence-based HR is now so high on the organisational agenda – because the development of HR management thinking over the last 30 years has been predicated on spurious correlations (e.g. happy employees are productive employees, diversity targets improve diversity) rather than causation.

There can be no better example of this than the “Best companies to work for” (*but see Update below) schemes. In the UK the image shown above was printed in The Sunday Times supplement announcing the ‘winners’ of this award in 2005. The graphs purport to show a causal connection between being a ‘best company’ and superior performance, when compared to the FTSE 100. Except that the small print had to admit that -

“in the last year … the FTSE 100 moved ahead with a 14.3% return against an 8.3% return for Best Companies”.

So not only had they failed to support their causal claim with the evidence on show, the correlation they used had been broken and they continued to refer to those with the worst performance as the ‘Best companies’. One could infer from the ‘Best Companies’ own data that perhaps it should be called the ‘Most Stupid Companies’ awards?

This is why lies and ‘damn lies’ often masquerade as meaningful statistics. Coming up with a solution first and then concocting the ‘evidence’ (sic) later to justify it will never be able to compete with the professionalism of the evidence-based. There will only ever be one winner in this competition.

For personal development linked to this topic visit the Consummate Professional Series Module 2 and Module 4

*Update 17th April 2012

This similar chart was taken from the i4cp site. Even if these graphs are true it is highly unlikely that they have anything to do with being (or not being) an i4cp member. Trying to mislead innumerate HR people is a sure sign of a non-evidence-based approach – interestingly Professor John Boudreau is on their Board of Directors.

Evaluation of learning – 3 simple rules of the game

Posted on April 20, 2011 by Paul Kearns

When is a profession not a profession? When it fails to follow its own professional advice. Imagine a surgeon saying – ‘Yes, I know the operating theatre should be a sterile environment but hey, what are a few bacteria among friends?’ Well that is exactly the sort of attitude adopted by the vast majority of training ‘professionals’ towards the subject of evaluation. They know their own professional standards demand they evaluate – as a very specific requirement in the learning needs, design, delivery, and evaluation cycle – but studiously avoid doing so.

So let’s try re-phrasing this. If we said that trainers do not collect any evidence that their methods work we might expect them to be operating in very ‘dirty departments’. Of course, if you’re teaching a bricklayer to lay bricks this does not matter too much because you at least have the evidence that the wall is still standing (or not). However, as soon as you move away from physical evidence (e.g. soft skills, talent development, diversity, leadership) who knows what ‘nasty bacteria’ the trainer might be cultivating in this breeding ground?

So why do trainers still resist evaluation so fiercely? If they don’t like it why don’t they just change the syllabus? Well because they can’t. Evaluation is not an optional extra, it is a fundamental element in learning theory – it is the feedback loop that reinforces the right behaviour. Think of the consequences of not doing evaluation – surgeons operating on patients without checking survival rates; rail companies not checking whether train drivers obey red signals; handing over your car keys to someone without a driving licence. All of these have happened in real life and it is only when people die that we decide we have to act to plug the gaps. So how can we continue to call some people ‘managers’ or, worse still, ‘leaders’ without any feedback that reassures us they know what they are doing? If we don’t the potential consequences will be just as disastrous.

In my experience there are two broad reasons why trainers’ practices fail to live up to any proper, professional standards. First, many ‘trainers’ are not professionally trained; so either do not understand the important role of evaluation or just dismiss it as unnecessary. Second, even if they attend a ‘professional trainer’s course’ the people teaching them don’t know the rules of evaluation or how to apply them. So what is the answer?

Actually it is extremely simple, but not easy, because it demands that we set the same standards as the surgeon, with no compromise allowed. The standard for anyone purporting to be a professional, evidence-based trainer, can be expressed in terms of 3 sacrosanct rules.

Accept that all learning (and therefore training and development input) has to be based on a closed-loop feedback cycle. So if you are supposedly ‘developing managers’ you need to reach a point where you can say they have been developed effectively.
Agree what ‘developed effectively’ means, in advance (not after the event as most evaluation models suggest) with each of the trainees. This has to be defined in terms of an output related to the role for which the person is being developed. For example, an ‘effective negotiator’ needs to spell out what that means – higher margins, more contracts, more durable customer relationships?
Realise evaluation always means value – that’s a financial matter. If there is no anticipated financial benefit to the business then there is no justification for spending any money on pure ‘training’ activity (defined as an input with no expected output).

All of these rules, yes they are unequivocal rules, are broken on a regular basis in every organisation I have ever worked with. Yet when they are fully understood, and applied consistently, they become the founding principles of a true learning organisation. Unlike the surgeon though, this is not intended to create a sterile environment, quite the opposite, it will be the most fertile ground you could ever possibly imagine in which beneficial learning will flourish.

For personal development linked to this topic visit the Consummate Professional Series

Managers with ‘ODD’ attitudes do not get EBM

Posted on April 14, 2011 by Paul Kearns

One of the many aspects of human behaviour that I have never been able to fathom is the ‘polar shift’ – moving from one extreme position to another without any rational explanation. Anyone who has children will have witnessed it. You are just about to go out and you say to your child ‘put your other shoes on’ and it ends up with them moodily selecting the most inappropriate shoes or simply refusing to go out. There is no halfway house, no moderation. The worst cases are now classed as ‘oppositional defiance disorder’ (ODD).

When such behaviour continues into adolescence we put it down to teenage petulance – ‘be back by midnight’ turns into either sulking in their room or arriving back at four in the morning. Being told to do something makes us want to demonstrate that we are not a slave to authority and what better way to demonstrate our independence than to do exactly the opposite of what the authority wants? But I think there’s more to it than that and, when it persists into adulthood, the behaviour becomes even more extreme and, frankly, bizarre.

Tell someone in HR or learning that they have no evidence to support their favourite activity and they immediately move from a position of zero evidence to demanding the highest, most sophisticated level of evidence possible. Trainers, who do no evaluation at all, suddenly want to use control groups, randomised control trials or double blind experiments when a simple chat with a trainee would suffice (‘have you used what we taught you – were there any problems?’)

HR departments who never want to measure anything suddenly start producing masses of ‘HR metrics’ and because this mountain of data fails to answer the most obvious question (what value do you add?) they go and change the name to ‘human capital analytics’ – the child’s equivalent of choosing their flip flops to walk in the snow.

Of course the other polar response to the simple question – ‘how do you know your methods work?’ – is the asinine challenge – ‘well you prove to me that they don’t’. It is a pity but these ‘managers’ are a little too old now to be sent to their room. No evidence-based manager is going to waste their organisation’s precious time and money looking to prove anything. They know only too well that you cannot prove a negative (e.g. you cannot prove a god does not exist). All the EB professional hopes for is a positive and constructive response, not the recalcitrant attitude of the teenager who simply refuses to do what is in their own, best interests.

The un-balanced scorecard needs that human touch

Posted on March 8, 2011 by Paul Kearns

Anyone who entered the world of management within the last 20 years will have come across something called the ‘balanced scorecard’ and might even have felt its cold, damp hand on their shoulder. Those of us who have been in management for more than 20 years saw how cruelly exposed traditional approaches to financial management, auditing and performance management were when faced with the pressures of rapid globalisation. So we knew the foundations of management had to fundamentally change. The main problem with change being forced though is that you tend to fall back on what you know best rather than think differently, outside of the box. Business had become addicted to measurement and decided that the only way forward was even more measurement and this time around the balanced scorecard appeared to offer more intelligent measurement.

The history of scorecards actually goes back much further than you might realise (at least to the 1950’s) and early attempts included the notion of a ‘dashboard’ of measures; just as a pilot needs a whole array of instruments to fly a plane. The eventual popularisers of this new game were of course Kaplan and Norton (‘Putting the balanced scorecard to work’ Harvard Business Review, Sept-Oct 1993) but eventually even they realised that more measurement wasn’t the answer. Instead of re-thinking the design of their measurement machine though they compounded their crime by bolting on an ugly, ridiculously complex and totally unnecessary gadget called the “strategy alignment map” (the one thing you could not fault Kaplan and Norton for was their alchemic ability at turning goose eggs into gold).

So why is the balanced scorecard concept still so popular today? Obviously some of those who use it say it works – without realising that they are not actually using a balanced scorecard (* see below). My guess is that it allows managers to look intelligent while behaving simplistically; that’s a very clever trick to pull off. They keep believing that as long as they follow their own set of ‘balanced’ measures they must be doing something right. This is exactly the opposite of what it was designed for – to get managers to use their brains and think things through more holistically rather than mechanistically – to create long term value rather than short term profit and long term value requires a long term HR strategy; and appropriate measures to match.

We hear the phrase ‘human capital’ used very glibly these days (except in this series of course) as though we are all mature managers now and understand how to get the best out of this most problematic resource. Kaplan and Norton’s nod to the problem of people management revealed itself in their ‘innovation and growth’ box of measures. Yet Norton himself confessed, in a foreword to ‘The HR Scorecard’ (another Harvard ‘golden egg’ by Huselid, Becker and Ulrich, 2001) that it is always this ‘people measures’ box that companies find the most difficult -

“ … the worst grades are reserved for their understanding of strategies for developing human capital. There is little consensus, little creativity, and no real framework for thinking about the subject. Worse yet, we have seen little improvement in this over the past eight years.”

Of course he didn’t offer any solution during those 8 years to the very problem that he and Kaplan had set for their clients. If he took another look today he would find the situation has not improved and his endorsement of the ‘HR Scorecard’ proved to be yet another empty promise. So the performance management ‘experts’ (sic), who still promote the balanced scorecard, dashboards and prisms have yet to provide the right ticks in their own boxes. *This means that after nearly twenty years there has never actually been a balanced scorecard – only a three-quarters, unbalanced version.

The real irony here is that the ‘people box’ has to be the most important ingredient in all performance measurement and management systems, so human capital measurement and reporting is the only genuine innovation to have arrived on the scene in the last 50 years. But because HCM is about human beings, who don’t always want to be measured or made accountable, and would rather play the system than make it work – it will never succumb to a simplistic, tick box mentality.

So, in summary, the balanced scorecard -

Doesn’t balance – and doesn’t acknowledge that value management comes from the whole system working well, not just its component parts
Emphasises the need for innovation and growth (and therefore organisational learning) but doesn’t offer any way of capitalising on the latent, creative and innovative behaviour of human capital and
Tries to deconstruct what creates value but, in doing so, often destroys it (you try putting deconstructed mayonnaise on your burger and see how you like it)

But apart from that it’s a really great idea.