9 signs you might be using the wrong personality test

Stamp that reads Test Failed says you're using the wrong personality test

Personality testing is a big part of the way organizations make hiring decisions — it has been for a some time now (it wasn’t popular before about 1980). With advances in technology there has been a great proliferation of personality assessments. They’re not all good. A personality test is much easier to generate than it is to validate. This quiz, below, can help you to know if you’re using the wrong personality test. (Have some fun with it.)

Directions: The following list of paired statements(questions) reflects things I occasionally hear when folks are evaluating personality tests. For each pair, one response is more problematic when it comes to evaluating personality tests. Reflecting on your current situation, which of the two statements would I be most likely to hear from you or others if I were a fly on the wall when you were getting the pitch from your vendor?

Quiz to raise the question "Am I using the right personality test?"

Response Key: For all odd numbered pairs the problematic statement is in column A, for even numbered items the more problematic one is in column B.

Some of the statements do require more assumption than others, don’t get too caught up in the scoring. These are my answers and rationale:

  1. “It sure worked for me” — Frequently a personality test is sold by having the decision maker complete the assessment. This isn’t a bad thing — I encourage users to complete a personality test for themselves. The potential problem is that this is frequently the primary (or sole) evaluation criterion for a decision maker. Vendors know this and some hawk an instrument that produces unrealistically favorable results. “It says I’m good, therefore it must be right.” As for column B, the 300 page manual, good ones are typically lengthy. It takes some pulp to present all the evidence supporting a properly constructed inventory.
  2. “A type’s a type” – The most popular personality assessment of all, the MBTI, presents results for an individual as one of 16 types. Scores, to the extent that they are reported, only reflect the likelihood that the respondent is a given type or style – not that they are more or less extraverted, for example. But research and common sense say that personality traits do vary in degree, someone can be “really neurotic.” Two individuals with the same type can be quite different behaviorally based on how much of a trait they possess. A very extraverted person is different from someone who is only slightly extraverted — same type, different people. (No, I don’t condone mocking or calling out anyone’s score, as it would appear I’m suggesting in column A, but with a good test such a statement is potentially valid.)
  3. “That’s a clever twist” – Few personality tests are fully transparent to the respondent – this helps control the issue of social desirability. But some go too far with “tricky” scoring or scales. This is a problem in two ways: 1) if the trick gets out (google that) the assessment loses its value, and 2) respondents don’t like being tricked. It’s better to be fairly obvious with an item than to deal with {very} frustrated respondents who may just take you to court.
  4. “It was built using retina imaging” – Here’s another statement that needs a little help to see what’s going on (no pun intended). I’m not against new technology, it’s driving ever better assessment. But sometimes the technology is misused or inadequately supported with research. There’s a reason that some personality assessments have been around for more than 50 years. Validity isn’t always sexy.
  5. “That’s what I heard in a TED talk” — My intent here was to implicate “faddish” assessments. They may say they’re measuring the hot topic of the day, but more often than not, what’s hot in personality assessment, at least as far as traits are concerned, is not new. Research has concluded that many traits are not meaningfully different from ones that have been around a while. Don’t fall for an assessment just because you like the vocabulary, check the manual to see if it’s legitimately derived. There’s a reason that scientists prefer instruments based on the Big 5 traits (not the big 50).
  6. “Now that’s what I call an algorithm” — More complicated isn’t necessarily better. Some very good — typically public domain — assessments can be scored by hand. Tests that use Item Response Theory (IRT) for scoring, do have more complicated algorithms than tests scored via Classical Test Theory (i.e., more like your 3rd grade teacher scored your spelling test). Still, a three parameter IRT scoring method isn’t necessarily better than a one parameter model and it isn’t three times more complicated anyway. Proprietary assessments typically protect their copyright with nontransparent scoring, but for the most part what’s obfuscated or obscure is what items go into a calculation, not that the calculation is necessarily complex. Good assessments should employ fairly straightforward scoring to render both raw scores and percentile, or normed scores.
  7. “It really has big correlations” — As with some prior items a bit more context is needed to get the point I’m trying to make. Here the issue is sufficiency. Yes, a good instrument will show some relatively high correlations, but they need to be the right correlations. (And they need to be truthful. Unfortunately, I know of cases where misleading statistics have been presented. It helps to know about research design and to have a realistic expectation for that validity correlation. If the vendor tells you that their assessment correlates with performance above .40, make them prove it. (And a .40 correlation equates to a 16% reduction in uncertainty, not a 40% reduction. Sometimes vendors get this confused.)
  8. “It’s too long, let’s cut some items” – It’s tempting to simply eliminate irrelevant scales or items for your specific need. After all, you’re not touching the items that comprise the traits you want to know. The problem is that the assessment is validated “as is.” Both the length of an assessment and its contents can influence scores. Priming biases are one example of how items interact with each other. Anytime you modify an assessment it needs to be validated. This is typically the case for short forms of assessments (i.e., they’ve been specifically validated), so it’s fair to ask about this alternate form.
  9. “That’s amazing” — By now you should see that a common factor in my problem statements has to do with how much goes on “out of view” (less is better) and how thorough the test manual is. “That’s amazing” is for magic shows, not science (I realize I’m parsing semantics here – you get my point).

A personality test can be — and most often, is — a legitimate assessment for many (most) jobs. (This even applies to machines. Researchers are using a variation of personality inventories to manipulate the perceived personality of robots.) Without exception, it’s critical to ensure that any assessment be validated for specific use, but you want to start with something that has been thoroughly researched. If everything has been done right, you can expect local results to be in line with the manual (assuming your tested population isn’t that different from the test manual sample(s)).

A lot goes into validating a personality test and test manuals are lengthy. Although this is good and necessary for adequately evaluating the test, it can be used in intimidating or misleading ways. It’s easy for claims to be made out of context even if the manual is true, especially when decisions are made that affect one’s job. It’s important to review that test manual, not just the marketing brochure. (The good news is these manuals are boringly redundant. For example, the same figure is used for each scale, or trait, when repeating testing for gender bias.) Although I’m sure your vendor is a “stand up” person, you can’t rely on this fact if your process gets challenged in court. It pays to review the manual thoroughly.

I hope your personality inventory passed the test.

Psychways is owned and produced by Talentlift, LLC.

How to tell if someone should NOT be your coach

Unhappy male coachee listening to an executive coach give advice. Frustrated client holding a hand to his face.

Coaching has become very popular as organizations face an increasing need for individuals (or groups) to learn and grow more substantially and quickly than ever. Based on favorable if scarce evidence supporting the effectiveness of coaching not presented here, coaching should be given serious consideration as a key component of any organization’s talent management strategy. But, as is the case with anything powerful, you have to be careful with coaching engagements, or you can get hurt – badly. Because there is no "one best way" to coaching, there’s considerable risk of engaging the wrong person as a coach. And I'll be the first to say you can’t judge a coach by their title (or solely based on published credentials). The best of well-intended sponsors/buyers/users of coaching services are at risk of making a mistake before the coach even gets started. Here, I share some of the key ways for you to know when someone should NOT be your coach.

  1. They overlook ethical matters. This may seem obvious, but it’s much more complicated -- and risky -- than most think. Ethical issues abound in any situation where personal assessments are made, but they’re especially prevalent in coaching. Key questions must be addressed: What’s in-bounds/out-of-bounds? How will data be collected and shared? What happens next?

Coaching is extraordinarily powerful with rightfully high expectations since it isn’t cheap. There’s a lot of pressure that can lead to shortcuts or, kindly put, bad judgment. But individual and organizational “lives” are at stake. You need to engage someone well-versed in the substantial ethical issues that are a part of all coaching engagements – however formal.

  1. They get the role wrong. An executive coach is NOT;

a. Colleague – Your coach (by “your coach,” I mean the coach you engage) may be someone from within your organization, but they shouldn’t work directly with the coachee. It’s hard for a coachee to confide in someone that’s already close enough to have preconceptions or may affect the coachee’s fate “back on the job.” Using a colleague as coach has its place, but I advise against it for the majority of situations. A number of bad outcomes can occur for both coach and coachee when the two already work together.

b. Vendor – Assessment tools, for example, are a big part of coaching. Your coach shouldn’t be indefensibly partial about what assessments or other “coachware” they use. There’s more than one personality test, believe me. Check twice if the coach markets their tools exclusively. You want someone who knows a range of tools and how they work (i.e., their psychometrics).

c. TrainerCorporate trainers are specifically skilled at building and transferring new skills. While this may be a part of what an effective coach does, it’s rarely the “main course.” The coaching context simply isn’t right for a blog-standard training approach. For example, coaches build closer (i.e., deeper trusting) relationships based on coachee-centered discourse and need. (But note mistake 2d, immediately below)

d. Friend – Your coach isn’t meant to be or become the coachee’s friend, but this is a significant risk in with considerable “gray area.” A coach is expected to be friendly (i.e., “nice”), in fact this is key to building a trusting relationship, but boundaries MUST BE established and maintained between the two. Beyond the professional conflict that can arise when coach and coachee become close friends, the potential exists for much more serious ethical conflicts with life-changing psychological and legal consequences. Because of the highly sensitive interpersonal dynamic that emerges when self-disclosure to an authority figure is involved, the stage is set for transference. This is a catastrophic – about as bad as it gets. Any inappropriate relations must be managed swiftly and surely. If there’s any suspicion that an inappropriate relationship is forming, you should end the coaching engagement. Immediately! Full stop. This is no time to be bashful. {Respecting all caution, it is okay, even expected, for your coach to use friendly behavior. But they must not cross ethical boundaries that are much more critical for a coach than they are for others, e.g., a colleague (but they are important here, too.}

e. Boss – Again, bosses (possibly, you) aren’t excluded from coaching, they’re actually expected to coach. But bosses need to “stay in their lane.” In all likelihood the boss has had a role in the calling for the need of a coach in the first place. Boss’ never get the same story that an outsider can. I hope the reasons are obvious.

f. Short-order cook – Your coach shouldn’t be overly concerned with accommodating the coachee’s every need. Here again, the temptation of the coach is to give what is asked for, after all, that’s service, right? Wrong. What a coachee wants isn’t always what they need, in fact it’s exceptional when it is. A good coach knows when to accommodate, when to resist and when to suggest otherwise. They must maintain control of the relationship.

g. Subject matter expert (SME) – A coach is not an expert in the specific, technical/functional aspects of the coachee’s current or future job. That’s what a mentor is, and the skillsets are very different. Mentors impart organizational wisdom and job-related instruction. Coaches work with the coachee to evoke more general insights and lay plans for action and follow-up.

h. Messenger – This is a BIG one. (and shouldn't be 'h') It’s imperative that the coach not only be able to make an accurate assessment, they must be permitted to do so. Using a coach to provide feedback is cowardly and ineffective. On a personally relevant note, coaching as a profession and trade is tainted by this unsavory tactic. Don’t do this. At minimum, it will destroy trust.

  1. They didn’t adequately address objectivity.

A common challenge when engaging a coach is insuring objectivity. A good coach can’t be influenced by demands, information or circumstances. Being objective isn’t necessarily about having unrelated, or no prior involvement with the coaching party (i.e., coachee, sponsor, others involved) – it’s about being able to set aside circumstantial information when necessary for the good of a professional engagement. It’s being a trusted expert.

Beyond being objective, the coach must be perceived as objective. As I’ve mentioned, it isn’t impossible, but it is rare, to find a coach- and role-appropriate level of objectivity when considering internal coaches. But internal coaches frequently have too much history with the coachee for to be adequately insulated from the organization’s dynamics. The type of relationship an internal coach builds is almost always different from that of an external coach; the reason is objectivity.

When I have worked as an internal consultant, I’ve never taken on the role of executive coach – with one exception. That was a job in which I practically worked as an outside consultant and had no exposure to the coachee. The likelihood of a conflict of interest jeopardizing objectivity is especially risky for folks in HR. You can’t expect a coachee to share their deepest work-related concerns when they know, or think, that their “confidante” is about to run off and determine their pay. Even though conflicts are a risk for external coaches, handling deeply sensitive and personal discussions is one of the main tasks of the coach and vital to building trust.

{Oftentimes a coach does have a say in the coachee’s fate. In these cases, it’s imperative to respect boundaries and to have a comprehensive coaching agreement in place. The coachee needs to know and formally agree what they’re getting into.}

  1. They didn’t engage an expert in coaching. Coaching requires a plethora of specialized skills. SOME of these include:

This is just a partial list and doctoral degrees are conferred for each of these requirements. Every coaching engagement is unique and requires the coach to adapt in ways that optimize the engagement. True experts are more than a “one horse show” and can adjust seamlessly and effectively.

All of these mistakes need to be avoided in order for a professional and trusted coach-coachee engagement to exist. Without any one of these, it may be more than a professional lapse of judgment - you could have a real crisis on your hands.

...Thought you should know…

Psychways is owned and produced by Talentlift, LLC.

The top 5 reasons succession planning goes wrong and how to fix them

Succession planning org chart with person icons

Succession planning may be – no – it IS the most important job of executive leadership. The critical aim of this work is to ensure leadership continuity by identifying individuals with the highest potential to fill key positions in an organization. This is work that affects more than just the future of individuals’ careers, it affects the fate of the entire organization. I have literally seen a company’s stock price swing more than 10% in a day when news about executive position replacements gets out. Even in moderately large organizations billions of dollars can be at stake when it comes to answering the question, who will lead? As such, succession planning represents possibly the highest stakes of all executive assessment. Unfortunately, most organizations are really bad at succession planning. And more often than not, those stock prices swing lower rather than higher based on news of new leadership. Maybe the investors are right.

Succession planning is typically construed as good defense. In order to ensure leadership continuity, a list of individuals most ready to backfill a given job is prepared so that in the event of an open position (typically unanticipated) a succession of leadership changes can be made. Backfills are made not just for the open position but for the “domino effect” that cascades through the organization based on even one or two key moves. While this may be a good replacement plan for key executives, it’s bad for true, strategic organizational succession planning. It’s like looking in the rearview mirror in order to go forward – you might just run over someone and you won’t get where you want to go.

Let’s examine some of the most challenging realities that plague most succession planning efforts.

Succession Planning - Done Wrong

  1. It’s based on backwards thinking.

The typical exercise involves identifying the next in line, i.e., "backfill," for a job that opens up, usually due to an executive departure from the organization. While this may be a good way to stay where you are as an organization, your competition is going forward at full speed. The error here is replicating what you’ve had versus positioning what you’ll need.

  1. It’s driven by those who need a successor.

This problem applies more broadly than succession planning. From a personal point of view, the assumption here is that if I win the lottery, then my groomed successor will replace me. Wrong. If you leave the organization, you most definitely won’t be the one making key executive moves – you’re not even around. The most likely person to make any backfill is the person to whom that position needing a backfill reports, not the one in the position. For this reason, it’s imperative that executives know not just their direct reports, they need to know the employee population at least two levels beneath them.

Guess what? I have facilitated numerous succession planning efforts where executives have no idea who reports to their direct reports. Photos don’t even jar their memory (and can be controversial in this context). “You rode up on the elevator with them.” Still don’t know them.

  1. It’s based on the strongest of psychological biases.

Too many positions are filled based on the “like me” method. Naturally, we’re wired to think that we are exactly what “my” position needs, therefore I am looking for a “mini-me.” Well, you may think you’re at the center of the universe (face it, we all do), but if you ask others, you’ll get a very different point of view. Others in the organization may not want your backfill to be a mini you. That’s a good perspective to cultivate but it’s almost impossible when you’re in the room. This is why politics play such a strong role in most succession planning.

  1. It’s personal, not organizational.

This is another bias that inserts itself in the succession planning process. Leaders are VERY sensitive about “their people.” In fact, a leader oftentimes acts as though “their people” are just like family members – and sometimes THEY ARE, but this is a whole other concern not to be addressed here. Regardless, they aren’t “you’re people,” they’re the organization’s people.

  1. It’s based on flawed judgement.

Even for the few occasions that I have someone tell me they’re a poor judge of people, guess who weighs in on talent to fill open positions? Yep, everyone has a point of view when it comes to selection. And the closer that selection is to the individual, the stronger their judgement gets.

Studies consistently find human judgement to be a bad predictor of actual talent. If only those who are right when they admit that they’re a poor judge of talent actually deferred to more objective, scientific means of assessment. But they don’t. Sometimes the best you can do is to present decision makers with well-designed psychometric instruments that do make accurate assessments and hope that reasoned, versus inferred judgement prevails. This works best when the judge knows a bit about how the given psychometric tools work. In many cases, science will make an impact. You’ve got to take the magic out of the assessment and encourage those who “lean in” to a better way.

Succession Planning - Done Right

  1. Think of succession planning as progression planning.

Instead of priming defensive and myopic mindsets with terms like “succession,” “my successor” and “backfill” use terms like “progression,” “strategic,” “organization,” and “future fill.” This can even help with the personal biases as you and history are intrinsically bound. (See #s 2, 3, 4 and 5) Good succession planning isn’t possible without good strategic planning. Your talent for the future should look like what you need in the future, not what you’ve had in the past.

  1. Have leaders discuss talent at least two levels below them.

The first time you do this you may find yourself in a circular loop, “we can’t talk about the talent because we don’t know this talent” meets, “we don’t know this talent because we’ve never talked about this talent.”

That’s actually a good start. When leaders admit they need deeper insight you have the opportunity to improve on those shallow evaluations. Ignorance can be your saving grace! I’d much rather work with a leader that “doesn’t know everything” and is right about that than one who’s confident in their wrongful thinking. Now’s a good time to introduce better assessments and more strategic thinking.

  1. Train leaders in good assessment and talent management.

This is a big deal. You have to take the “like me” person out of assessment. Otherwise you have the old cliché, “when you’re a hammer, the world looks like a nail.” And since diversity and inclusion are nowhere near where they need to be in organizations – especially at the senior most levels, you need the seasoned group of executives to really recognize and know talent that isn’t at all like them. But good, accurate, assessment is hard and typically counter intuitive. Still, it’s not impossible to have a leader acknowledge that their best replacement won’t look like them.

  1. Ensure leaders discuss not only “their” function, make them responsible for all of the organization's functions.

Leaders think in their silos and don’t want others messing with their kingdom. That’s all wrong. You need to open up and break personal “myndsets” and create organizational mindsets. After all, these are individuals entrusted with the future of the organization – not just one function or group. By getting leaders to talk about talent in other groups you also improve the likelihood of cross-functional moves. These are critical to effective succession planning as they work to create organization leaders versus expert leaders. Well-rounded talent knows more than accounting.

  1. Use properly validated assessments.

Study after study show that good psychometrics beat good assessors. While there are exceptions, you aren’t one of them. Moreover, research finds that “good assessors” primarily are good at assessing specific characteristics or traits – but not all. A comprehensive set of psychological assessments used by an expert in workplace psychology should be mandatory for proper succession planning. Furthermore, studies show that training assessors with the framing reference of properly validated psychometrics actually improves their personal evaluations.

Good succession planning shouldn’t be a blind date. Open leadership’s eyes to the talents of new, unknown talent and give them the tools to truly know that talent. Only by clarifying what’s needed in the future for the organization can you break some of psychology’s strongest biases to truly ensure organization continuity AND progress.

Psychways is owned and produced by Talentlift, LLC.