Psychological Testing: What CEOs Need to Know

On a late September morning in 1976, I took my driving test at Rahway State Prison in New Jersey. Yes, the same prison where the Scared Straight documentary was filmed less than two years later I achieved adolescent independence on the road course that sits inside the confines of a state prison.

However, unbeknownst to me, I was being evaluated psychologically. I had already passed the written exam and while the road test was being administered the instructor was not only evaluating my driving skills, he was also testing me psychologically. As I approached a stop sign just after the halfway point of the driving test, he started yelling and screaming giving me multiple instructions at once. I was conditioned to manage stress as a high school athlete so his manner didn’t alter my concentration. The instructor congratulated me when we arrived back at the DMV office. I had earned my freedom. Finally.

I look back at this experience as not only a right of passage, but it was also the first time someone had tried to gain information about me that they could not normally otherwise. How effectively I manage pressure was not assessed on the written exam. Only through the practice of driving a motor vehicle could the instructor evaluate me psychologically.

Psychological tests require people to perform a behavior to measure a personal attribute, trait, or characteristic to predict an outcome, which describes or helps to understand human behavior. Intelligence tests, personality tests, interest and vocational inventories, college entrance exams, classroom tests, structured interviews, motivation tests, and the aforementioned driving test are all psychological tests. 

Caveat Emptor

Psychological tests are essential in today’s unpredictable business environment. Productivity, leadership effectiveness, job success, job satisfaction, and determining who will benefit from specialized services are all outcomes of psychological testing. Since psychological testing has gone mainstream, CEOs are now required to know how to properly critique a test.

First, CEOs need to become familiar with a test. Title, author, test duration, ownership, purpose, what the test measures, and what it predicts are test familiarity components.

Second, CEOs should read the test manual to understand concepts such as validity, reliability, fairness, test construction, the composition of norm groups, and administration procedures.

Once those factors have been established, then a technical evaluation needs to be conducted. Who comprises the norm groups, what are the types of norms, what is the evidence of validity, reliability, and fairness and the standard error of measurement all need to be asked and answered. If the test fails to meet scientific standards then the test should not be used. Otherwise, decisions based on the information the test provides won’t be sound. It’s no different from a blood test that provides false positives and false negatives putting people’s lives at risk. Think Theranos.  

Whereas the blood-testing industry is highly regulated, psychological testing is not. CEOs are strongly encouraged to protect themselves and their companies by raising their awareness because psychological testing has become part of our business culture. It’s also become big business in the 21st century. 1995 sales totaled $7M then quickly rose to $263M in 1997. By 2002, sales reached $700M. Emergen Research (1) reports the global assessment services market was valued at $6.5B in 2019 and is forecasted to reach $11.5B by 2027 with 7.2% CAGR (compound annual growth rate).

The strongest demand is from K-12, higher education, corporate and government sectors for psychometric, aptitude and coding tests. North America accounts for 41% of the market share followed by Europe, Asia Pacific, Central and South America and MEA (Middle East and Africa). Psychological testing is relied upon more and more because it reduces time and costs of recruiting preventing organizations from hiring or rejecting candidates on the basis of intentional or unintentional biases.

Personalized programs like The Prosperity Trait Index© (PTi) combine standardized results to assess strengths and weaknesses to identify skill gaps, which are the primary reasons for underperformance. A blueprint for an educational program is then created to close skill gaps and enhance performance as a result. It’s the exact same intention as the Stanford-Binet IQ test where IQ is proven not to be a fixed trait.

What is a Good Psychological Test?

Not all tests are created equal even though they might measure the same psychological trait. Psychological testing can be traced back to 2200 BC in China. Intelligence tests emerged in both France and Britain in the late 1700’s and US began using them in the mid 1800’s. Personality tests originated in the US during World War I to identify who would manage combat stress. Vocational tests surfaced in the 1940’s to predict who would succeed in specific occupations. 

Mental measurement is a method of collecting information about people to help understand, predict and change behavior through the assignment of numbers according to specific rules. Psychometrics materialized as a field of scientific study over the past century where behavioral scientists, psychologists and educators examine the quantitative and technical aspects of mental measurement so good tests are distinguished from bad tests.

All good tests satisfy a range of criteria. If not, then those are considered bad tests. Good tests are considered scientific measurement when a behavior is expressed as a number. The end result of a good test is not only a derived score, but a verbal description of a person’s trait the test measures.

First, good tests include behavior samples that are obtained under standardized conditions. Good tests require people to respond to items that representatively sample the behavior to measure a trait or predict an outcome. The test is administered in the same way to all people and have rules for scoring.

Second, good tests are valid meaning they measure what they say they measure and predict what they intend to predict. Good tests are reliable meaning test scores remain stable over time. Good tests demonstrate face validity where items are understood in the same way. Good tests have an acceptable level of error. Good tests are fair where it’s proven no demographic classification is advantaged or disadvantaged. In employment settings, large differences in average scale scores across demographic groups can result in lower rates of selection of ethnic minorities, women, or older applicants, which is termed Adverse Impact. 

Third, good tests have been administered to large groups of people who are similar to the group the test has been designed showing standardization. Data, called norms, indicate the average performance of a group and the distribution of scores above and below this average. Good tests are objective where people respond to structured true/false statements, multiple choice or rating scales.  

Overall, good tests meet rigorous scientific standards through complex statistical analyses. Bad tests fail to meet these standards. Psychological tests that meet scientific standards are the most sophisticated tools in an assessment process.

The Value of Feedback

The effectiveness of psychological test feedback depends on the quality of the assessment that leads to it, meaning it needs to be a good test that satisfies scientific standards. Only then can a test provide accurate feedback. Psychologists have come a long way since the beginning of the 20th century when feedback produced mostly negative, but largely ignored effects on performance (5). Today however, research shows feedback improves performance on average. Still, many leaders rely on subjective and biased feedback where it decreases feedback credibility in over a third of cases (5). 

Feedback is a managerial practice whose effects widely impact organizational performance and commitment. Simply, feedback is information about performance and is most effective when it’s specific, clear, task-directed, immediate and targeted at changeable behaviors (3).  Accurate feedback from a psychological test presents the reasons why performance has met or not met goals and how performance can improve.

Therefore, people are more likely to adopt a self-regulated approach to learning and performance improvement when test feedback meets these criteria. Most importantly, people are more likely to be receptive and more focused on job-task details and future performance when test feedback conveys something about the person and their job performance. Effective test feedback is an educational tool that reduces the gap between intended goals and our actual behavior. In effect, accurate test feedback reduces uncertainty about our performance. 

Test feedback describes the quality and quantity of past performance while also setting the direction to change future performance. Therefore, test feedback is an important mechanism for adapting behavior towards attaining individual and organizational goals because it’s proven to improve intrinsic motivation, fairness, organizational commitment, cooperation, discretionary effort, efficacy, job satisfaction and organizational citizenship (5). 

Test feedback is easy and inexpensive to deliver, and improves job performance. In the landmark study conducted by Kluger and DiNisi (2), the driver of organizational behavior rests on feedback distinguishing between correct behavior and goals. Consider the following facts:

• Feedback directs people’s attention on three levels — themselves, the task and task details
• Correct feedback aligns with organizational goals
• Incorrect feedback can negatively affect organizational goals
• Feedback has a stronger impact on performance when combined with goal setting
• Employees who link their goals to organizational goals are 3.5 times more likely to be engaged in their work

Good to Great

Good tests transform into great tests when they provide more than a score and a description of behavior. Positive and negative feedback produces significant performance effects. Known as “Feedback Valence”, when applied appropriately work outcomes include motivation, learning, intention to improve skills through goal setting, satisfaction with the task and with performance, goals and expectations, and improved work performance (3). 

Positive feedback signals good performance leading to continued effort, effective behaviors and performance and an enhanced self-belief that one can accomplish their goals called Self-Efficacy. Negative feedback signals poor performance that motivates improvement and learning by pointing out weaknesses and gaps. However, constant negative feedback can lead to resistance where people downplay it or dismiss it altogether leading to diminished motivation, a downward revision of goals and expectations or even complete task avoidance. This can often lead to Omission Bias where one develops a tendency to favor inaction over action. Psychological inertia sets in where people people believe it’s better to wait and see rather than aggressively pursue goals. This is commonly known as being “Stuck.”

The Law of Effect (3) proves feedback enhances performance. Positive feedback creates a sense of satisfaction increasing the probabilities that a behavior will repeat. Negative feedback creates a sense of dissatisfaction increasing the probabilities that a behavior will not repeat. The Law of Effect led to Operant Conditioning where people are trained to repeat positive behaviors through consistent positive feedback and rewards. 

Our behavior is regulated by goals, which are organized in a hierarchy. The gap that exists between our goals and the feedback we receive drives our behavioral regulation. Test feedback — positive and negative — contributes to mastery and competence needed for intrinsic motivation known as Self-Determination Theory (3). When goal setting is accompanied with test feedback, both learning and job performance improve because more resources are allocated to the task at hand — not just for individuals, but also at the team level. 

Consistent with Social Learning Theory and Control Theory, research shows feedback has a main effect on performance. People who receive negative feedback perform at higher levels than people who receive only positive feedback (4). People who receive objective and credible negative feedback set higher goals and perform at higher levels than people who received less credible and subjective feedback. Therefore, dissatisfaction created by negative feedback is a primary driver of behavioral change. Leaders who respond to negative feedback with rigidity and denial expedites the deterioration of organizational performance exponentially. As a result, these leaders develop organizational Learned Helplessness where the workforce experiences collective powerlessness of their current performance and a lack of control of their future in the marketplace. 

Test feedback acts as a mirror — it’s a reflection of one’s behavior. Nobody’s perfect just as no one is completely flawed. Test feedback provides the greatest effects when a combination of positive and negative feedback is presented. As a result, people gain more confidence in their ability to control themselves and master performance through continuous improvement. 

Conclusion 

Strategy and talent only go so far. CEOs face a new world and psychometrics are a tool to leverage the opportunities that are hidden within the contours of potential growth. Psychological test results reveal the Unknown Unknowns creating a competitive advantage. Asking the right questions and knowing the right answers, distinguishing between a good test from a bad test, and placing a high value on test feedback construct a pathway to higher performance, talent optimization and consistent strategic execution. 

References

1. Emergen Research (2020). Assessment Services Market By Product Type, By Service Type, By Medium, By Sectors, Forecasts to 2027, Oct, 8, 2020 https://www.emergenresearch.com/amp/industry-report/assessment-services-market 
2. Kluger, A.N., & DeNisi, A. (1996). The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory. Psychological Bulletin, 119, (2), pp. 254-284.
3. Leheta, D. (2016) The Effects of Feedback Valence on Work Performance, Learning and Attitudes: Looking at Valence, Trend and Consistency Together. Dissertation, Georgia State University, 2016. https://scholarworks.gsu.edu/managerialsci_diss/29 
4. Podsakoff, P. & Farh, J.L. (1989). Effects of Feedback Sign and Credibility on Goal Setting and Task Performance. Organizational Behavior and Human Decision Processes, (44), (1), August 1989, Pages 45-67
5. Tagliabue, M., Sigurjonsdottir, S.S.,  & Sandaker, I. (2020) The Effects of Performance Feedback on Organizational Citizenship Behaviour: A Systematic Review and Meta-Analysis, European Journal of Work and Organizational Psychology, 29:6, 841-861.

Written by Stephen Long, Ph.D.

Have you read?

# Best CEOs In the World Of 2022.
# TOP Citizenship by Investment Programs, 2022.
# Top Residence by Investment Programs, 2022.
# Global Passport Ranking, 2022.
# The World’s Richest People (Top 100 Billionaires, 2022).
# Jamie Dimon: The World’s Most Powerful Banker.