One of the "truisms" floating around is that "You get what you measure." Belief in this assertion has led many to a metrics-based style of management, but the results have been uneven at best. Why?
any believe that "You get what you measure." The belief persists, in part, because of anecdotal evidence; because some experiments do appear to be consistent with the assertion; and because so many of us believe that the rest of us believe it.
The Mars Climate Orbiter, as it would have appeared if it had not been lost upon entry into Mars orbit. Mars Climate Orbiter was lost because of a mismatch between Metric and English units in a file used by its trajectory models. The measurements were all objective, and they were even precise, but confusion led to a loss of the orbiter. Photo courtesy US National Aeronautics and Space Administration.
Still, there are reasons to question its validity. What follows is a catalog of possible explanations for variances between the promise and the reality of metrics-based management. In this Part I, we examine three assumptions underlying the measurement process. See Part II for more.
We assume that indirect measurements work
Sometimes we try to measure attributes that aren't directly measurable. For instance, when we try to measure immeasurables like loyalty or initiative, we actually measure something else that we assume is highly correlated with what we're trying to measure. We usually do this using interviews or surveys.
But too often, rigorous proof of the assumed correlation is unavailable. Sometimes, we comfort ourselves, saying, "it's so obvious," but this is risky — the history of management, psychology and science is replete with assumptions that, though obvious, were nonetheless false.
We assume that all attributes are measurable
We assume (wrongly) that all organizational attributes are measurable
The word "measurement" evokes our experiences determining physical attributes like length, weight, or temperature. This leads us to assume that whatever we want to know can be determined by a suitable measurement, but that assumption can lead to trouble. Consider something as important as progress. Suppose a team has been working for three weeks, when, suddenly, someone realizes that their entire approach will never work. This is certainly progress — they've learned something important. But it probably won't register as progress in the organization's metrics. Most likely, it will be reported as a setback.
Measuring non-physical attributes, such as the advance of knowledge, is often possible when changes are incremental. But at times, our metrics fail, and they tend to fail precisely when we most want to know where we stand.
We assume that objectivity implies precision
Sometimes we use measures that are objective but imprecise. That is, we assume wrongly that multiple identical measurements would yield nearly identical results. For instance, when we measure attitudes using a survey "instrument," we assume that the results we obtain are relatively context-independent. We don't actually know that the results are independent of, say, the time of the month, or the price of the company's shares — we just assume it.
Rarely do we test these assumptions. Indeed, we often assume that these factors do affect the results. We know this because we sometimes observe organizations gaming the measurement for favorable results, or trying to influence the results by releasing favorable news.
Assumptions like these can account for some — not all — of the poor performance of metrics-based management. In a future issue, we'll examine the effects of employee behavior. TopNext Issue
Are your projects always (or almost always) late and over budget? Are your project teams plagued by turnover, burnout, and high defect rates? Turn your culture around. Read 52 Tips for Leaders of Project-Oriented Organizations, filled with tips and techniques for organizational leaders. Order Now!
Elise slowly walked back to her office, beaten. Her supervisor, Alton, had just given Elise her performance review — her third consecutive "meets expectations." No point now to her strategy of giving 120% to turn it all around. She is living a part of the Pygmalion Effect, and she's about to experience the Pygmalion Side Effects.
If your job responsibilities sometimes require that you tell powerful people that they must do something differently, you could find yourself in danger from time to time. You can learn a lot from tugboat captains.
When an important item remains on our To-Do list for a long time, it's possible that we've found ways to avoid facing it. Some of the ways we do this are so clever that we may be unaware of them. Here's a collection of techniques we use to avoid engaging difficult problems.
When a group decides to take an action that nobody agrees with, but which no one is willing to question, we say that they're taking a trip to Abilene. Here are some tips for noticing and preventing trips to Abilene.
Nearly everyone complains that email is a time waster. Yet much of the problem results from our own actions. Here's Part III of a little catalog of things we do that help waste our time.
I offer email and telephone coaching at both corporate and individual rates.
Contact me for details at rbrenner@ChacoCanyon.com
or (617) 491-6289, or toll-free in the continental US at (866) 378-5470.
Get the e-book!
Past issues of Point Lookout are available in four e-books:
Are you a writer, editor or publisher on deadline?
Are you looking for an article that will get people talking and get compliments flying your way? You can have 500 words in your inbox in one hour. License any article from this Web site. More info
Public seminars
Managing Virtual Teams for Real Results
Managing global or dispersed teams is challenging — miscommunications, misunderstandings, and interpersonal conflict all thrive in the typical environment of the distributed team. And they're even more common in global teams, because of time-zone offsets and language and cultural differences. We'll inventory the challenges distributed and global teams face, and provide tools for anticipating and addressing them. The focus of this program is practical — attendees will learn concrete techniques for preventing and dealing with the problems that accompany global and distributed teams. Read more about this program. Here are some upcoming dates for this program:
Organizational Politics for People Who Hate Politics
Have you ever felt powerless to implement an important new idea? Have you ever been "blind-sided" at a meeting? Have you ever lost two good employees because you could find no way to keep them from attacking each other? These are some of the issues of organizational politics. Many of us have become enmeshed in them from time to time, but we've also known some people who seem to be able to engage and prosper. How is that done? We'll inventory the challenges of organizational politics, and provide tools for anticipating and addressing them. The focus of this program is practical — attendees learn concrete techniques for dealing with the problems that arise in workplace politics, while keeping their integrity intact. Read more about this program. Here are some upcoming dates for this program:
The Politics of Meetings for People Who Hate Politics
There's a lot more to running an effective meeting than having the right room, the right equipment, and the right people. With meetings, the whole really is more than the sum of its parts. How the parts interact with each other and with external elements is as important as the parts themselves. And those interactions are the essence of politics for meetings. This program explores techniques for leading meetings that are based on understanding political interactions, and using that knowledge effectively to meet organizational goals. Read more about this program. Here's an upcoming date for this program:
Person-to-Person Communication for Project Managers
When we talk, listen, send or read emails, read or write memos, or when we leave or listen to voice mail messages, we're communicating person-to-person. And whenever we communicate person-to-person, we risk being misunderstood, offending others, feeling hurt, and being confused. There are so many ways for things to go wrong that we could never learn how to fix all the problems. A more effective approach avoids problems altogether, or at least minimizes their occurrence. In this very interactive program you'll learn a model of inter-personal communications that can help you stay out of the ditch. In those moments of intense involvement, when we're most likely to slip, you'll have a new tool to use to keep things constructive. Read more about this program. Here are some upcoming dates for this program:
Ever wonder if there isn't a better way to travel? Travel is essential, but the hassles of travel aren't. Read 202 Tips for Business Travel to learn how to convert business travel from a time-wasting hassle to a breeze. Revised and updated for 2008 with 101 new tips! Check it out!
Projects never go quite as planned. We expect that, but we don't expect disaster. How can we get better at spotting disaster when there's still time to prevent it? How to Spot a Troubled Project Before the Trouble Starts is filled with tips for executives, senior managers, managers of project managers, and sponsors of projects in project-oriented organizations. Check it out!
Are you doing work you love? Are you less in love with the job? Bad boss, long commute, troubling ethical questions, hateful colleague? Read Go For It! Sometimes It's Easier If You Run to learn what we can do when we love the work but not the job. It helps you get moving again!
The key to managing virtual or global teams is creating a sense of team despite the obstacles of separation. Read my tips booklet, 303 Tips for Virtual and Global Teams, to learn how to make your virtual global team sing. Newly revised and updated for 2008 with 101 new tips! Check it out!
Are you managing a change effort that faces rampant cynicism, passive non-cooperation, or maybe even outright revolt? Read 101 Tips for Managing Change to learn how to plan and execute your change efforts to inspire real, passionate support. Check it out!
A Tip a Day arrives by email each business day. It's 20 to 30 words at most, and gives you a new perspective on the hassles and rewards of work life. Most tips also contain links to related articles. Free!
Do you ever wonder if all these meetings are really necessary? (They aren't) Or whether there isn't some better way to get this work done? (There is) Read 101 Tips for Effective Meetings to learn how to make meetings more productive — and more rare. Check it out!
Exchange your "personal trade secrets" — the tips, tricks and techniques that make you an ace — with other aces, anonymously. Visit the Library of Personal Trade Secrets.
any believe that "You get what you measure." The belief persists, in part, because of anecdotal evidence; because some experiments do appear to be consistent with the assertion; and because so many of us believe that the rest of us believe it.
Top 
Next IssueAre your projects always (or almost always) late and over budget? Are your project teams plagued by turnover, burnout, and high defect rates? Turn your culture around. Read 52 Tips for Leaders of Project-Oriented Organizations, filled with tips and techniques for organizational leaders. Order Now!
Pygmalion Side Effects: Bowling a Strike
Become a Tugboat Captain
How We Avoid Making Decisions
Trips to Abilene
Email Antics: Part III
Download to
Ever wonder if there isn't a better way to travel? Travel is essential, but the hassles of travel aren't. Read 202 Tips for Business Travel to learn how to convert business travel from a time-wasting hassle to a breeze. Revised and updated for 2008 with 101 new tips! Check it out!
Projects never go quite as planned. We expect that, but we don't expect disaster. How can we get better at spotting disaster when there's still time to prevent it? How to Spot a Troubled Project Before the Trouble Starts is filled with tips for executives, senior managers, managers of project managers, and sponsors of projects in project-oriented organizations. Check it out!
add value to your Web sites quickly and easily. Pick up the HTML to link to this article in a choice of formats.
Are you doing work you love? Are you less in love with the job? Bad boss, long commute, troubling ethical questions, hateful colleague? Read Go For It! Sometimes It's Easier If You Run to learn what we can do when we love the work but not the job. It helps you get moving again!
The key to managing virtual or global teams is creating a sense of team despite the obstacles of separation. Read my tips booklet, 303 Tips for Virtual and Global Teams, to learn how to make your virtual global team sing. Newly revised and updated for 2008 with 101 new tips! Check it out!
Are you managing a change effort that faces rampant cynicism, passive non-cooperation, or maybe even outright revolt? Read 101 Tips for Managing Change to learn how to plan and execute your change efforts to inspire real, passionate support. Check it out!
A Tip a Day arrives by email each business day. It's 20 to 30 words at most, and gives you a new perspective on the hassles and rewards of work life. Most tips also contain links to related articles. Free!
Do you ever wonder if all these meetings are really necessary? (They aren't) Or whether there isn't some better way to get this work done? (There is) Read 101 Tips for Effective Meetings to learn how to make meetings more productive — and more rare. Check it out!
