Point Lookout An email newsletter from Chaco Canyon Consulting
Point Lookout, a free weekly email newsletter from Chaco Canyon Consulting
   April 18, 2007 Volume 7, Issue 16
 
   Recommend this issue to a friend
   Join the Friends of Point Lookout
   HTML to link to this article…
Archive: By Topic    By Date
Links to Related Articles
Sign Up for A Tip A Day!
   Create a perpetual bookmark to the current issue AddThis Button

Ten Reasons Why You Don't Always Get What You Measure: Part I

by Rick Brenner

One of the "truisms" floating around is that "You get what you measure." Belief in this assertion has led many to a metrics-based style of management, but the results have been uneven at best. Why?

Many believe that "You get what you measure." The belief persists, in part, because of anecdotal evidence; because some experiments do appear to be consistent with the assertion; and because so many of us believe that the rest of us believe it.

The Mars Climate Orbiter, which was lost in 1999
The Mars Climate Orbiter, as it would have appeared if it had not been lost upon entry into Mars orbit. Mars Climate Orbiter was lost because of a mismatch between Metric and English units in a file used by its trajectory models. The measurements were all objective, and they were even precise, but confusion led to a loss of the orbiter. Photo courtesy US National Aeronautics and Space Administration.
Still, there are reasons to question its validity. What follows is a catalog of possible explanations for variances between the promise and the reality of metrics-based management. In this Part I, we examine three assumptions underlying the measurement process. See Part II for more.
We assume that indirect measurements work
Sometimes we try to measure attributes that aren't directly measurable. For instance, when we try to measure immeasurables like loyalty or initiative, we actually measure something else that we assume is highly correlated with what we're trying to measure. We usually do this using interviews or surveys.
But too often, rigorous proof of the assumed correlation is unavailable. Sometimes, we comfort ourselves, saying, "it's so obvious," but this is risky — the history of management, psychology and science is replete with assumptions that, though obvious, were nonetheless false.
We assume that all attributes are measurable We assume (wrongly)
that all organizational
attributes are measurable
The word "measurement" evokes our experiences determining physical attributes like length, weight, or temperature. This leads us to assume that whatever we want to know can be determined by a suitable measurement, but that assumption can lead to trouble. Consider something as important as progress. Suppose a team has been working for three weeks, when, suddenly, someone realizes that their entire approach will never work. This is certainly progress — they've learned something important. But it probably won't register as progress in the organization's metrics. Most likely, it will be reported as a setback.
Measuring non-physical attributes, such as the advance of knowledge, is often possible when changes are incremental. But at times, our metrics fail, and they tend to fail precisely when we most want to know where we stand.
We assume that objectivity implies precision
Sometimes we use measures that are objective but imprecise. That is, we assume wrongly that multiple identical measurements would yield nearly identical results. For instance, when we measure attitudes using a survey "instrument," we assume that the results we obtain are relatively context-independent. We don't actually know that the results are independent of, say, the time of the month, or the price of the company's shares — we just assume it.
Rarely do we test these assumptions. Indeed, we often assume that these factors do affect the results. We know this because we sometimes observe organizations gaming the measurement for favorable results, or trying to influence the results by releasing favorable news.

Assumptions like these can account for some — not all — of the poor performance of metrics-based management. In a future issue, we'll examine the effects of employee behavior. Go to top  Top  Next issue: When Stress Strikes  Next Issue
AddThis Button


52 Tips for Leaders of Project-Oriented OrganizationsAre your projects always (or almost always) late and over budget? Are your project teams plagued by turnover, burnout, and high defect rates? Turn your culture around. Read 52 Tips for Leaders of Project-Oriented Organizations, filled with tips and techniques for organizational leaders. Order Now!
Your comments are welcome
Would you like to see your comments posted here? Send me your comments.
About Point Lookout
Thank you for reading this article. I hope you enjoyed it and found it useful, and that you'll consider recommending it to a friend.

Point Lookout is a free weekly email newsletter. Browse the archive of past issues. Subscribe for free.

Support Point Lookout by joining the Friends of Point Lookout, as an individual or as an organization.

Do you face a complex interpersonal situation? Send it in, anonymously if you like, and I'll give you my two cents.

Related articles
More articles on Personal, Team and Organizational Effectiveness:
Bowling a strikePygmalion Side Effects: Bowling a Strike
Elise slowly walked back to her office, beaten. Her supervisor, Alton, had just given Elise her performance review — her third consecutive "meets expectations." No point now to her strategy of giving 120% to turn it all around. She is living a part of the Pygmalion Effect, and she's about to experience the Pygmalion Side Effects.
Tugboats workingBecome a Tugboat Captain
If your job responsibilities sometimes require that you tell powerful people that they must do something differently, you could find yourself in danger from time to time. You can learn a lot from tugboat captains.
Detour SignHow We Avoid Making Decisions
When an important item remains on our To-Do list for a long time, it's possible that we've found ways to avoid facing it. Some of the ways we do this are so clever that we may be unaware of them. Here's a collection of techniques we use to avoid engaging difficult problems.
Abilene,  Texas, USATrips to Abilene
When a group decides to take an action that nobody agrees with, but which no one is willing to question, we say that they're taking a trip to Abilene. Here are some tips for noticing and preventing trips to Abilene.
Spell checkerEmail Antics: Part III
Nearly everyone complains that email is a time waster. Yet much of the problem results from our own actions. Here's Part III of a little catalog of things we do that help waste our time.

See also Personal, Team and Organizational Effectiveness and Critical Thinking for more related articles.

Coaching services
I offer email and telephone coaching at both corporate and individual rates. Contact me for details at rbrenner@ChacoCanyon.com or (617) 491-6289, or toll-free in the continental US at (866) 378-5470.
Get the e-book!
Past issues of Point Lookout are available in four e-books:

Available in early 2009: 2007-2008.

Reprinting this article
Are you a writer, editor or publisher on deadline? Are you looking for an article that will get people talking and get compliments flying your way? You can have 500 words in your inbox in one hour. License any article from this Web site. More info
Public seminars
Managing Virtual Teams for Real Results
Managing global or dispersed teams is challenging — miscommunications, misunderstandings, and interpersonal conflict all thrive in the typical environment of the distributed team. And they're even more common in global teams, because of time-zone offsets and language and cultural differences. We'll inventory the challenges distributed and global teams face, and provide tools for anticipating and addressing them. The focus of this program is practical — attendees will learn concrete techniques for preventing and dealing with the problems that accompany global and distributed teams. Read more about this program. Here are some upcoming dates for this program:

Organizational Politics for People Who Hate Politics
Have you ever felt powerless to implement an important new idea? Have you ever been "blind-sided" at a meeting? Have you ever lost two good employees because you could find no way to keep them from attacking each other? These are some of the issues of organizational politics. Many of us have become enmeshed in them from time to time, but we've also known some people who seem to be able to engage and prosper. How is that done? We'll inventory the challenges of organizational politics, and provide tools for anticipating and addressing them. The focus of this program is practical — attendees learn concrete techniques for dealing with the problems that arise in workplace politics, while keeping their integrity intact. Read more about this program. Here are some upcoming dates for this program:

The Politics of Meetings for People Who Hate Politics
There's a lot more to running an effective meeting than having the right room, the right equipment, and the right people. With meetings, the whole really is more than the sum of its parts. How the parts interact with each other and with external elements is as important as the parts themselves. And those interactions are the essence of politics for meetings. This program explores techniques for leading meetings that are based on understanding political interactions, and using that knowledge effectively to meet organizational goals. Read more about this program. Here's an upcoming date for this program:

Person-to-Person Communication for Project Managers
When we talk, listen, send or read emails, read or write memos, or when we leave or listen to voice mail messages, we're communicating person-to-person. And whenever we communicate person-to-person, we risk being misunderstood, offending others, feeling hurt, and being confused. There are so many ways for things to go wrong that we could never learn how to fix all the problems. A more effective approach avoids problems altogether, or at least minimizes their occurrence. In this very interactive program you'll learn a model of inter-personal communications that can help you stay out of the ditch. In those moments of intense involvement, when we're most likely to slip, you'll have a new tool to use to keep things constructive. Read more about this program. Here are some upcoming dates for this program:


Contact Information
Download vCard
Download vCard
Richard Brenner
Chaco Canyon Consulting
700 Huron Avenue, Suite 11J
Cambridge MA, 02138

Phone: (617) 491-6289
Toll-free: (866) 378-5470 in the continental US
Fax: (617) 395-2628
Email: rbrenner@ChacoCanyon.com
Copyright © 1998-2008 Richard Brenner. All Rights Reserved.
Site Map  Terms of Use  Privacy Policy  Returns Policy
Date and time limits of special offers on this site are New York time.
Last Modified: Wednesday, 23-Jul-2008 04:18:22 EDT
Valid HTML 4.0! Valid CSS! Next Issue State of the Art Teamwork Essays on Teamwork, Conflict and Project Management April Is Workplace Conflict Awareness Month Workshops and Seminars Point Lookout Archive for 2006 Point Lookout Archive for 2001 Point Lookout Archive for 2002 Point Lookout Archive for 2003 Point Lookout Archive for 2004 Point Lookout Archive for 2005 E-Books and E-Booklets for People at Work Google
Want to tame the hassles of business travel?
202 Tips for Business TravelEver wonder if there isn't a better way to travel? Travel is essential, but the hassles of travel aren't. Read 202 Tips for Business Travel to learn how to convert business travel from a time-wasting hassle to a breeze. Revised and updated for 2008 with 101 new tips! Check it out!
More
How to Spot a Troubled Project
How to Spot a Troubled Project Before the Trouble StartsProjects never go quite as planned. We expect that, but we don't expect disaster. How can we get better at spotting disaster when there's still time to prevent it? How to Spot a Troubled Project Before the Trouble Starts is filled with tips for executives, senior managers, managers of project managers, and sponsors of projects in project-oriented organizations. Check it out!
More
Support
Point Lookout
by starting your Amazon search here
When you start here, a part of every purchase you make goes to support Point Lookout, at no cost to you.
Search Now:
Amazon Logo
Webmasters: Add Value to Your Sites!
Webmasters, Two links in a chainadd value to your Web sites quickly and easily. Pick up the HTML to link to this article in a choice of formats.
More
Love the Work But Not the Job?
Go For It: Sometimes It's Easier If You RunAre you doing work you love? Are you less in love with the job? Bad boss, long commute, troubling ethical questions, hateful colleague? Read Go For It! Sometimes It's Easier If You Run to learn what we can do when we love the work but not the job. It helps you get moving again!
More
Virtual Team or Virtual Catastrophe?
303 Tips for Virtual and Global TeamsThe key to managing virtual or global teams is creating a sense of team despite the obstacles of separation. Read my tips booklet, 303 Tips for Virtual and Global Teams, to learn how to make your virtual global team sing. Newly revised and updated for 2008 with 101 new tips! Check it out!
More
Is your change project stuck in neutral?
101 Tips for Managing ChangeAre you managing a change effort that faces rampant cynicism, passive non-cooperation, or maybe even outright revolt? Read 101 Tips for Managing Change to learn how to plan and execute your change efforts to inspire real, passionate support. Check it out!
More
Subscribe to my free newsletter
Point Lookout, a free, weekly, email newsletter, gives concrete tips and suggestions for dealing with the challenging but everyday situations we all face. Subscribe instantly or read more about it.
More
Subscribe to
A Tip A Day!
A Tip A DayA Tip a Day arrives by email each business day. It's 20 to 30 words at most, and gives you a new perspective on the hassles and rewards of work life. Most tips also contain links to related articles. Free!
More
Is your organization mired in Meeting Madness?
101 Tips for Effective MeetingsDo you ever wonder if all these meetings are really necessary? (They aren't) Or whether there isn't some better way to get this work done? (There is) Read 101 Tips for Effective Meetings to learn how to make meetings more productive — and more rare. Check it out!
More
Are You an Ace at What You Do?
Exchange your "personal trade secrets" — the tips, tricks and techniques that make you an ace — with other aces, anonymously. Visit the Library of Personal Trade Secrets.
More
Download a
catalog of services
If your teams don't yet consistently achieve state-of-the-art teamwork, check out this catalog. I can help!
More