« Working with Other Applications | Main | I was right - it is incestuous. »

Reliability vs. Validity

I just read an article by Roger Martin which resonates with some of my feelings about "Project Management Systems" and the like. He posits the two as opposite ends of a spectrum. The key point is that driving toward reliability (repeatability) requires reducing the number of variables and standardizing measurements. In doing so, your results are less valid - that is they are less likely to reflect reality.

I do a poor job of describing it here so go read it yourself and see which direction you are heading. Personally it helps me justify my bias against big project management tools.

RELATED POSTS

Comments (4)

Joël Séguin:

I don't get why you can't simply aim for both Reliability and Validity. Information and process reliability systems are there because reliability IS important for shareholders and the like. Validity is as important, but is also more internal. What strikes me is that for every example given, the reliability "system" can't be there if the associated validity "system" isn't. You can't manage knowledge without the creativity to compose knowledge at first. So I guess the debate is about existing reliable empty shells.

I don't agree that standardization reduces validity. You can be valid and standardize at the same time, or when impossible, standardize after. "Guesstimating" is a form of valid thinking which is not standardized, but by putting that into a smart, adapted, standardized system, you can still keep the essence of the validity, which is your guess (from the extent of your experience).

Dumb, unadapted, unadaptable standards could be the source of where the validity drops.

Jack:

Joël,

At some level I agree with you. They are NOT exclusive. But certainly you can see the point that an excess of standardization can reduce validity. To make it simple if you round the corners off of all your square pegs to fit your corporate goal of all pegs in round holes you might find that customers who are buying your square pegs are a bit disappointed. Also the people in your company who put the sharp edge on your pegs might decide it is not worth getting all their nice sharp edges getting ground down time and time again and may leave for a place which appreciates sharp edges.
When possible it is nice to have both. Balance is essential.

Mark:

I am not to sure about the content of the article but your point "Personally it helps me justify my bias against big project management tools" is one that I am wrestling with currrently.

Can anyone tell me the real benefit of Enterprise PM Tools? They all rely on the source data from say MS Project and it is here that most of the problems exist. Once I get my project managers to use MS project correctly (in a way that means I get the information I need) the three most valuable pieces of information fall out easily, e.g. is the project plan up to date, what is the earned value analysis, who is planned to do what when.

I'd love to know what other real benefits they provide, and before it is suggested, my senior management team are never going to log on to project server to get their own real time information.

Grateful for any contribution.

Mark

Jack:

One benefit is with large geographically dispersed teams. In that case team members can log in, see what they are supposed to do and report progress against the tasks they are assigned. Kind of like a timecard system but with some limited gantt chart ability.

Where I think this is limited is that on a large project that sort of individual progress report is less useful than information about what the current problems are and what is keeping people from progressing. Figuring this stuff out is not possible with a dumb web app. People having problems need to talk to their managers and solve problems. Then their managers should communicate what is being done to take care of those issues.

If I boil it all down, my opinion is that ERP's are too complicated and yet too dumb. They have a high cost and in my opinion, questionable returns.

Post a comment

(Comments are moderated to fight SPAM and will be published after I have a chance to approve them. Thanks for waiting.)

About

The previous article is Working with Other Applications.

The next article is I was right - it is incestuous..

Current articles are in the main index page and you can find a complete list of articles in the archives.

Creative Commons License
This weblog is licensed under a Creative Commons License.
Powered by
Movable Type 3.34