2008/07/15

Bad Science or Science Done Badly?

Is 'Science', as practiced by Academic Researchers, executed poorly?

More specifically:
Is the practice of Research as undertaken by Academics, as effective as it could be?

This posits that an aspect of "Professional Research" is intentionally increasing your capability and effectiveness.

Computing/Information Technology is a Cognitive Amplifier - exactly suited to central parts of "Professional Research" - e.g. learning, recalling and searching published papers and books.

If an individual researcher can increase their "knowledge uptake" just 7% in a year, after a decade they know twice as much, given uptake builds on existing knowledge.

What is Research about if not Knowledge: Gathering, Analysis, Representation, Taxonomy/Ontology, Management and Communication?
This field began in 1995 and is broadly known as "Knowledge Management".

An observation:
Electronic recalling & searching are orders of magnitude faster than pure paper systems. They are logically mandated by accepting the 'increase effectiveness' proposition.

First Corollary:
If all published materials in a field are available as 'full text' electronically, then searches are guaranteed to be correct and complete.

Second Corollary:
Digital document repositories aren't limited to the restrictions of paper systems - they offer near infinite orderings, indexing and representations of the same content - and the ability to search in many ways.
Importantly, there are many methods for whole communities to use the effort spent by individuals in learning & understanding material - to leverage their work - summaries, commentaries, annotated bibliographies, ...

Mihaly Csikszentmihalyi in "Flow: The Psychology of Optimal Experience" investigated the best researchers - Noble Prize winners. He was looking at the 'What' of their work - the internal state necessary to do the best thinking. The 'How' - the techniques, mindset and practices of The Best Researchers - seems to still be an unstudied question, or at least unknown in most institutions.

There have been two powerful studies of the 'How' of the best performers:
- Kerry Gleeson, "Personal Effectiveness Program"
- David E. Kelley, "How to be a Star at Work" (Also published as "Star Performer")

Whilst Gleeson is a traditional consultant and not part of Academe, Dr. Kelley is still a professor at Carnegie Mellon University. His research, published as a book in 1998, was both methodologically sound and based in solid theory.

Kelley's research yielded some surprising insights:
  • Star Performers, those at least twice as 'productive', are known to exist in at least one cognitive practice field: Computer Programming.
  • The usual Psychology Tests/Instruments don't reveal any differences in 'Stars'.
  • the differences between 'brightest' and 'best' is not the machinery they have, but How they do their work.
Csikszentmihalyi and Kelley agree on a central point:
the techniques/states they uncovered are teachable and learnable.

Thesis:
  • Increasing Individual Effectiveness of Professional Researchers should be a major area of study and practice for Researchers, their employers and their Professional bodies.
  • Research is rarely practiced by individuals in isolation. Increasing Team Effectiveness of Professional Researchers is needed as well. Studies of programming teams by Dr. G.M. Weinberg suggest at least a factor to two increase in whole team 'productivity'.
  • "Professional Researchers" are paid to research. They are spending Other People's Money, not their own. They have a moral and Professional Duty to spend that money wisely and well.