Using statistics

He uses statistics as a drunken man uses lampposts — for support rather than illumination. Andrew Lang

Peter Norvig wrote an excellent article on how to conduct empirical studies and how to interpret the resulting data.

He starts with common mistakes like a lacking or wrong control group, too few subjects for your experiment, or the lack of a theory. He continues with common mistakes in data interpretation. Mixing up conditional probabilities P(A|B) with P(B|A), or confusing causality with correlation.

Norvig is well known to almost every computer scientist as he published the leading book "A Modern Approach To Artificial Intelligence".

If you're too lazy to read Novigs article or found interest in this topic, you may also like to watch Peter Donnelly's talk at TED 2005. He talks about common misconceptions in statistics and presents many comprehensible examples.