Tag Archives: big data

The Topsy-Turvy World of Trendspotting

Poster of Alexander Trend forecasters project everything from staffing and hiring needs to next year’s “it” color. Through a combination of instinct, experience, statistical modeling and not a little bit of finger-crossing, they tell clients where best to place their billion-dollar bets. Even granting the occasional self-fulfilling prophecy, it’s never been an easy gig, and the consequences of failure can be ruinous.

Today, big data is changing the field, providing unprecedented amounts of information even as it churns out predictive algorithms no one quite understands. In this article, I take a look at the past and future of this prognosticative trade and examine …

How Trend Forecasters Work

The Causation-Correlation Conflation

Not equal signThe question of cause  has haunted science and philosophy from their earliest days, in part because humans are wired for pattern-matching and confirmation bias. For all our supposed rationality, we confuse coincidence with correlation and correlation with causality.

Consequently, scientists must carefully design and control their experiments to remove bias, circular reasoning, self-fulfilling prophecies and hidden variables. They must respect the requirements and limitations of their methods, draw from representative samples and not overstate their results. Sometimes, however, that’s easier said than done. Read on to hear about…

10 Correlations that are Not Causations