Ben Beaumont-Thomas 

Bookworm data hack lets you map every word in 100,000 screenplays

The hack by university professor Benjamin Schmidt allows you to search for the frequency of words used in TV and film – with findings like Woody Allen using ‘sex’ less and less
  
  

Woody Allen
Woody Allen: money better than sex? Photograph: Andrew Medichini/AP

If you’ve ever wanted to find out that Steven Soderbergh never put the word ‘sex’ in his films in the late 90s, or that ‘fuck’ has overtaken ‘shit’ as South Park’s curse of choice, you’re in luck: an ingenious data visualisation by US professor Benjamin Schmidt is mapping all the words used in 100,000 films and TV show episodes.

It uses the Bookworm tool, which has already visualised the words used in science publications, newspapers and Congress bills, and scans a database of captions to draw down its data.

You can tailor your enquiry to an impressive degree, searching the words used in individual shows like Friends and Family Guy, or by individual film directors including John Ford, Claude Chabrol, and Takashi Miike. There’s also scope for searching by genre, language, film studio and even filming location. Once you’ve got your search parameters, a graph is created with frequency of the words used plotted over the last few decades.

One example graph plots the amount that Woody Allen mentions ‘sex’ and ‘money’. As he gets older, the former declines while the latter ascends – psychologists can make of that what they will.

We took the tools out for a spin, and found the following factoids.

  • Across all film and TV, “sex” became more and more popular as the 20th century went along, reaching an apex in 2008 – since when it has markedly dropped off. Maybe we’re not all so single-minded after all.
  • Doctor Who mentioned “time” a faintly ridiculous 4,400 times per million words for its 1983 episodes, later settling back down to its usual level of 1,500.
  • For Steven Spielberg, “love” always conquers “dead”, with Saving Private Ryan notching a big 1,205 mentions of love. Only Indiana Jones and the Temple of Doom sees more “dead” mentions (see below).
  • Robert Altman also felt the love more as his career went on, reaching a swoony peak with Gosford Park and A Prairie Home Companion.
  • Aside from a dip in the mid-60s, Alfred Hitchcock became increasingly preoccupied with “dead” – rising from around 100 words per million in the 30s to 550 in his last films.
  • After an initial infatuation, Homer Simpson gets over his “doughnut” obsession for the central few series of the Simpsons, only to rekindle his love in the latest episodes.
  • “Meth” doesn’t deviate from a steady 350-400 mentions per million throughout Breaking Bad.
  • Martin Scorsese hit headlines for his “fuck”-filled Wolf of Wall Street, and it’s a word he’s become steadily more fond of, rising from around 500 per million in the 80s to 1,750 in recent years – and Shutter Island actually has more “fucks” per million than Wolf.
  • The cast of Lost, much like the audience, used the word “confused” more and more as the seasons passed.
  • Bookworm suggests that “feck” was used at a constant rate of 730 words per million throughout every episode of Father Ted. It might be a glitch.

We could play with this all day, so post any funny or interesting bits of data you find below.

 

Leave a Comment

Required fields are marked *

*

*