For journalists, Craig Newmark is the meteor that came hurtling to Earth and destroyed their world. Craigslist, the unassuming site that bears his name, is thought by some to have killed one of the major cash cows of the US newspaper industry – classified advertising.
When he started the site in early 1995, he says: "I had as few thoughts as possible and certainly no vision. In particular, I had no idea that it might affect the news business." Newmark has his roots in technology, not in media or journalism. He worked at IBM for 17 years before going to San Francisco to work for the stockbroker Charles Schwab.
Free to a good home
Craigslist is a great advertisement for open-source technology. Newmark programmed most of the site in Perl, and the servers run Linux, Apache and MySQL. He told the tech consultancy O'Reilly last year that the only software Craigslist has paid for is accounting software.
The site began shortly after his move to the Bay area of California as a simple mailing list for events; the mailing list moved to the web in 1996. The look of the site hasn't changed that much over the years, but it has grown well beyond its roots.
Newmark refers to the negative impact of Craigslist on journalism as "an urban myth", and certainly making Newmark the poster child for all the woes of the US newspaper industry doesn't really hold up to scrutiny. For the first 10 years of Craigslist, the US newspaper industry continued to grow. In 2005, US newspapers enjoyed their highest advertising revenues in history, raking in $49.4bn (then £28bn), according to the Newspaper Association of America.
Classified advertising continued to grow during seven of the 10 years after the site was founded. However, newspaper revenues have dropped since 2005, the decline accelerating in the past two years as the recession adds to double-digit falls in formerly lucrative auto and housing classifieds. Is that down to Craigslist or the worst recession in decades?
According to figures on the site, Craigslist has 20bn page views a month, and its users post 40m ads a month on 570 sites in 50 countries. The site is in the top 10 in the US for traffic, according to the web data company Alexa, and Hitwise puts it just outside the top 10 at 14.
Craigslist does all this with a staff of "about 30". Among those is Jeremy Zawodny, who joined from Yahoo in July 2008. He works on MySQL and much of the key back-end infrastructure for the site. In a presentation in April, Zawodny pointed to a number of the challenges that Craigslist faces – including its rapid growth, need to archive hundreds of millions of postings "forever", and the increasingly pressing problem of being able to search the site, with all those postings: it gets up to 50m queries per day. MySQL's own full text search was simply too slow, and prone to corrupt data.
The solution Zawodny found was to implement Sphinx, an open source software search engine that integrates natively with MySQL and is optimised for indexing and search. It's even able to implement "nearby search". The only external worries that Zawodny – and hence Craigslist – has is about the possible effects of Oracle's acquisition of Sun, announced in April, on MySQL, which Sun owns.
With its intense focus on having a really efficient database, and what appears (to the uncritical eye) like a very simple web interface, Craigslist doesn't look much like the sort of business that a news organisation would want. But many news executives would love its revenues – partly because some of them are convinced those revenues used to belong to them. Without them, traditional news organisations are scrambling to find new business models even while they try to continue within their old structures; the result is not pretty.
However, Newmark sees a range of business models to support journalism in the future – including sponsorship, advertising, philanthropic and pay-per-view models. News organisations will become "networks of reporters, editors and factcheckers", he says.
Craigslist is as much about people connecting with each other as it is about selling things, Newmark says; the site was describing itself as a community long before the term came into vogue in digital media. The mission of Craigslist is public service, or "customer service done in good conscience", he says.
To run a site such as Craigslist requires "continuous engagement between a company and its customers, its community", he says, which he believes applies to all companies and even governments.
"There is this large movement where the internet is facilitating large-scale, grassroots democracy in a way that has never been possible," he says.
He is now working on how the US government can provide better customer service through the internet.
"In these tight economic times, the need for accountability in government is stronger than ever," he says.
Journalism used to be one of the main ways to hold government to account, but with these internet-led initiatives to open government, maybe government accountability is getting a much-needed upgrade.
