Rosie Williams

Rosie Williams

Open Data Reporter at No Fibs
Rosie Williams (BA Sociology) owns and runs, a site that hosts database journalism projects that engage the public with open data.
Rosie Williams
Rosie Williams
Rosie is a long time activist who began lobbying the federal government at the age of 16. You can find out more about Rosie at


By Rosie Williams,

10 January, 2014


In this report called Life in the Clickstream: The Future of Journalism, Australia’s Media Alliance discusses the ways in which new technologies and economic events in recent years have put the future of an entire industry under threat.

The internet brought with it a cultural expectation that content be free. The separation of online classifieds from journalistic publications to stand alone sites means that papers are left to fund their activities primarily from their readers, readers who are accustomed to the price of their news being subsidised by advertisers.

If mainstream media publications survive this ‘perfect storm’ it might only be through changing their relationship with their audience to a more collaborative approach.

It is interesting to think of the effect  a more collaborative attitude might have on society which is seen the mainstream media historically as representing the interests of advertisers rather than readers. In recent times we have seen Fairfax media receive a Walkley for creating a searchable database publishing information from the federal government pecuniary interests register.

Interns transcribed the PDF’s in order to create the database which provided a rich source for stories regarding discrepancies in what ought to have been declared. In this case the transcription had to be done by hand as the interests data are published as images of PDF’s which can’t be mechanically scraped.

Once published, this data was then able to be mechanically scraped from the SMH site and made available on ScraperWiki by open government hacker Alexander Sadlier who then made it available to me for re-use in eXpenseAus – my own version of a searchable database.

I have combined this data with travel/office expenses data published by Finance & Deregulation both from my own work and as scraped by Nick Evershed from The Guardian for their own project to produce a page which lists politician visits (data scraped by Nick) with results from the pecuniary interests register (scraped by Alex) to show politician visits alongside the locations of their investment properties so the public can see if they have made travel expense claims for trips that may include visits to investment property locations.

Nick, Alexander (who also worked on LobbyLens) and I are part of a new movement of people who have a keen interest in opening government data for use by researchers and the public. The fact that the Pecuniary Interests Register is online at all (instead of available for inspection by appointment only) is attributable to the work of the Open Australia Foundation, a not-for-profit group who make government information more accessible through their projects (as do I) by creating databases and lobbying for data to be made available for this use.

The Guardian UK took this effort to a new level a few years ago when they set up their own lobbying effort to get the information useful to the public made freely available. I think a media collaboration on open data in Australia would be very useful and it would be simple to implement. For every article that relates to open data/FOI published in main stream, independent publications or blogs, there could be a badge included on the page which says that the article is part of a drive for open data.

This badge could lead to a page (on the publishers own site or a separate site) which explains to the public very simply what open data is about and why it is in the public interest to have data published in usable formats as well as links to all the other badged articles (either within each publication or a collaborative list).

Even journalists do not necessarily understand the difference between data made available in a PDF and the same data in a CSV file that can be sucked straight into a database for republishing – a point made very clear in this blog by the Open Knowledge Foundation.

The last year has seen a move toward collaboration with the public as two mainstream publications (SMH) & (Guardian) sought the help of the public in providing the research labour to detect potentially dodgy travel claims. I propose we take this collaboration one step further and encourage the media to join forces in asking the government to release data in usable formats – as it is already required to do under its own policies.

Such a campaign would raise public awareness of the issue of open data and open government considerably and the government would no longer be able to enjoy the current lack of accountability regarding its standard of transparency. Information is gathered by the government for its citizens and as citizens we have a right to access that data and have it made available in ways we can actually engage with it. The media has an essential role in making the public aware of such issues and I believe the time is right to move forward on this issue.