CRESC - Data Mining and Crowd Sourcing

I will be at the CRESC 'The Social Life od Methods' conference in Oxford today. The conference takes place at St. Huges College and covers different aspects of methods in social sciences.

I am presenting a joint 'CASA' paper focusing on data mining and crowd sourcing. The team behind it is Andy Hudson-Smith from Digital Urban, Richard Milton, Steven Gray and myself. We are focusing on the development of tools that make use of the recent wave of digital social networking tools and methods.

For the first time it is possible to gather large scale social data sets containing detailed information about activity, interests and network of thousands of individuals. This is extremely exciting and interesting for spatial analysis. The city is not just a built artefact, but a living structure continuously changing with the inhabitants shaping it. Using this data we can start to understand the immediate connection between the environment and individual a well as collective decision making and opinion.


Image by single-blogs / A visualisation of my over 1200 facebook connections and how they are connected, using the Gephi software.

To illustrate this I will be using three recent projects that we have developed here at CASA. It will feature the 'Mood Mapper' turned into MapTube. The brand new Survey Mapper system that allows everyone to set up a survey and get the result on a map and the New City Landscapes generated from tweets sent in an urban area.

From the abstract:

The paper describes how we are harnessing the power of Web 2.0 and related technologies to create new methods to collect, map and visualise geocoded data as an aid to further our understanding of social, spatial and temporal change in cities. The authors begin with an insight into the ‘Ask’ survey system developed as part of the National infrastructure for e-Social Simulation (NeISS) project. ‘Ask’ provides social scientists with series of online tools to collect and visualise data in near real-time allowing the creation of ‘mood maps’ linked to a backend geographic information system. We examine the systems use to date, specifically by the BBC, and the implications of allowing anyone to survey the world, continent, nation, city or indeed street via our social survey system.

The authors expand on the concept with the additional of data mining social networks such as Twitter to collect, map and analyse social related data. Developed around the populist name ‘Tweet-o-Meter’ we have developed a system to mine data within a 30 km range of urban areas, focusing on New York, London, Paris, Munich, Tokyo, Moscow, Sydney, Toronto, San Francisco, Barcelona and Oslo. The system mines all geo-located Tweets creating a vast database of social science data and numerous challenges for both visualization and analysis. The paper concludes by arguing that data mining has notable potential to aid our understanding of the complex social, spatial and temporal structures of the city environment.


Image by single-blogs / Moscow New City Landscape, explore areas you know close up and find new locations you have never heard of. Click HERE for a full screen view.