Seminar 5th July 2016 6 p.m. 58/1023 (Murray Building L/R G)
Writing Python to process millions of row of mobile data - in a weekend
Robin Wilson
University of Southampton
- Web page
- http://southampton-python.github.io/
- Categories
- Built Environment, Complex Systems, Computational Social Science, Data Aggregation, Data Management, Data Science, Database, Digital Humanities, Environmental hazards, Healthcare modelling, Human environment interaction, Human population, Mobile phone data, Python, Sensor Networks, Social and Socio-economic Systems, Software Engineering, statistical analysis, Wireless Communications
- Submitter
- Thomas Kluyver
In April 2105 there was a devastating earthquake in Nepal, killing thousands and displacing hundreds of thousands more. Robin Wilson was working for the Flowminder Foundation (www.flowminder.org) at the time, and was given the task of processing millions of rows of mobile phone call records to try and extract useful information on population displacement due to the disaster. The aid agencies wanted this information as quickly as possible – so he was given the unenviable task of trying to produce preliminary outputs in one bank-holiday weekend…
This talk is the story of how he wrote code in Python to do this, and what can be learnt from his experience. Along the way he’ll show how Python enables rapid development, introduce some lesser-used built-in data structures, and show a slightly different approach to data processing.
Robin Wilson is a Research Fellow at the University of Southampton, and an Analyst at the Flowminder Foundation. His research focuses on applications of satellite imagery, and he is currently finishing the development of an algorithm to monitor air pollution from satellite imagery at an unprecedented resolution. He is an experienced Python programmer, and has released a number of open-source Python modules (see github.com/robintw). He blogs at blog.rtwilson.com and tweets @sciremotesense.