#BigDataWeek Community Meetup
Just back from the excellent Big Data Week Community Meet Up event organised by @StewartTownsend of @datasift and Oracle. The event centered around a panel consisting of:
- Doug Cutting, co-founder of the Apache Hadoop project and creator of Nutch and Lucene
- Nick Halstead - Founder/CTO DataSift
- Hilary Mason – Bit.ly Chief Scientist
- Andy Kirk – Visualising Data
- Edd Dumbill – Program Chair Strata Conference – Moderator
Edd kicked off the panel by asking first the audience and then the panel to succintly define big data, not an easy thing to do given the hype and marketing bluster around it. I liked Hilary Mason's description best:
We can ask questions of the data and get the answer back before we've forgotten the question.
Some of the other questions covered included:
- The role of data scientists in big data teams, in summary you need one but they're hard to find and you're not always going to find them where you expect.
- What will social data analysis allow us to do in 5 years? bit.ly Chief Scientist Hilary thinks they'll be able to derive more understanding of human nature (by analyisng pictures of dogs, apparently there's more dog pictures shared through bit.ly than cat pictures, who knew?) but @nik countered with the view that social data analytics will be imapcted by the changing attitudes of the public to privacy, a view echoed by Doug Cutting.
- One final question from the floor asked about the dark side of Big Data, what happens when these large data sets and processing capablilities are used for evil rather than good. @Nik from Datasift expects regulation to play a larger role in the future, Hilary thinks we should be mindful of the data we collect and how it could be used in the future.
A big event for a big topic, I didn't manage to get to the sessions during the day, but enjoyed the panel. Also bumped in to some ex-colleagues who have formed Tumra, a big data focused startup, check them out.
Well done to Stewart and the team for organising.
