Crawling Online Social Networks
- 1 September 2015
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in 2015 Second European Network Intelligence Conference
Abstract
Researchers put in tremendous amount of time and effort in order to crawl the information from online social networks. With the variety and the vast amount of information shared on online social networks today, different crawlers have been designed to capture several types of information. We have developed a novel crawler called SINCE. This crawler differs significantly from other existing crawlers in terms of efficiency and crawling depth. We are getting all interactions related to every single post. In addition, are we able to understand interaction dynamics, enabling support for making informed decisions on what content to re-crawl in order to get the most recent snapshot of interactions. Finally we evaluate our crawler against other existing crawlers in terms of completeness and efficiency. Over the last years we have crawled public communities on Facebook, resulting in over 500 million unique Facebook users, 50 million posts, 500 million comments and over 6 billion likes.Keywords
This publication has 18 references indexed in Scilit:
- Crawling Facebook for social network analysis purposesPublished by Association for Computing Machinery (ACM) ,2011
- Walking in Facebook: A Case Study of Unbiased Sampling of OSNsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Prying Data out of a Social NetworkPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- User interactions in social networks and their implicationsPublished by Association for Computing Machinery (ACM) ,2009
- Eight friends are enoughPublished by Association for Computing Machinery (ACM) ,2009
- Link privacy in social networksPublished by Association for Computing Machinery (ACM) ,2008
- Measurement and analysis of online social networksPublished by Association for Computing Machinery (ACM) ,2007
- Parallel crawling for online social networksPublished by Association for Computing Machinery (ACM) ,2007
- Analysis of topological characteristics of huge online social networking servicesPublished by Association for Computing Machinery (ACM) ,2007
- Sampling from large graphsPublished by Association for Computing Machinery (ACM) ,2006