NeTS: Small: Incentivizing Internet-Scale Web Mining with Webcoin

Overview: Four major search engines, Google in particular, hold a unique position in enabling the use of the Internet, as they alone direct over 98% of Internet users to the content they seek, using proprietary indices. While the contribution of these companies is undeniable, their design is necessarily affected by their economic interests, which may or may not align with those of the users, raising concerns regarding their effect on the availability of information around the globe. While multiple academic and commercial projects aimed to distribute and democratize the Web search, they failed to gain much traction, mostly due to inferior results and lack of incentives for participation. In this project, we show how complex networking-intensive tasks can be crowdsourced using Bitcoin’s incentive model. We present Webcoin, a novel distributed digital-currency which utilizes networking resources rather than computational, and can only be mined through Web indexing. Webcoin provides both the incentives and the means to create Google-scale indices, freely available to competing services and the public. In designing Webcoin, while utilizing Bitcoin’s incentive model, we address several major drawbacks associated with Bitcoin itself. First,Webcoin disrupts its dependence from computation and instead fosters crowdsourced Web crawling and indexing. In this way, it addresses the problem of the unparalleled computation power involved in Bitcoin mining, currently at 8–10 exa-hashes per second. The wasted power serves no other purpose beyond Bitcoin, and by 2020 Bitcoin could consume as much energy as Denmark. Second, Webcoin is a first digital crypto-currency that dominantly relies on networking resources in the mining process. Such resources are inherently distributed and cannot be scaled at the same rate as computational resources, increasing Webcoin’s fairness, relative to Bitcoin, by a factor of 10M. Third, contrary to Bitcoin, whose mining process creates nothing useful for the rest of the world, Webcoin forces miners to do useful, Internet-scale, work, i.e., crawl and index the Web. This helps break the key barrier towards democratizing the search engine space. The key goal of this project is to design, implement, and evaluate Webcoin, and make it publicly available. Intellectual Merit: The proposed research will address fundamental questions that are key to developing and deploying Webcoin. The transition of the cryptographic principles upon which Bitcoin is built into the networking domain is non-trivial. A major difference between the domains is that while the first is governed by a priori mathematical and statistical principles, the latter depends on real-world state, and is thus susceptible to manipulation. Can we, and how, provide the same security guarantees as in Bitcoin, in such a challenging scenario? Next, the system must provide incentives for every single Webcoin miner to publish its index, so that the entire product of the indexing effort is made available to all. How do we create such incentives and cope with the significant scalability challenges? Further, the system should not only create a Google-scale Web index, but provide guarantees for its quality. How do miners verify the quality of the Web index in a scalable manner? Finally, how do we democratize the mining process itself: enable the use of existing personal computers and surplus residential bandwidth for mining in the background, while guaranteeing equal likelihood to everyone to mine a block and earn the associated rewards? Broad