Load reduction in the KAD peer-to-peer system

Distributed hash tables (DHTs) have been actively studied in literature and many different proposals have been made on how to organize peers in a DHT. However, very few DHTs have been imple- mented in real systems and deployed on a large scale. One exception is kad, a DHT based on Kademlia, which is part of eDonkey, a peer-to- peer file sharing system with several million simultaneous users. In this paper, we investigate the publishing and searching mechanisms in kad. We designed and implemented Mistral, a content spy that can capture up to ten million references to published content in several hours. At first evaluation, we notice that publishing new content in a kad system is much more expensive than searching and retrieving existing content. Indeed, measurements show that of all the Internet traffic generated by kad-based peer-to-peer networks, 90% is for publishing and 10% for re- trieving existing files. Moreover, the most frequently published keywords are meaningless stopwords.We propose to add a stopword filtering mech- anism to the search and publish procedures of kad-based peer-to-peer systems.