In addition to its fast and efficient file transfer protocol, Bittorrent and other peer-to-peer file sharing networks bring another interesting technology to the masses: the Distributed Hash Table (DHT). In the case of Bittorrent, this is used to look up and download a torrent file based on its magnet link. The DHT network for Bittorrent is called Mainline DHT based the Kademlia DHT, although the network itself is separate from other implementations.
For Mainline DHT, there is an interesting open source client and library, called “mldht”. There are two instances on Github, with moreus/mldht as the original and the fork the8472/mldht. Although, both seems to be somewhat active. Both depend on the EdDSA library which the8472 has also forked. To get started, grab the source, and make sure the “mldht” code depend on or have access to the “ed25519” project. To confirm run the 28 unit tests from “mldht”. They should all pass.
The “mldht” project includes a stand-alone DHT server node which can be started through the executable Launcher class. Its main configuration is in “config.xml” which gets written to the current directory if it does not already exist. It’s based on “config-defaults.xml”. To be able to connect to the DHT network, I had to change the option “multihoming” to false in the config file.
Furthermore, in order to use the CLI client utility, the CLI server has to be enabled. I ended up with the following configuration file.
Once started, the activity can be observed in log/dht.log. There will be plenty of noise.
While the server is running, the CLI Client executable can be used to issue a few commands. Of interest is the “SAMPLE” command which lists random hash keys from random peers. Using that output, the “GETPEERS” can be used to look up specific hashes (make sure to remove the space formatting from the sample output). Finally, a torrent file can be downloaded with the “GETTORRENT” command.
Assuming the Java classpath is set correctly, example use would look like:
The hash used above is the Wikileaks “insurance” file posted last December, with the name “2016-12-09_WL-Insurance.aes256”. The “mldht” project does not contain any tools to actually read the torrent, but we can use the Transmission client:
The expected output would look like this:
Given that this worked fine, I’m thinking it should be trivial to create a custom ML DHT client which performs the steps above. I hope to come back to that in a future article.