bittorrent.org | This is the repository backing www | Stream Processing library

I am working on Mainline DHT and I don't understand one nuance.

Here: https://www.bittorrent.org/beps/bep_0005.html writes: "A "distance metric" is used to compare two node IDs or a node ID and an info hash for "closeness."

Also writes: "Announce that the peer, controlling the querying node, is downloading a torrent on a port. announce_peer has four arguments: "id" containing the node ID of the querying node, "info_hash" containing the info hash of the torrent, "port" containing the port as an integer, and the "token" received in response to a previous get_peers query."

So, for example, we have a peer with ID 223456789zxcvbnmasdf which IP is 86.23.145.714 and port is: 7853 I know that this peer downloaded 2 torrent files with info hashes: 023456789zxcvbnmasdf and 123456789zxcvbnmasdf.

So how exactly my k-bucket record should look like? Like this:

I need help to understand what does upload_only=3 mean in extended BitTorrent handshake.

I know about http://www.bittorrent.org/beps/bep_0021.html, and according to it extended handshake could be like:

I read official documetation here and wikipedia article about bittorrent client here but couldn't find how tracker exactly work. As per my understanding tracker should know which peer have which piece of some file. For example client 1 have 5 pieces of file 1 and 10 pieces of file 2. In official documentation I see tracker have fields like peer_id ip port uploaded downloaded left and event but I don't see where is the information about which file we are tracking. For example if I ask tracker hey I need pieces for linux.torrent file, how tracker would answer me?

When using Bittorrent, I saw there are the parameters "numwant", "corrupt" and "key" in URL.

However, I found these paremeters don't be defined in BEP 3 (http://www.bittorrent.org/beps/bep_0003.html), so could someone tell me the meaning of the parameters, and where are the 3 parameters defined?

Also, before asking the questsion, I had searched the keyword "numwant" in the site www.bittorrent.org, and just found "numwant" appears in BEP 8, but the definition or explanation of the keyword can't be found.

I read the DHT Protocol in bep_0005 page.

But when I send a ping query or a find_node query, the server response a garbled text (both of router.bittorrent.com:6881 or dht.transmissionbt.com:6881)

Here is the Java source code bellow

Using the package https://github.com/golang/crypto/tree/master/ed25519 i m trying to get a public key for a given private key.

Those data are from http://www.bittorrent.org/beps/bep_0044.html: test 2 (mutable with salt)

Problem is, ed25519.Public() won t return the same public key when i fed it with the given private key. The golang implementation returns the last 32 bytes of the PVK. But in my test data this is unexpected.

The code here https://play.golang.org/p/UJNPCyuGQB

In my new weekend project I decided to write an bittorrent client from scratch, no ready to use libraries at all. After two days looking for documentation I'm already about to give up :smile:. I know there are the BEPs, but they are far from enough to understand all the specification. After reading a lot more I think the tracker and peer protocols seems to be old and easy to understand/implement (yes, I know, to write a good code with balance, peer selection, optimizations, this is not easy as I just said, but all I want to is to do the basics to learn, not to compete with tens of good clients out there.)

So, I have decided to start by the DHT which seems to the the more complex part and also the less documented. When you stop looking for bittorrent DHT or mainline DHT and start looking for kademlia DHT you have a lot more information but it not so obvious how to put it all together.

Here is what I understand so far (and there are gaps which I hope to fill in):

1. I start with my DHT tree empty
2. use find_nodes on my bootstrap node
3. add the received nodes to my own tree, so I can then select the ones closer to my own ID
4. start issuing find_nodes to the selected ones and add their responses to my tree
5. go back to 3 until I stop receiving unknown/new nodes
6. if I receive an announce_peer with an info_hash than I should save its information on a local DB (the info_hash and ip/port of the sender)
7. if a node uses get_peers with an info_hash I have in my DB then I send the information otherwise I should send a list of closer nodes I have in my own tree (closest to that info_hash)
8. when I use get_peers on other nodes I will receive peers or nodes, in the later case I think the nodes are closer to the info_hash and not to my own nodeId so, should I add these nodes to my tree or start a new tree based on them?
9. when I want to announce I am interested on an info_hash should I use announce_peer everywhere or just to the nodes with nodeId closer to the target info_hash? How much is closer enough?

At this point I have a lot of nodes which IDs are closer to my own ID, and informations about info_hash'es I am not really interested.

I am afraid that I have a giant stupid question: why I did that?

I mean: my selfish reason to do all this work is to locate peers to the info_hash I'm interested in. I understand that the information of one info_hash is likely to be saved on a node which ID is closer to that info_hash. So my chances to find its information is bigger if I create a tree of nodes closer to the info_hash and not closer to my own ID (at this point, if you know the subject, you already noticed how lost I am).

Should I create multiples trees? One for me (to be there to save the information of info_hashes closer to my nodeID people send me), and other tree closer to each one of my target info_hashes so I can retrieve their information?

Should I create a single tree closer to my node ID and hope for the best when querying this tree for the info_hashes I need?

Should I give up since I have completely misunderstood the idea behind DHT at all?

Well, any real documentation, flowcharts, any thing will be welcome!

I'm trying to run the test vector as described in BitTorrent BEP 44 test #1, but I'm not creating the same signature as they do:

305ac8aeb6c9c151fa120f120ea2cfb923564e11552d06a5d856091e5e853cff 1260d3f39e4999684aa92eb73ffd136e6f4f3ecbfda0ce53a1608ecd7ae21f01

Instead, the signature I create using libsodium is:

c44ad65291c2b1087218db8a43e3fa7b73cfa01b585b0ff9e6b962ed50e701a1 6065277417ff5bbae43d9b76e52129d27bf2e33e8b043ea67ace7ff91dae4d02

Using this code:

I am trying to write my own bittorrent client in java, it is a make work project to refresh my java skills. I have been stuck at a certain point in development and although there are forums out there for bittorrent development they don't appear to be very active.

So far I can do the following:

1. Get a list of peers from Tracker sites
2. Connect to a peer
3. Send the 'Handshake' message
4. Receive the 'Handshake' message back
5. Send the 'Interested' message.
6. Receive the 'Unchoke' message back.

The problem I am having is if I try sending extended data (http://www.bittorrent.org/beps/bep_0010.html) to the peer prior to sending the 'Interested' message. The protocol states you must send extended data immediately after receiving the handshake. If I send the extended data to the peer the peer immediately closes the connection, ie, in Wireshark I see a FIN,ACK. I have used Wireshark and uTorrent to see what extended data is being sent and I have tried to copy that data, but to no avail. I have tried all different combinations of extended data parameters but no matter what I do I cannot get past this point.

Here is an example of extended data message that works, copied from Wireshark.

d1:ei0e4:ipv44:czH12:complete_agoi-1e1:md11:upload_onlyi3e11:lt_donthavei7e12:ut_holepunchi4e11:ut_metadatai2e6:ut_pexi1e10:ut_commenti6ee1:pi43276e4:reqqi255e1:v13:Torrent 3.52:ypi48680e6:yourip4:jFe

And here is extended data from my client that doesn't work. btw, I have checked to make sure the ip variables are correct (they are sent as 4 bytes).

d1:ei0e4:ipv44:czH12:complete_agoi-1e1:md11:upload_onlyi3e11:lt_donthavei7e12:ut_holepunchi4e11:ut_metadatai2e6:ut_pexi1e10:ut_commenti6ee1:pi51361e4:reqqi255e1:v15:MyTorrent v0.152:ypi40898e6:yourip4:RO e

Also, once I receive the 'Unchoke' message I am able to successfully retrieve pieces.

Any information would be appreciated.