Progress - Testing Torrents

I'm updating because of the progress on downloading of a torrent to disk, the first complete results being saved a few hours ago (Woo hoo!). There's been mountains of changes in order to get this to happen, so the next couple of posts will be the build up to how the first torrent saved to disk.

Changes to Test Torrents
First, the testing grounds have greatly changed. Previously my tests have been on the latest Ubuntu ISO and on MIT's The Structure and Interpretation of Computer Programs Lectures. This last week I've changed specifically to small multiple file torrents, to which end shaver recommended a couple of the confidently legal downloads on http://www.mininova.org/.
These torrents are noticably different, particularly the treatment by peers; connections to peers are rarely accepted (presumably because each client is already well connected to the swarm); unchoking is comparatively rare. Unchoking is where the peer you're connecting to says it's okay to request blocks to download.

Changes to Peer Manager
Someone may recall I was only opening one connection to a peer and shutdowning(-ish) when that connection died. With the more 'pop' torrents mentioned above, this isn't enough for testing. The probability of hitting the one peer that accepts connections is pretty low. (How low? Not sure, I can see how those graphs and charts in Azureus could have come out of testing questions)
So, the Peer Manager is finally doing more than holding Peer objects that do nothing. Now, it attempts to connect to all the initial peers provided by the tracker, and performs a health check every 30 seconds to see if we need to attempt more connections. There's a timeout before re-trying peers, currently 3 minutes though I need to find the standard for this.

Changes to Piece downloading
Previously, I'd written so a Peer object (representing a conversation with a Peer) had a list of pieces we 'wanted' that we knew the peer had. The first 'want' would attempt to be downloaded, and when finished the data would be passed to the Piece Manager. This isn't good enough. Too often a disconnect or similar will happen before the full piece is downloaded, meaning many peers have partial pieces they can't release to the Piece Manager.
To correct this, partial pieces are now managed by the Piece Manager. This means any work on downloading a piece will be immediately useful, repeated downloads will be reduced thus the total download reduced, and seeding will occur earlier (this I want to enhance further - if a single piece is targetted first, we can start sharing earlier, which improves trade negotiations ;-) ).

Next: Saving Folder, Tracker Changes, Little Things