Friday, July 9, 2010

Info - File Information in Bittorrent

The information about files is stored in the info section of the .torrent file. There is a slight difference in this structure if the torrent contains one file or whether it contains more than one file. To make handling some of that information a tad easier, I have a collection of methods for common tasks and to hide the single/multiple file differences.

Subtle Differences
The structure of the info data is different depending on whether it's a torrent containing one file or many. This makes for a large number of "if" conditions when handling the info data, and it's just easier if this is hidden away. The old Firepuddle Firefox/Bittorrent extension didn't get around to handling multiple files; the info data is just the tip of the iceberg of pain that multiple file torrents can be.

Common Functions
The methods the service I've written provides are fairly simple; though partly that's to keep the handling of different info data structures in one place. listFiles(in nsIVariant decodedInfo) returns an array of file paths. A single file has its path stored in a torrent file as an array, for example ["foldername","foldername","filename"] which avoids the issue of an OS specific directory separator character. Handy. So, that means the result of listFiles is an array of arrays of strings. fileCount(in nsIVariant decodedInfo) returns the number of files. fileSize(in nsIVariant decodedInfo, in long fileIndex) returns the size in bytes of a file, given an index from the list. I should note, the ordering of the listFiles function is that provided by the info data.

Not so necessary functions
A couple of functions I don't believe I've used for handling the downloading/uploading of a torrent, but I've needed for another application of bittorrent are fileInTorrent(in nsIVariant decodedInfo, in nsIVariant filePathArray) returning a boolean result (the file path array is expected to be an array of strings, as described previously), and findFile(in nsIVariant decodedInfo, in nsIVariant filePathArray) to provide the index in the fileList of a particular file path. -1 if not found, as normal.

Tetris Block functions
During a session the content of torrent is transferred as a series of blocks, in no pre-ordained order. These blocks are some subsection of the torrent, and one block may end up split into multiple files.
To figure out where a block belongs, blockToFileParts(in nsIVariant decodedInfo, in long offset, in long blockLength) returns an array of objects. Each object specifies a fileIndex, the offset in the file, and how many bytes from the offset belong to that file. The structure of this returned data is like so: [{index:...,offset:...,size:...},...].
The torrent is split into a series of pieces that have a checksum. Sometimes not all files in a torrent are desired, so not all pieces will be needed. To find out which pieces are needed for a particular file piecesInFile(in nsIVariant decodedInfo, in long fileIndex) will return an array of numbers, each number representing a piece index necessary for the file specified by the fileIndex parameter.

Implementation of FileInfo XPCOM component
IDL for FileInfo XPCOM component


var service = Components.
var fakeInfo = {
files : [
{ length : 10, path : ["a"] },
{ length : 10, path : ["b"] },
{ length : 10, path : ["c"] },
{ length : 10, path : ["d"] },
{ length : 10, path : ["e"] }
// Add piece length later, because it has a space in it's key.
fakeInfo["piece length"] = 6;
// Check service methods
// Check the list files/file count/file size/file in torrent/find file
var fileList = service.listFiles(fakeInfo);
service.fileCount(fakeInfo); // 5
service.fileSize(fakeInfo, 0); // 10
service.fileInTorrent(fakeInfo, ["a"]); // this will return true
service.fileInTorrent(fakeInfo, ["z"]); // this will return false
service.findFile(fakeInfo, ["c"]); // this will return 2
service.findFile(fakeInfo, ["z"]); // this will return -1 (not in torrent)
service.blockToFileParts(fakeInfo, 10, 9); // returns [{index:1,offset:0,size:9}]
service.piecesInFile(fakeInfo, 1); // returns [1,2,3] (pieces are indexed from 0)

Note that none of this code deals with the relationship between pieces and hashes. At the moment I don't think that's needed in this service, though I might consider a verify piece against hash method at some point. This service will probably change by the time I've finished refactoring.

More information on the .torrent structure ( Bittorrent Protocol Specification (BEP0003)
The bencoding XPCOM component.
The bittorrent tracker XPCOM component.

No comments:

Post a Comment