Tycho file swarm test

As part of the benchmarking process I have fixed some performance issues with the Tycho swarm file distribution software.

In some circumstances the HTTP send code in the Tycho mediator would send messages serially to other mediators. This limited the overall bandwidth usage of the swarm to the bandwidth of the seed peer. This is exactly what file swarms are supposed to avoid, each new peer should contribute more bandwidth to the swarm.

I altered the send method to a threaded implementation and as predicted this corrected the problem. Below is a graph comparing the time to publish a single file from holly to the nodes of the cluster using first the HTTP protocol (Apache) and the Tycho software . We vary the size of the file and number of nodes we publish to and measure the average time to distribute the file.

You can clearly see that the additional time for each extra node using Tycho is less than when using HTTP (look at the gradient of the lines). A problem we have is that the bandwidth of the Ethernet network is high enough that for smaller message sizes we do not have enough nodes to saturate the network. We either need lower bandwidth or more nodes (and it us unfair to run more than one client per node).

For this reason we have shelved the tests until we have access to a machine with more nodes (such as the new one at Reading).

I have moved to pretty much full time writing up using the W2 wiki. The initial focus is the review chapter (the hardest).