Blog

Posted 2005/12/01

Initial Tycho registry benchmarks

Spent Wednesday day and night getting an initial benchmark done, in this post I explain my methodology and publish some results.

  1. Time to complete a simple query while increasing the number of records in the registry.
  2. The overhead of the p2p registry link.

These tests were done a single 3ghz machine at home. Tycho modules I used were the HSQLDB data store with SQL as the query language, LDIF as the response markup and the simple HTTP P2P registry connector. Where I say registry I mean a registry within a mediator.

I populated a single registry with N unique records of the simplest form Tycho lets you do:

('producer','socket://192.168.0.16:NUMBER/producer/NAME',
'NAMENUMBER','','')

The records were made unique by adding an incrementing number to the end of the name field.

Using a random number between 1 and the number of records in the registry I could create a query which I knew would match a single record. I timed how long the query took to complete and repeated the test 1000 time before incrementing the number of registry records. I used a new random query each time to try and avoid using a query which was less costly for the database to run (like say the first or last record every time).

I then started a two registries, the first was left empty and I used the same mechanism from the first test to populate the second registry. The test client was connected to the empty registry. The registries automatically discovered and communicated with each other. I then ran exactly the same client as the first test, the empty registry routed every query to the second registry.

By plotting these results together you can first see how well the registry store performs with simple serial queries with an increasing number of records in the registry. You can see the results from the same test with all the queries routed via a second registry, by comparing the two columns you can see the overhead of the P2P communication in this simple setup.

		Time msecs        	Time msecs
Records      	1 registry		2 registries

1000    	4.0772			6.2258
2000    	4.2952			6.3022
3000		4.8090			6.4516
4000		6.1277			7.6849
5000		7.8638			8.5851
10000		8.2883			10.281
15000		12.550			14.198
20000		18.941			20.587
25000		26.945			28.560
30000		37.017			40.263

As you can see the difference between the two columns is around 2 milliseconds.

This is the start of a large number of tests, a lot of my time was spent doing the plumbing to automate this kind of thing. Probably next on my list is to do this test using the IRC P2P code.