Initial Tycho registry benchmarks
Spent Wednesday day and night getting an initial benchmark done, in this post I explain my methodology and publish some results.
- Time to complete a simple query while increasing the number of records in the registry.
- The overhead of the p2p registry link.
These tests were done a single 3ghz machine at home. Tycho modules I used were the HSQLDB data store with SQL as the query language, LDIF as the response markup and the simple HTTP P2P registry connector. Where I say registry I mean a registry within a mediator.
I populated a single registry with N unique records of the simplest form Tycho lets you do:
('producer','socket://192.168.0.16:NUMBER/producer/NAME',
'NAMENUMBER','','')
The records were made unique by adding an incrementing number to the end of the name field.
Using a random number between 1 and the number of records in the registry I could create a query which I knew would match a single record. I timed how long the query took to complete and repeated the test 1000 time before incrementing the number of registry records. I used a new random query each time to try and avoid using a query which was less costly for the database to run (like say the first or last record every time).
I then started a two registries, the first was left empty and I used the same mechanism from the first test to populate the second registry. The test client was connected to the empty registry. The registries automatically discovered and communicated with each other. I then ran exactly the same client as the first test, the empty registry routed every query to the second registry.
By plotting these results together you can first see how well the registry store performs with simple serial queries with an increasing number of records in the registry. You can see the results from the same test with all the queries routed via a second registry, by comparing the two columns you can see the overhead of the P2P communication in this simple setup.
Time msecs Time msecs Records 1 registry 2 registries 1000 4.0772 6.2258 2000 4.2952 6.3022 3000 4.8090 6.4516 4000 6.1277 7.6849 5000 7.8638 8.5851 10000 8.2883 10.281 15000 12.550 14.198 20000 18.941 20.587 25000 26.945 28.560 30000 37.017 40.263
As you can see the difference between the two columns is around 2 milliseconds.
This is the start of a large number of tests, a lot of my time was spent doing the plumbing to automate this kind of thing. Probably next on my list is to do this test using the IRC P2P code.