Ladies and gentlemen: Lord Voldemort

After the performance tests I ran with Tephlon, I became really curious about how the “big guys” of NoSQL are actually performing.

Then, my browser arrived at Vineet Gupta’s article (extremely informative), and I got interested in project voldemort: a key-value store provider written in Java by Linkedin engineers, on the inspiration of Amazon Dynamo.

Tephlon is not aimed to support cluster storage, and it has a synchronous approach to storage. It’s written in PHP and, in a typical setup, the same PHP process who serves the page needs to get the data stored.

So there’s an ocean of different prerequisites between Tephlon and Voldemort, even if evolving from the one to the other should pretty easy for a web application, since the aim is the same (providing persistent key-value stores).

Disregard that, I was too curious to set up some little basic performance tests.

Let’s say a bit more of project-voldemort. This distributed key-value store  is really powerful. And, as the name hints, there’s a lot of magic it’s capable of:

• Gossip based state propagation
• Partitioning and replication
• cluster rebalancing
• versioning (vector clocks)
• pluggable storage engines and serializers
• pluggable nio socket implementation

Setting it up in an Eclipse environment is as easy as installing GIT plugin from standard upload sites which come with latest Eclipse IDE, cloning their git repository, and execute the build.xml ant script.

Then open a terminal, navigate through your Eclipse workspace directories and find voldemort project directory. It contains a bin/ folder. Now execute the following.

 bin/voldemort-server.sh config/single_node_cluster &

Next step is to open voldemort.examples.ClientExample class (fast class search in Eclipse: ctrl+T), right click on it and execute as java application.

What I did next is writing a bit more code in ClientExample.java, to see better how fast this thing is.

public class ClientExample {

    public static void main(String[] args) {

        // In real life this stuff would get wired in
        String bootstrapUrl = "tcp://localhost:6666";
        StoreClientFactory factory =
             new SocketStoreClientFactory(new ClientConfig().setBootstrapUrls(bootstrapUrl));
        StoreClient<String, String> client = factory.getStoreClient("test");

        Versioned<String> value = client.get("key");

        long start;
        int testSize = 1000000;
        ArrayList<Long> opTimes = new ArrayList<Long>();
        for(int i = 1; i < testSize; i++) {
            start = System.currentTimeMillis();
            value = client.get("key" + i);
            if(value == null) {
                value = new Versioned<String>("x");
            }
            value.setObject("val" + i);
            client.put("key" + i, value);
            opTimes.add(System.currentTimeMillis() - start);
        }

        float sum = 0, avg = 0, min = Float.MAX_VALUE, max = Float.MIN_VALUE;
        for(Long optime: opTimes) {
            sum += optime;
            if(optime > max) {
                max = optime;
            }
            if(optime < min) {
                min = optime;
            }
        }
        avg = sum / opTimes.size();

        System.out.println("RW (GET & SET " + testSize + " entries)\n"
                           + "total time (s)\t: " + sum / 1000 + "\n" +
                           "avg optime (s):\t" + avg / 1000 + "\n"
                           + "longest optime (s):\t" + max / 1000 + "\n" +
                           "shortest optime (s):\t" + min / 1000);
    }

So here are some results on my little laptop (2.0GHz core 2 duo, 2GB memory):

RW (GET & SET 1000000 entries)
total time (s)	: 835.361
avg optime (s):	8.3536183E-4
longest optime (s):	0.214
shortest optime (s):	0.0

Quite amazing!