Sunday, April 28, 2013

Performance check twemproxy and redis

Hi friends today m sharing with you Performance check  twemproxy and redis

Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. Redis in itself is  a great data store supporting a variety of datastructure and highly rich feature base. Redis is a event based system running single thread giving throughput  as high as 500k on localhost.

twemproxy (pronounced "two-em-proxy"), aka nutcracker is a fast and lightweight proxy for memcached and redis protocol. It was primarily built to reduce the connection count on the backend caching servers. This is also an event based system running single thread giving a high throughput.
With twemproxy it is also possible to shard data over multiple redis instances.

We performed  various testes with twemproxy and redis  to find out the drop in through by applying twemproxy before redis. Since both proxy and redis runs in single thread, to optimize on cpu resources it is also possible to pinn any of these process to a single core.
following are the test result performed on our laptops with intel i5  4 core porcessors.
Non-Pipelined Pinned Non-pipelined Non-Pinned Pipelined Pipelined and Pinned
Zero Hops - Direct Redis 120k 120k 450k 500k
Zero Hops - with twemproxy 75k 80k 370k 370k
One Hop - Client to (Twemproxy & redis) 63k 66k 108k 109k
One Hop -b/w Twemproxy & redis 88k 90k 117k 118k
One Hop - Client to direct Redis 64k 65k 119k 119k
Two hops (client redis and proxy on diff machine) 45k 46k 180k 180k

 it is clear that redis alone can give a QPS of 500k when both client and server are running on same host. This performance drops by ~20% but still very high. It drops as number of network hops increases for obvious reasons.