Pre-sharding is a common technique for preemptively managing the growth of your Redis infrastructure. At StepOut we’ve been running on a cluster of 64 pre-sharded Redis nodes for quite a while without any real pain to speak of. That said, in an effort to trim some of the fat from our systems resource allocation I started looking at some alternatives to our current setup. What I arrived at was a fairly different topology than we started with, consisting of only 8 shards on a single host box (replicated for performance and durability).
* Based on Rackspace cloud pricing
The only (significant) downside to the proposed configuration is that we would need to re-shard our data. Ugh. I considered upgrading to the Redis 2.6RC early in order to take advantage of the new dump and restore commands, but it seemed like too big of a gamble to introduce both pre-release software and a major infrastructural change. Luckily, the final solution was actually much more simple.
Thanks to the structure of the Redis AOF, you can concatenate individual backups into a single file.
It’s also pretty handy to run the
redis-check-aof tool to verify that the process was completed correctly.
Then in the application, we just had to make a slight change to our key hashing function:
The approach requires some downtime, but it ended up being fairly minimal, especially when considering some of the alternative solutions. Anecdotally, I’ve also noticed some performance improvements dealing with fewer instances, although it’s still a little early to call that conclusively.
Discussion on Hacker News.