To start YCSB load with cluster enabled option for

Question

I am Performing YCSB bench marking on Redis cluster. I have created redis cluster and its working with the following condition.

If I specify to enable cluster mode in redis client with -c parameter. The chunks are moved correctly.

./redis-cli -h -c "host ip" -p "port"

if I dont specify -c parameter, it moved the chunk with error

./redis-cli -h "host ip" -p "port"

SO in YCSB load option, I don't know how to enable the cluster option ( -c parameter).

Currently I am using the following conmmand without -c option

./bin/ycsb load redis -s -P workloads/workloada -p "redis.host=host ip" -p "redis.port=port" > outputLoad.txt

It returns an error. Can you help me to resolve the issue?

Answer 1

In YCSB/redis/src/main/java/com/yahoo/ycsb/db/RedisClient.java and inside

public void init() throws DBException

add the following lines

Set<HostAndPort> jedisClusterNodes = new HashSet<HostAndPort>();
//Jedis Cluster will attempt to discover cluster nodes automatically
jedisClusterNodes.add(new HostAndPort(host, port));
jedis = new JedisCluster(jedisClusterNodes);

Comment out

jedis.connect(); 

and

jedis.disconnect();

Also, dont forget to import the following in RedisClient.java

import redis.clients.jedis.Protocol;
import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.JedisCluster;
import java.util.HashSet;

Run the test with the same command as you run test for redis, it should work.

Answer 2

Don't know if you figured this problem or not.

But basically if you want to use YCSB on Redis Cluster, you need to implement a Redis Cluster Client for YCSB. It's very similar to the client that YCSB has for Redis. You can copy and paste the Redis (single instance) and change the code in the init(). Configure and compile with maven then you can run the YCSB through Redis Cluster Client.

More detail:

Because YCSB using java, so we need to use jedis. There is a thing in jedis called JedisCluster, we need to use that in order to make connection between YCSB and Redis Cluster. More details can be found on jedis github.

For the maven part, remember to change the pom.xml in your client code folder, and also add your client side module in your YCSB root folder (also pom.xml, under nosql).

---

Things worth mentioning :

I did experiment for Redis Cluster using YCSB. The setting is standard, 4 nodes without replicas, 8 cores machine, 8 GB mem, running YCSB from another machine with the same set up with 6 threads for workers.

The Redis Cluster is 3x - 4x slower than Redis Single Instance in all aspects. (Mainly for the threshold comparison, where throughput cannot be bumped higher by adjusting the -target parameter)

I don't really know whether it's my problem or it's the problem of jedis/jedisCluster/YCSB/Redis Cluster/etc.

If you successfully run the YCSB on Redis Cluster, please tell me the result. I will be super curious on your result.

Thanks