Setup Redis Cluster on Amazon EC2 Linux

Redis, a powerful in-memory data store, is a cornerstone of modern applications.

Redis Cluster is a distributed architecture that partitions your data across multiple Redis nodes. This sharding technique not only enhances performance and scalability but also improves fault tolerance. By distributing the workload, Redis Cluster can handle significantly higher traffic volumes and larger datasets than a single instance.

Architecture

3 Ec2 instances, each running 2 redis servers on each machine.

Steps to setup Cluster.

1. Install Redis on your ec2 machine.
2. Create 2 folders

mkdir 7000
mkdir 7001

3. Setup port:7000 server

Create a config File 7000.conf

port 7000
bind 0.0.0.0
protected-mode no

cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-node-timeout 5000

appendonly yes
appendfilename "appendonly-7000.aof"

logfile "/var/log/redis-7000.log"
daemonize yes
dir /home/ec2-user/7000
pidfile /var/run/redis-7000.pid

4. Setup port:7001 server

port 7001
bind 0.0.0.0
protected-mode no

cluster-enabled yes
cluster-config-file nodes-7001.conf
cluster-node-timeout 5000

appendonly yes
appendfilename "appendonly-7001.aof"

logfile "/var/log/redis-7001.log"
daemonize yes
pidfile /var/run/redis-7001.pid
dir /home/ec2-user/7001

5. Start Redis server

redis-stable/src/redis-server 7000/7000.conf
redis-stable/src/redis-server 7001/7001.conf

6. Create the Cluster

Use the redis-cli --cluster create command to define all 6 nodes and specify replication relationships. Replace <IP1>, <IP2>, <IP3> with the private IPs of your EC2 machines.

redis-cli --cluster create \
<IP1>:7000 <IP2>:7000 <IP3>:7000 \   # Masters
<IP1>:7001 <IP2>:7001 <IP3>:7001 \   # Replicas
--cluster-replicas 1

7. Verify the Cluster

redis-cli -h <IP1> -p 7000 CLUSTER NODES

Result

3e77841d8b778a1834418c4697b61588e3ee841b server1:7001@17001 slave a331e6a75368d90ac25162bdc31ab90a68f6a1bc 0 1736761582000 3 connected
a73b1f94501d9faeb2c99011f7dcf4f7f5b0f229 server1:7000@17000 master - 0 1736761582551 5 connected 10923-16383
8813c422ac4ff9725b2d61b18c9d42a176ed7946 server2:7001@17001 slave a73b1f94501d9faeb2c99011f7dcf4f7f5b0f229 0 1736761583152 5 connected
a331e6a75368d90ac25162bdc31ab90a68f6a1bc server3:7000@17000 master - 0 1736761582149 3 connected 5461-10922
85b7594e955ffce4cfab100c7d690eafd1bddef4 server3:7001@17001 slave 109c04c63f20710ede1e78c320508b1e4dc61ef0 0 1736761582000 1 connected
109c04c63f20710ede1e78c320508b1e4dc61ef0 server2:7000@17000 myself,master - 0 0 1 connected 0-5460

Explanation of the Result

<Node-ID> <Address> <Role> <Replication Info> <Ping-Sent> <Pong-Received> <Config-Epoch> <Link-State> <Slots>

Masters:

a73b1f94501d9faeb2c99011f7dcf4f7f5b0f229 → Slots: 10923-16383
a331e6a75368d90ac25162bdc31ab90a68f6a1bc → Slots: 5461-10922
109c04c63f20710ede1e78c320508b1e4dc61ef0 → Slots: 0-5460

Slaves:

3e77841d8b778a1834418c4697b61588e3ee841b → Replicates: a331e6a75368d90ac25162bdc31ab90a68f6a1bc
8813c422ac4ff9725b2d61b18c9d42a176ed7946 → Replicates: a73b1f94501d9faeb2c99011f7dcf4f7f5b0f229
85b7594e955ffce4cfab100c7d690eafd1bddef4 → Replicates: 109c04c63f20710ede1e78c320508b1e4dc61ef0

Takeaways:

• Redis Cluster is essential for scaling Redis deployments to handle growing data volumes and traffic.
• Sharding distributes data across multiple nodes, improving performance and reducing latency.
• Fault tolerance is enhanced through automatic failover mechanisms.
• Effective cluster management is crucial for optimal performance and scalability.

To learn how to setup simple redis click here