CLUSTER SETSLOT is responsible of changing the state of an hash slot in the receiving node in different ways. It can, depending on the subcommand used:
MIGRATINGsubcommand: Set a hash slot in migrating state.
IMPORTINGsubcommand: Set a hash slot in importing state.
STABLEsubcommand: Clear any importing / migrating state from hash slot.
NODEsubcommand: Bind the hash slot to a different node.
The command with its set of subcommands is useful in order to start and end cluster live resharding operations, which are accomplished by setting an hash slot in migrating state in the source node, and importing state in the destination node.
Each subcommand is documented below. At the end you’ll find a description of how live resharding is performed using this command and other related commands.
This subcommand sets a slot to migrating state. In order to set a slot in this state, the node receiving the command must be the hash slot owner, otherwise an error is returned.
When a slot is set in migrating state, the node changes behavior in the following way:
- If a command is received about an existing key, the command is processed as usually.
- If a command is received about a key that does not exists, an
ASKredirection is emitted by the node, asking the client to retry only that specific query into
destination-node. In this case the client should not update its hash slot to node mapping.
- If the command contains multiple keys, in case none exist, the behavior is the same as point 2, if all exist, it is the same as point 1, however if only a partial number of keys exist, the command emits a
TRYAGAINerror in order for the keys interested to finish being migrated to the target node, so that the multi keys command can be executed.
This subcommand is the reverse of
MIGRATING, and prepares the destination
node to import keys from the specified source node. The command only works if
the node is not already owner of the specified hash slot.
When a slot is set in importing state, the node changes behavior in the following way:
- Commands about this hash slot are refused and a
MOVEDredirection is generated as usually, but in the case the command follows an
ASKINGcommand, in this case the command is executed.
In this way when a node in migrating state generates an
ASK redirection, the client contacts the target node, sends
ASKING, and immediately after sends the command. This way commands about non-existing keys in the old node or keys already migrated to the target node are executed in the target node, so that:
- New keys are always created in the target node. During an hash slot migration we’ll have to move only old keys, not new ones.
- Commands about keys already migrated are correctly processed in the context of the node which is the target of the migration, the new hash slot owner, in order to guarantee consistency.
ASKINGthe behavior is the same as usually. This guarantees that clients with a broken hash slots mapping will not write for error in the target node, creating a new version of a key that has yet to be migrated.
This subcommand just clears migrating / importing state from the slot. It is
mainly used to fix a cluster stuck in a wrong state by
Normally the two states are cleared automatically at the end of the migration
SETSLOT ... NODE ... subcommand as explained in the next section.
NODE subcommand is the one with the most complex semantics. It
associates the hash slot with the specified node, however the command works
only in specific situations and has different side effects depending on the
slot state. The following is the set of pre-conditions and side effects of the
- If the current hash slot owner is the node receiving the command, but for effect of the command the slot would be assigned to a different node, the command will return an error if there are still keys for that hash slot in the node receiving the command.
- If the slot is in migrating state, the state gets cleared when the slot is assigned to another node.
- If the slot was in importing state in the node receiving the command, and the command assigns the slot to this node (which happens in the target node at the end of the resharding of an hash slot from one node to another), the command has the following side effects: A) the importing state is cleared. B) If the node config epoch is not already the greatest of the cluster, it generates a new one and assigns the new config epoch to itself. This way its new hash slot ownership will win over any past configuration created by previous failovers or slot migrations.
It is important to note that step 3 is the only time when a Redis Cluster node will create a new config epoch without agreement from other nodes. This only happens when a manual configuration is operated. However it is impossible that this creates a non-transient setup where two nodes have the same config epoch, since Redis Cluster uses a config epoch collision resolution algorithm.
@simple-string-reply: All the subcommands return
OK if the command was successful. Otherwise an error is returned.
Redis Cluster live resharding explained
CLUSTER SETSLOT command is an important piece used by Redis Cluster in order to migrate all the keys contained in one hash slot from one node to another. This is how the migration is orchestrated, with the help of other commands as well. We’ll call the node that has the current ownership of the hash slot the
source node, and the node where we want to migrate the
- Set the destination node slot to importing state using
CLUSTER SETSLOT <slot> IMPORTING <source-node-id>.
- Set the source node slot to migrating state using
CLUSTER SETSLOT <slot> MIGRATING <destination-node-id>.
- Get keys from the source node with
CLUSTER GETKEYSINSLOTcommand and move them into the destination node using the
CLUSTER SETSLOT <slot> NODE <destination-node-id>in the source or destination.
- The order of step 1 and 2 is important. We want the destination node to be ready to accept
ASKredirections when the source node is configured to redirect.
- Step 4 does not technically need to use
SETSLOTin the nodes not involved in the resharding, since the configuration will eventually propagate itself, however it is a good idea to do so in order to stop nodes from pointing to the wrong node for the hash slot moved as soon as possible, resulting in less redirections to find the right node.