Compare commits

...
38 Commits
Author SHA1 Message Date
Oran Agra 2c9af0f00d Redis 7.2.4 2024-01-09 13:51:49 +02:00
BinbinandOran Agra 122e375eeb Fix CLUSTER SHARDS crash in 7.0/7.2 mixed clusters where shard ids are not sync (#12832)
Crash reported in #12695. In the process of upgrading the cluster from
7.0 to 7.2, because the 7.0 nodes will not gossip shard id, in 7.2 we
will rely on shard id to build the server.cluster->shards dict.

In some cases, for example, the 7.0 master node and the 7.2 replica node.
From the view of 7.2 replica node, the cluster->shards dictionary does not
have its master node. In this case calling CLUSTER SHARDS on the 7.2 replica
node may crash.

We should fix the underlying assumption of updateShardId, which is that the
shard dict should be always in sync with the node's shard_id. The fix was
suggested by PingXie, see more details in #12695.

(cherry picked from commit 77f48cd894)
2024-01-09 13:51:49 +02:00
BinbinandOran Agra d70a9d1653 Use shard-id of the master if the replica does not support shard-id (#12805)
If there are nodes in the cluster that do not support shard-id, they
will gossip shard-id. From the perspective of nodes that support shard-id,
their shard-id is meaningless (since shard-id is randomly generated when
we create a node.)

Nodes that support shard-id will save the shard-id information in nodes.conf.
If the node is restarted according to nodes.conf, the server will report a
corrupted cluster config file error. Because auxShardIdSetter will reject
configurations with inconsistent master-replica shard-ids.

A cluster-wide consensus for the node's shard_id is not necessary. The key
is maintaining consistency of the shard_id on each individual 7.2 node.
As the cluster progressively upgrades to version 7.2, we can expect the
shard_ids across all nodes to naturally converge and align.

In this PR, when processing the gossip, if sender is a replica and does not
support shard-id, set the shard_id to the shard_id of its master.

(cherry picked from commit e4811e61d6)
2024-01-09 13:51:49 +02:00
BinbinandOran Agra e52f3bfdd1 Un-register notification and server event when RedisModule_OnLoad fails (#12809)
When we register notification or server event in RedisModule_OnLoad, but
RedisModule_OnLoad eventually fails, triggering notification or server
event
will cause the server to crash.

If the loading fails on a later stage of moduleLoad, we do call
moduleUnload
which handles all un-registration, but when it fails on the
RedisModule_OnLoad
call, we only un-register several specific things and these were
missing:

- moduleUnsubscribeNotifications
- moduleUnregisterFilters
- moduleUnsubscribeAllServerEvents

Refactored the code to reuse the code from moduleUnload.

Fixes #12808.

(cherry picked from commit 4b0f42817a)
2024-01-09 13:51:49 +02:00
Meir Shpilraien (Spielrein)andOran Agra 94b5ec459a Before evicted and before expired server events are not executed inside an execution unit. (#12733)
Redis 7.2 (#9406) introduced a new modules event, `RedisModuleEvent_Key`.
This new event allows the module to read the key data just before it is removed
from the database (either deleted, expired, evicted, or overwritten).

When the key is removed from the database, either by active expire or eviction.
The new event was not called as part of an execution unit. This can cause an
issue if the module registers a post notification job inside the event. This job will
not be executed atomically with the expiration/eviction operation and will not
replicated inside a Multi/Exec. Moreover, the post notification job will be executed
right after the event where it is still not safe to perform any write operation, this will
violate the promise that post notification job will be called atomically with the
operation that triggered it and **only when it is safe to write**.

This PR fixes the issue by wrapping each expiration/eviction of a key with an execution
unit. This makes sure the entire operation will run atomically and all the post notification
jobs will be executed at the end where it is safe to write.

Tests were modified to verify the fix.

(cherry picked from commit fad56faead)
2024-01-09 13:51:49 +02:00
SankarandOran Agra 802a70e3ad Clear owner_not_claiming_slot bit for the slot in clusterDelSlot (#12564)
Clear owner_not_claiming_slot bit for the slot in clusterDelSlot to keep it
consistent with slot ownership information.

(cherry picked from commit 11ded735b3)
2024-01-09 13:51:49 +02:00
BinbinandOran Agra 22c9e05439 Use CLZ in _dictNextExp to get the next power of two (#12815)
In the past, we did not call _dictNextExp frequently. It was only
called when the dictionary was expanded.

Later, dictTypeExpandAllowed was introduced in #7954, which is 6.2.
For the data dict and the expire dict, we can check maxmemory before
actually expanding the dict. This is a good optimization to avoid
maxmemory being exceeded due to the dict expansion.

And in #11692, we moved the dictTypeExpandAllowed check before the
threshold check, this caused a bit of performance degradation, every
time a key is added to the dict, dictTypeExpandAllowed is called to
check.

The main reason for degradation is that in a large dict, we need to
call _dictNextExp frequently, that is, every time we add a key, we
need to call _dictNextExp once. Then the threshold is checked to see
if the dict needs to be expanded. We can see that the order of checks
here can be optimized.

So we moved the dictTypeExpandAllowed check back to after the threshold
check in #12789. In this way, before the dict is actually expanded (that
is, before the threshold is reached), we will not do anything extra
compared to before, that is, we will not call _dictNextExp frequently.

But note we'll still hit the degradation when we over the thresholds.
When the threshold is reached, because #7954, we may delay the dict
expansion due to maxmemory limitations. In this case, we will call
_dictNextExp every time we add a key during this period.

This PR use CLZ in _dictNextExp to get the next power of two. CLZ (count
leading zeros) can easily give you the next power of two. It should be
noted that we have actually introduced the use of __builtin_clzl in
#8687,
which is 7.0. So i suppose all the platforms we use have it (even if the
CPU doesn't have an instruction).

We build 67108864 (2**26) keys through DEBUG POPULTE, which will use
approximately 5.49G memory (used_memory:5898522936). If expansion is
triggered, the additional hash table will consume approximately 1G
memory (2 ** 27 * 8). So we set maxmemory to 6871947673 (that is, 6.4G),
which will be less than 5.49G + 1G, so we will delay the dict rehash
while addint the keys.

After that, each time an element is added to the dict, an allow check
will be performed, that is, we can frequently call _dictNextExp to test
the comparison before and after the optimization. Using DEBUG HTSTATS 0
to
check and make sure that our dict expansion is dealyed.

Using `./src/redis-server redis.conf --save "" --maxmemory 6871947673`.
Using `./src/redis-benchmark -P 100 -r 1000000000 -t set -n 5000000`.
After ten rounds of testing:
```
unstable:           this PR:
769585.94           816860.00
771724.00           818196.69
775674.81           822368.44
781983.12           822503.69
783576.25           828088.75
784190.75           828637.75
791389.69           829875.50
794659.94           835660.69
798212.00           830013.25
801153.62           833934.56
```

We can see there is about 4-5% performance improvement in this case.

(cherry picked from commit a0059b3607)
2024-01-09 13:51:49 +02:00
BinbinandOran Agra 44eff3aad4 Optimize dict expand check, move allow check after the thresholds check (#12789)
dictExpandAllowed (for the main db dict and the expire dict) seems to
involve a few function calls and memory accesses, and we can do it only
after the thresholds checks and can get some performance improvements.

A simple benchmark test: there are 11032768 fixed keys in the database,
start a redis-server with `--maxmemory big_number --save ""`,
start a redis-benchmark with `-P 100 -r 1000000000 -t set -n 5000000`,
collect `throughput summary: n requests per second` result.

After five rounds of testing:
```
unstable     this PR
848032.56    897988.56
854408.69    913408.88
858663.94    914076.81
871839.56    916758.31
882612.56    920640.75
```

We can see a 5% performance improvement in general condition.
But note we'll still hit the degradation when we over the thresholds.

(cherry picked from commit f3720f74bd)
2024-01-09 13:51:49 +02:00
Oran Agra 8e5683fb1c Fix possible corruption in sdsResize (CVE-2023-41056)
#11766 introduced a bug in sdsResize where it could forget to update
the sds type in the sds header and then cause an overflow in sdsalloc.
it looks like the only implication of that is a possible assertion in HLL,
but it's hard to rule out possible heap corruption issues with clientsCronResizeQueryBuffer
2024-01-09 13:51:49 +02:00
Oran Agra bb9130d9c6 Redis 7.2.3 2023-11-01 14:38:13 +02:00
Viktor SöderqvistandOran Agra 9d1150494b Don't crash when adding a forgotten node to blacklist twice (#12702)
Add a defensive checks to prevent double freeing a node from the cluster blacklist.

(cherry picked from commit 4453eabb8e)
2023-11-01 14:38:13 +02:00
Oran Agra 19d6702358 Fix fd leak causing deleted files to remain open and eat disk space (#12693)
This was introduced in v7.2 by #11248

(cherry picked from commit 4d2ac760c3)
2023-11-01 14:38:13 +02:00
Oran Agra 629b757cc4 Redis 7.2.2 2023-10-18 10:44:10 +03:00
BinbinandOran Agra 708a7e9ea1 Fix crash when running rebalance command in a mixed cluster of 7.0 and 7.2 (#12604)
In #10536, we introduced the assert, some older versions of servers
(like 7.0) doesn't gossip shard_id, so we will not add the node to
cluster->shards, and node->shard_id is filled in randomly and may not
be found here.

It causes that if we add a 7.2 node to a 7.0 cluster and allocate slots
to the 7.2 node, the 7.2 node will crash when it hits this assert. Somehow
like #12538.

In this PR, we remove the assert and replace it with an unconditional removal.

(cherry picked from commit 195e6a362d)
2023-10-18 10:44:10 +03:00
BinbinandOran Agra 67bb61fa5c Fix redis-cli pubsub_mode and connect minor prompt / crash issue (#12571)
When entering pubsub mode and using the redis-cli only
connect command, we need to reset pubsub_mode because
we switch to a different connection.

This will affect the prompt when the connection is successful,
and redis-cli will crash when the connect fails:
```
127.0.0.1:6379> subscribe ch
1) "subscribe"
2) "ch"
3) (integer) 1
127.0.0.1:6379(subscribed mode)> connect 127.0.0.1 6380
127.0.0.1:6380(subscribed mode)> ping
PONG
127.0.0.1:6380(subscribed mode)> connect a b
Could not connect to Redis at a:0: Name or service not known
Segmentation fault
```

(cherry picked from commit 91925792d6)
2023-10-18 10:44:10 +03:00
guybe7andOran Agra 93fee40390 WAITAOF: Update fsynced_reploff_pending just before starting the initial AOFRW fork (#12620)
If we set `fsynced_reploff_pending` in `startAppendOnly`, and the fork doesn't start
immediately (e.g. there's another fork active at the time), any subsequent commands
will increment `server.master_repl_offset`, but will not cause a fsync (given they were
executed before the fork started, they just ended up in the RDB part of it)
Therefore, any WAITAOF will wait on the new master_repl_offset, but it will time out
because no fsync will be executed.

Release notes:
```
WAITAOF could timeout in the absence of write traffic in case a new AOF is created and
an AOFRW can't immediately start.
This can happen by the appendonly config is changed at runtime, but also after FLUSHALL,
and replica full sync.
```

(cherry picked from commit d967ed0e08)
2023-10-18 10:44:10 +03:00
Nir RattnerandOran Agra 64aa4537ba Fix overflow calculation for next timer event (#12474)
The `retval` variable is defined as an `int`, so with 4 bytes, it cannot properly represent
microsecond values greater than the equivalent of about 35 minutes.

This bug shouldn't impact standard Redis behavior because Redis doesn't have timer
events that are scheduled as far as 35 minutes out, but it may affect custom Redis modules
which interact with the event timers via the RM_CreateTimer API.

The impact is that `usUntilEarliestTimer` may return 0 for as long as `retval` is scaled to
an overflowing value. While `usUntilEarliestTimer` continues to return `0`, `aeApiPoll`
will have a zero timeout, and so Redis will use significantly more CPU iterating through
its event loop without pause. For timers scheduled far enough into the future, Redis will
cycle between ~35 minute periods of high CPU usage and ~35 minute periods of standard
CPU usage.

(cherry picked from commit d16dd42764)
2023-10-18 10:44:10 +03:00
Chen TianjieandOran Agra 5bcce7a39d Use server.current_client to decide whether cluster commands should return TLS info. (#12569)
Starting a change in #12233 (released in 7.2), CLUSTER commands use client's
connection to decide whether to return TLS port or non-TLS port, but commands
called by Lua script and module's RM_Call don't have a real client with connection,
and would currently be regarded as non-TLS connections.

We can use server.current_client instead when it is available. When it is not (module calls
commands without a real client), we may see this as an undefined behavior, and return null
or default port (currently in this PR it returns default port, judged by server.tls_cluster).

(cherry picked from commit 115ed5e156)
2023-10-18 10:44:10 +03:00
BinbinandOran Agra 20b20d404c Fix that slot return in CLUSTER SHARDS should be integer (#12561)
An unintentional change was introduced in #10536, we used
to use addReplyLongLong and now it is addReplyBulkLonglong,
revert it back the previous behavior.

(cherry picked from commit 2a958dc37b)
2023-10-18 10:44:10 +03:00
JachinandOran Agra 139d93266d Fix compile on macOS 13 (#12611)
Use the __MAC_OS_X_VERSION_MIN_REQUIRED macro to detect the
macOS system version instead of using MAC_OS_X_VERSION_10_6.

From MacOSX14.0.sdk, the default definitions of MAC_OS_X_VERSION_xxx have
been removed in usr/include/AvailabilityMacros.h. It includes AvailabilityVersions.h,
where the following condition must be met:
`#if (!defined(_POSIX_C_SOURCE) && !defined(_XOPEN_SOURCE)) || defined(_DARWIN_C_SOURCE)`
Only then will MAC_OS_X_VERSION_xxx be defined.
However, in the project, _DARWIN_C_SOURCE is not defined, which leads to the
loss of the definition for MAC_OS_X_VERSION_10_6.

(cherry picked from commit 0267fbb0a3)
2023-10-18 10:44:10 +03:00
BinbinandOran Agra 4d78a6061f Support NO ONE block in REPLICAOF command json (#12633)
The current commands.json doesn't mention the special NO ONE arguments.
This change is also applied to SLAVEOF

(cherry picked from commit f6d839fde2)
2023-10-18 10:44:10 +03:00
Yossi GottliebandOran Agra c2ebbd0668 Fix issue of listen before chmod on Unix sockets (CVE-2023-45145)
Before this commit, Unix socket setup performed chmod(2) on the socket
file after calling listen(2). Depending on what umask is used, this
could leave the file with the wrong permissions for a short period of
time. As a result, another process could exploit this race condition and
establish a connection that would otherwise not be possible.

We now make sure the socket permissions are set up prior to calling
listen(2).
2023-10-18 10:44:10 +03:00
Oran Agra 68d6dffe50 Redis 7.2.1 2023-09-06 20:56:15 +03:00
nihohitandOran Agra 22087fc2b4 Update command tips on more admin / configuration commands (#12545)
Updated the command tips for ACL SAVE / SETUSER / DELUSER, CLIENT SETNAME / SETINFO, and LATENCY RESET.
The tips now match CONFIG SET, since there's a similar behavior for all of these commands - the
user expects to update the various configurations & states on all nodes, not only on a single, random node.
For LATENCY RESET the response tip is now agg_sum.

Co-authored-by: Shachar Langbeheim <shachlan@amazon.com>
(cherry picked from commit 1c2c30ef03)
2023-09-06 20:56:15 +03:00
secwallandOran Agra b3d6f4bbf2 Check shard_id pointer validity in updateShardId (#12538)
When connecting between a 7.0 and 7.2 cluster, the 7.0 cluster will not populate the shard_id field, which is expect on the 7.2 cluster. This is not intended behavior, as the 7.2 cluster is supposed to use a temporary shard_id while the node is in the upgrading state, but it wasn't being correctly set in this case.

(cherry picked from commit 0a680287a6)
2023-09-06 20:56:15 +03:00
BinbinandOran Agra d5f5378d47 Update sort_ro reply_schema to mention the null reply (#12534)
Also added a test to cover this case, so this can
cover the reply schemas check.

(cherry picked from commit 2550dcf968)
2023-09-06 20:56:15 +03:00
bodong.ybdandOran Agra 93784e2ee3 Fix sort_ro get-keys function return wrong key number (#12522)
Before:
```
127.0.0.1:6379> command getkeys sort_ro key
(empty array)
127.0.0.1:6379>
```
After:
```
127.0.0.1:6379> command getkeys sort_ro key
1) "key"
127.0.0.1:6379>
```

(cherry picked from commit 7d43d7604e)
2023-09-06 20:56:15 +03:00
nihohitandOran Agra 41275dd3d3 Align CONFIG RESETSTAT/REWRITE tips with SET. (#12530)
Since the three commands have similar behavior (change config, return
OK), the tips that govern how they should behave should be similar.

Co-authored-by: Shachar Langbeheim <shachlan@amazon.com>
(cherry picked from commit 056498406c)
2023-09-06 20:56:15 +03:00
Oran AgraandGitHub 42d7d976ed Release Redis 7.2.0 GA 2023-08-15 12:38:36 +03:00
Oran Agra 606f3502c9 Redis 7.2.0 2023-08-15 10:04:17 +03:00
Oran Agra 146644355e Merge branch 'unstable' into 7.2 to prepare for 7.2.0 GA 2023-08-15 09:43:09 +03:00
Oran AgraandGitHub ff1c49d07b Release Redis 7.2 RC3 2023-07-10 14:55:20 +03:00
Oran Agra cd05870817 Redis 7.2 RC3 2023-07-10 11:52:53 +03:00
Oran Agra 1bc735bf30 Merge remote-tracking branch 'origin/unstable' into 7.2 2023-07-10 10:26:53 +03:00
Oran AgraandGitHub b1fe900842 Release Redis 7.2 RC2 2023-05-15 13:08:15 +03:00
Oran Agra df68e754a7 Redis 7.2 RC2 2023-05-15 09:53:36 +03:00
Oran Agra f4ba22d9b6 Merge remote-tracking branch 'origin/unstable' into 7.2 2023-05-15 09:50:01 +03:00
Oran Agra 45f46a1a5a Redis 7.2 RC1 2023-03-22 15:58:15 +02:00
42 changed files with 1047 additions and 153 deletions
+505 -11
View File
@@ -1,16 +1,510 @@
Hello! This file is just a placeholder, since this is the "unstable" branch
of Redis, the place where all the development happens.
Redis 7.2 release notes
=======================
There is no release notes for this branch, it gets forked into another branch
every time there is a partial feature freeze in order to eventually create
a new stable release.
--------------------------------------------------------------------------------
Upgrade urgency levels:
Usually "unstable" is stable enough for you to use it in development environments
however you should never use it in production environments. It is possible
to download the latest stable release here:
LOW: No need to upgrade unless there are new features you want to use.
MODERATE: Program an upgrade of the server, but it's not urgent.
HIGH: There is a critical bug that may affect a subset of users. Upgrade!
CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
SECURITY: There are security fixes in the release.
--------------------------------------------------------------------------------
https://download.redis.io/redis-stable.tar.gz
More information is available at https://redis.io
================================================================================
Redis 7.2.4 Released Tue 09 Jan 2024 10:45:52 IST
================================================================================
Upgrade urgency SECURITY: See security fixes below.
Security fixes
==============
* (CVE-2023-41056) In some cases, Redis may incorrectly handle resizing of memory
buffers which can result in incorrect accounting of buffer sizes and lead to
heap overflow and potential remote code execution.
Bug fixes
=========
* Fix crashes of cluster commands clusters with mixed versions of 7.0 and 7.2 (#12805, #12832)
* Fix slot ownership not being properly handled when deleting a slot from a node (#12564)
* Fix atomicity issues with the RedisModuleEvent_Key module API event (#12733)
================================================================================
Redis 7.2.3 Released Wed 01 Nov 2023 12:00:00 IST
================================================================================
Upgrade urgency: HIGH, Fixes critical bugs affecting most users.
Bug fixes
=========
* Fix file descriptor leak preventing deleted files from freeing disk space on
replicas (#12693)
* Fix a possible crash after cluster node removal (#12702)
================================================================================
Redis 7.2.2 Released Wed 18 Oct 2023 10:33:40 IDT
================================================================================
Upgrade urgency SECURITY: See security fixes below.
Security fixes
==============
* (CVE-2023-45145) The wrong order of listen(2) and chmod(2) calls creates a
race condition that can be used by another process to bypass desired Unix
socket permissions on startup.
Platform / toolchain support related changes
=================================================
* Fix compilation error on MacOS 13 (#12611)
Bug fixes
=========
* WAITAOF could timeout in the absence of write traffic in case a new AOF is
created and an AOF rewrite can't immediately start (#12620)
Redis cluster
=============
* Fix crash when running rebalance command in a mixed cluster of 7.0 and 7.2
nodes (#12604)
* Fix the return type of the slot number in cluster shards to integer, which
makes it consistent with past behavior (#12561)
* Fix CLUSTER commands are called from modules or scripts to return TLS info
appropriately (#12569)
Changes in CLI tools
====================
* redis-cli, fix crash on reconnect when in SUBSCRIBE mode (#12571)
Module API changes
==================
* Fix overflow calculation for next timer event (#12474)
================================================================================
Redis 7.2.1 Released Wed 06 Sep 2023 15:00:00 IDT
================================================================================
Upgrade urgency SECURITY: See security fixes below.
Security Fixes
==============
* (CVE-2023-41053) Redis does not correctly identify keys accessed by SORT_RO and,
as a result, may grant users executing this command access to keys that are not
explicitly authorized by the ACL configuration.
Bug Fixes
=========
* Fix crashes when joining a node to an existing 7.0 Redis Cluster (#12538)
* Correct request_policy and response_policy command tips on for some admin /
configuration commands (#12545, #12530)
================================================================================
Redis 7.2.0 GA Released Tue Aug 15 12:00:00 IDT 2023
================================================================================
Upgrade urgency LOW: This is the first stable Release for Redis 7.2.
Bug Fixes
=========
* redis-cli in cluster mode handles `unknown-endpoint` (#12273)
* Update request / response policy hints for a few commands (#12417)
* Ensure that the function load timeout is disabled during loading from RDB/AOF and on replicas. (#12451)
* Fix false success and a memory leak for ACL selector with bad parenthesis combination (#12452)
* Fix the assertion when script timeout occurs after it signaled a blocked client (#12459)
Fixes for issues in previous releases of Redis 7.2
--------------------------------------------------
* Update MONITOR client's memory correctly for INFO and client-eviction (#12420)
* The response of cluster nodes was unnecessarily adding an extra comma when no
hostname was present. (#12411)
================================================================================
Redis 7.2 RC3 Released Mon July 10 12:00:00 IDT 2023
================================================================================
Upgrade urgency LOW: This is the third Release Candidate for Redis 7.2.
Upgrade urgency SECURITY: If you're using a previous release candidate of 7.2.
Security Fixes:
* (CVE-2022-24834) A specially crafted Lua script executing in Redis can trigger
a heap overflow in the cjson and cmsgpack libraries, and result in heap
corruption and potentially remote code execution. The problem exists in all
versions of Redis with Lua scripting support, starting from 2.6, and affects
only authenticated and authorized users.
* (CVE-2023-36824) Extracting key names from a command and a list of arguments
may, in some cases, trigger a heap overflow and result in reading random heap
memory, heap corruption and potentially remote code execution. Specifically:
using COMMAND GETKEYS* and validation of key names in ACL rules.
New Features
============
New administrative and introspection commands and command arguments
-------------------------------------------------------------------
* Make SENTINEL CONFIG [SET|GET] variadic. (#10362)
Potentially Breaking / Behavior Changes
=======================================
* Cluster SHARD IDs are no longer visible in the cluster nodes output,
introduced in 7.2-RC1. (#10536, #12166)
* When calling PUBLISH with a RESP3 client that's also subscribed to the same channel,
the order is changed and the reply is sent before the published message (#12326)
New configuration options
=========================
* Add a new loglevel "nothing" to disable logging (#12133)
* Add cluster-announce-human-nodename - a unique identifier for a node that is
be used in logs for debugging (#9564)
Other General Improvements
==========================
* Allow CLUSTER SLOTS / SHARDS commands during loading (#12269)
* Support TLS service when "tls-cluster" is not enabled and persist both plain
and TLS port in nodes.conf (#12233)
* Update SPOP and RESTORE commands to replicate unlink commands to replicas
when the server is configured to use async server deletes (#12320)
* Try lazyfree the temporary zset in ZUNION / ZINTER / ZDIFF (#12229)
Performance and resource utilization improvements
=================================================
* Optimize PSUBSCRIBE and PUNSUBSCRIBE from O(N*M) to O(N) (#12298)
* Optimize SCAN, SSCAN, HSCAN, ZSCAN commands (#12209)
* Set Jemalloc --disable-cache-oblivious to reduce memory overhead (#12315)
* Optimize ZINTERCARD to avoid create a temporary zset (#12229)
* Optimize HRANDFIELD and ZRANDMEMBER listpack encoded (#12205)
* Numerous other optimizations (#12155, #12082, #11626, #11944, #12316, #12250,
#12177, #12185)
Changes in CLI tools
====================
* redis-cli: Handle RESP3 double responses that contain a NaN (#12254)
* redis-cli: Support URIs with IPv6 (#11834)
Module API changes
==================
* Align semantics of the new (v7.2 RC2) RM_ReplyWithErrorFormat with RM_ReplyWithError.
This is a breaking change that affects the generated error code. (#12321)
* Forbid RM_AddPostNotificationJob on loading and on read-only replicas (#12304)
* Add ability for module command filter to know which client is being handled (#12219)
Bug Fixes
=========
* Fix broken protocol when PUBLISH is used inside MULTI when the RESP3
publishing client is also subscribed for the channel (#12326)
* Fix WAIT to be effective after a blocked module command being unblocked (#12220)
* Re-enable downscale rehashing while there is a fork child (#12276)
* Fix possible hang in HRANDFIELD, SRANDMEMBER, ZRANDMEMBER when used with `<count>` (#12276)
* Improve fairness issue in RANDOMKEY, HRANDFIELD, SRANDMEMBER, ZRANDMEMBER, SPOP, and eviction (#12276)
* Cluster: fix a race condition where a slot migration may revert on a subsequent failover or node joining (#12344)
Fixes for issues in previous releases of Redis 7.2
--------------------------------------------------
* Fix XREADGROUP BLOCK with ">" from hanging (#12301)
* Fix assertion when a blocked command is rejected when re-processed. (#12247)
* Fix use after free on a blocking RM_Call. (#12342)
================================================================================
Redis 7.2 RC2 Released Mon May 15 12:00:00 IST 2023
================================================================================
Upgrade urgency LOW: This is the second Release Candidate for Redis 7.2.
INFO fields and introspection changes
=====================================
* Add a few low level event loop metrics to help diagnose latency (#11963)
Performance and resource utilization improvements
=================================================
* Minor performance improvement to SADD and HSET (#12019)
Platform / toolchain support related changes
=================================================
* Upgrade to Jemalloc 5.3.0, resolves a rare fork child hang (#12115)
* Fix a compiler fortification induced crash when used with link time optimizations (#11982)
* Fix local clients detection, 127.*.*.* instead of 127.0.0.1 (#11664)
* Report AOF failure status to systemd in shutdown (#12065)
Changes in CLI tools
====================
* redis-cli: Reimplement and improve help hints based on actual command arg docs (#10515)
* redis-cli: Add option --count for tuning SCAN based features (#12042)
* redis-benchmark: Add --seed option to seed the random number generator (#11945)
Module API changes
==================
* Add RM_RdbLoad and RM_RdbSave APIs (#11852)
* Add RM_ReplyWithErrorFormat that can support format string (#11923)
* Fix: Delete empty key when RM_ZsetAdd, RM_ZsetIncrby, RM_StreamAdd fail (#12129)
Bug Fixes
=========
* LPOS with RANK set to LONG_MIN returning wrong result (#12167)
* Avoid unnecessary full sync after master restart in a rare case (#12088)
* Iterate clients fairly when processing background chores (#12025)
* Avoid incorrect shrinking of query buffer when reading large data from clients (#12000)
* Sentinel: Fix config rewrite error when old known-slave is used (#11775)
* ACL: Disconnect pub-sub subscribers when revoking allchannels permission (#11992)
* Add a missing fsync of AOF file in rare cases (#11973)
Fixes for issues in previous releases of Redis 7.2
--------------------------------------------------
* Fix tracking of command duration metrics for MULTI, EVAL, WAIT and modules (#11970)
================================================================================
Redis 7.2 RC1 Released Wed Mar 22 12:00:00 IST 2023
================================================================================
Upgrade urgency LOW: This is the first Release Candidate for Redis 7.2.
Redis Release Candidate (RC) versions are early versions that are made available
for early adopters in the community to test them. We do not consider
them suitable for production environments.
Introduction to the Redis 7.2 release
=====================================
Redis 7.2 includes optimizations, several new commands, some improvements,
bug fixes, and several new module APIs.
In particular, users should be aware of the following changes:
1. Redis 7.2 uses a new format (version 11) for RDB files, which is incompatible
with older versions.
2. See section about breaking changes mentioned below.
3. If you use modules, see the module API breaking changes section below.
Here is a comprehensive list of changes in this release compared to 7.0.10.
Each one includes the PR number that added it so that you can get more details
at https://github.com/redis/redis/pull/<number>
New Features
============
* Introduce WAITAOF command, to block the client until a specified number
of Redises have synced all previous write commands to the AOF on disk,
see https://redis.io/commands/waitaof/
New user commands or command arguments
--------------------------------------
* WAITAOF blocks until writes have been synced to disk (#11713)
* Add WITHSCORE option to ZRANK and ZREVRANK (#11235)
New administrative and introspection commands and command arguments
-------------------------------------------------------------------
* CLIENT SETINFO lets client library report name and version Redis (#11758)
* CLIENT NO-TOUCH for clients to run commands without affecting LRU/LFU of keys (#11483)
* Introduce Shard IDs to logically group nodes in cluster mode based on
replication. Shard IDs are automatically assigned and visible via
`CLUSTER MYSHARDID`. (#10536)
Command replies that have been extended
---------------------------------------
* ACL LOG - Add entry id, timestamp created, and timestamp last updated time (#11477)
* COMMAND DOCS - Repurpose arg names as the unique ID (#11051)
* CLIENT LIST has `T` flag to indicate CLIENT NO-TOUCH (#11483)
* CLIENT LIST show lib-name, lib-ver (#11758)
Potentially Breaking / Behavior Changes
=======================================
* Client side tracking for scripts now tracks the keys that are read by the
script instead of the keys that are declared by the caller of EVAL / FCALL (#11770)
* Freeze time sampling during command execution and in scripts (#10300)
* When a blocked command is being unblocked, checks like ACL, OOM, etc are
re-evaluated (#11012)
* Unify ACL failure error message text and error codes (#11160)
* Blocked stream command that's released when key no longer exists carries a
different error code (#11012)
* Command stats are updated for blocked commands only when / if the command
actually executes (#11012)
* The way ACL users are stored internally no longer removes redundant command
and category rules, which may alter the way those rules are displayed as part
of `ACL SAVE`, `ACL GETUSER` and `ACL LIST` (#11224)
* Client connections created for TLS-based replication use SNI if possible (#11458)
* Stream consumers: Re-purpose seen-time, add active-time (#11099)
* XREADGROUP and X[AUTO]CLAIM create the consumer regardless of whether it was
able to perform some reading/claiming (#11099)
* ACL default newly created user set sanitize-payload flag in ACL LIST/GETUSER #11279
* Fix HELLO command not to affect the client state unless successful (#11659)
* Normalize `NAN` in replies to a single nan type, like we do with `inf` (#11597)
Deprecations
============
* Mark the QUIT command as deprecated (#11439)
* Delete RDB loading code for pre-release RDB formats (#11058)
Performance and resource utilization improvements
=================================================
* Significant memory optimization of small list type keys (#11303)
* Significant memory optimization for small set type keys (#11290)
* Significant memory optimization for large sets (#11595)
* Significant speed optimization in ZRANGE replies WITHSCORES in case of integer scores (#11779)
* Significant speed optimization in double replies, mainly sorted sets commands (#10587)
* Optimize the performance of commands with multiple keys in cluster mode (#11044)
* Incrementally reclaim OS page cache of RDB file (#11248)
* Improve memory management of cluster bus links when there is a large number of pending messages (#11343)
* Minor performance improvement for workloads that use commands without pipelining (#11220)
Changes in CLI tools
====================
* redis-cli accepts commands in subscribed mode (#11873)
Other General Improvements
==========================
* WAIT now no longer waits for the replication offset after your last command,
but rather the replication offset after your last write (#11713)
* Automatically propagate node deletion to other nodes in a cluster when
`CLUSTER FORGET` is called, allowing nodes to be deleted with a single call
in most cases (#10869)
* Blocking commands that were disallowed in scripts now behave in scripts the
same they did in MULTI (#11568)
Platform / toolchain support related changes
=================================================
* 32-bit builds compiled without HAVE_MALLOC_SIZE (not jemalloc or glibc)
will consume more memory (#11595)
* Use jemalloc by default also on ARM (#11407)
* Adds stack trace and register dump support in crash report for illumos/solaris (#11335)
New configuration options
=========================
* locale-collate runtime config to control setlocale affecting Lua and SORT (#11059)
* Add CONFIG SET and GET loglevel feature in Sentinel (#11214)
INFO fields and introspection changes
=====================================
* Added 4 new info fields for authentication errors and commands denied access
for keys, channels and commands (#11288)
* INFO SERVER includes a list of listeners (#9320)
Module API changes
==================
* Make it possible for module commands to be part of ACL categories (#11708)
* Add K flag to RM_Call to allow running blocking commands and set a callback to get the response (#11568)
* Add RM_AddPostNotificationJob to allow writes after keyspace notification hooks (#11199)
* RedisModule_Event_Key to notify about keys being unlinked together with reason and value (#9406)
* Add RM_BlockClient[Set|Get]PrivateData to associate a module data with the blocked client (#11568)
* APIs to allow modules to participate / handle AUTH validation (#11659)
* RM_GetContextFlags supports a new flag: REDISMODULE_CTX_FLAGS_SERVER_STARTUP (#9320)
* Add REDISMODULE_OPTIONS_ALLOW_NESTED_KEYSPACE_NOTIFICATIONS and RedisModule_GetModuleOptionsAll (#11199)
* RM_BlockClientOnKeysWithFlags allows module to request being unblocked when the key is deleted (#11310)
* Introduce aux_save2 makes it possible to skip saving that field in the RDB and
enable loading the file in the absence of the module (#11374)
* Add a dry run flag to RM_Call to do validations before actual execution (#11158)
* Add RM_Microseconds and RM_CachedMicroseconds (#11016)
* Add RM_ACLAddLogEntryByUserName API to be used without a user object (#11659)
* Make it possible to keep the RM_Call reply for longer than the context lifetime in case
auto memory was not used (#11568)
Potentially Breaking Changes in Module API
------------------------------------------
* RM_Call only enforces OOM on scripts if 'M' flag is set (#11425)
* Block some specific characters in module command names (#11434)
* Fix replication inconsistency on modules that uses keyspace notifications (#10969)
* Prevent command, configs, data types registration after the onload handler (#11708)
Bug Fixes
=========
* Introduce socket shutdown to properly disconnect a client while a fork is active (#11376)
* CLIENT RESET clears the CLIENT NO-EVICT flag (#11483)
* Reduce memory usage on strings loaded by a module from an RDB file (#11050)
* Fix a bug where nodes in a cluster may not replicate or handle internal events for
keys deleted when another node in the cluster claimed a slot (#11084)
* Fix HINCRBYFLOAT not to create a key if the new value is invalid (#11149)
* Make cluster config file saving atomic and fsync acl file saving (#10924)
* WAIT command would not block if used in RM_Call (#11713)
* Minor fixes to command metadata in COMMAND command (#11201, #10273)
Thanks to all the users and developers who made this release possible.
We'll follow up with more RC releases, until the code looks production ready
and we don't get reports of serious issues for a while.
A special thank you for the amount of work put into this release by:
- Meir Shpilraien
- Guy Benoish
- Viktor Söderqvist
- Zhu Binbin
- Oran Agra
- sundb
- Ran Shidlansik
- Zhenwei Pi
- Jason Elbaum
- Karthik Subbarao
- Madelyn Olson
- Huang Zhw
- Ping Xie
- Ozan Tezcan
- Chen Tianjie
- Deng Ju
- Wen Hui
- Brennan Cathcart
- Itamar Haber
- Shaya Potter
- Roshan Khatri
- Slava Koyfman
- Zhu Tian
- Moti Cohen
- Arad Zilberstein
- Basel Naamna
- Mingyi Kang
- Uri Yagelnik
- Filipe Oliveira
- Zhao Zhao
- Valentino Geron
- Yaacov Hazan
- Adi Pinsky
- David Carlier
- Li Changjun
Happy hacking!
+1 -1
View File
@@ -333,7 +333,7 @@ static int processTimeEvents(aeEventLoop *eventLoop) {
processed++;
now = getMonotonicUs();
if (retval != AE_NOMORE) {
te->when = now + retval * 1000;
te->when = now + (monotime)retval * 1000;
} else {
te->id = AE_DELETED_EVENT_ID;
}
+6 -5
View File
@@ -417,13 +417,16 @@ int anetUnixGenericConnect(char *err, const char *path, int flags)
return s;
}
static int anetListen(char *err, int s, struct sockaddr *sa, socklen_t len, int backlog) {
static int anetListen(char *err, int s, struct sockaddr *sa, socklen_t len, int backlog, mode_t perm) {
if (bind(s,sa,len) == -1) {
anetSetError(err, "bind: %s", strerror(errno));
close(s);
return ANET_ERR;
}
if (sa->sa_family == AF_LOCAL && perm)
chmod(((struct sockaddr_un *) sa)->sun_path, perm);
if (listen(s, backlog) == -1) {
anetSetError(err, "listen: %s", strerror(errno));
close(s);
@@ -467,7 +470,7 @@ static int _anetTcpServer(char *err, int port, char *bindaddr, int af, int backl
if (af == AF_INET6 && anetV6Only(err,s) == ANET_ERR) goto error;
if (anetSetReuseAddr(err,s) == ANET_ERR) goto error;
if (anetListen(err,s,p->ai_addr,p->ai_addrlen,backlog) == ANET_ERR) s = ANET_ERR;
if (anetListen(err,s,p->ai_addr,p->ai_addrlen,backlog,0) == ANET_ERR) s = ANET_ERR;
goto end;
}
if (p == NULL) {
@@ -508,10 +511,8 @@ int anetUnixServer(char *err, char *path, mode_t perm, int backlog)
memset(&sa,0,sizeof(sa));
sa.sun_family = AF_LOCAL;
redis_strlcpy(sa.sun_path,path,sizeof(sa.sun_path));
if (anetListen(err,s,(struct sockaddr*)&sa,sizeof(sa),backlog) == ANET_ERR)
if (anetListen(err,s,(struct sockaddr*)&sa,sizeof(sa),backlog,perm) == ANET_ERR)
return ANET_ERR;
if (perm)
chmod(sa.sun_path, perm);
return s;
}
+16 -12
View File
@@ -976,18 +976,6 @@ void stopAppendOnly(void) {
int startAppendOnly(void) {
serverAssert(server.aof_state == AOF_OFF);
/* Wait for all bio jobs related to AOF to drain. This prevents a race
* between updates to `fsynced_reploff_pending` of the worker thread, belonging
* to the previous AOF, and the new one. This concern is specific for a full
* sync scenario where we don't wanna risk the ACKed replication offset
* jumping backwards or forward when switching to a different master. */
bioDrainWorker(BIO_AOF_FSYNC);
/* Set the initial repl_offset, which will be applied to fsynced_reploff
* when AOFRW finishes (after possibly being updated by a bio thread) */
atomicSet(server.fsynced_reploff_pending, server.master_repl_offset);
server.fsynced_reploff = 0;
server.aof_state = AOF_WAIT_REWRITE;
if (hasActiveChildProcess() && server.child_type != CHILD_TYPE_AOF) {
server.aof_rewrite_scheduled = 1;
@@ -2454,7 +2442,23 @@ int rewriteAppendOnlyFileBackground(void) {
server.aof_lastbgrewrite_status = C_ERR;
return C_ERR;
}
if (server.aof_state == AOF_WAIT_REWRITE) {
/* Wait for all bio jobs related to AOF to drain. This prevents a race
* between updates to `fsynced_reploff_pending` of the worker thread, belonging
* to the previous AOF, and the new one. This concern is specific for a full
* sync scenario where we don't wanna risk the ACKed replication offset
* jumping backwards or forward when switching to a different master. */
bioDrainWorker(BIO_AOF_FSYNC);
/* Set the initial repl_offset, which will be applied to fsynced_reploff
* when AOFRW finishes (after possibly being updated by a bio thread) */
atomicSet(server.fsynced_reploff_pending, server.master_repl_offset);
server.fsynced_reploff = 0;
}
server.stat_aof_rewrites++;
if ((childpid = redisFork(CHILD_TYPE_AOF)) == 0) {
char tmpfile[256];
+54 -17
View File
@@ -119,6 +119,20 @@ static inline int defaultClientPort(void) {
return server.tls_cluster ? server.tls_port : server.port;
}
/* When a cluster command is called, we need to decide whether to return TLS info or
* non-TLS info by the client's connection type. However if the command is called by
* a Lua script or RM_call, there is no connection in the fake client, so we use
* server.current_client here to get the real client if available. And if it is not
* available (modules may call commands without a real client), we return the default
* info, which is determined by server.tls_cluster. */
static int shouldReturnTlsInfo(void) {
if (server.current_client && server.current_client->conn) {
return connIsTLS(server.current_client->conn);
} else {
return server.tls_cluster;
}
}
/* Links to the next and previous entries for keys in the same slot are stored
* in the dict entry metadata. See Slot to Key API below. */
#define dictEntryNextInSlot(de) \
@@ -938,13 +952,13 @@ static void updateAnnouncedHumanNodename(clusterNode *node, char *new) {
static void updateShardId(clusterNode *node, const char *shard_id) {
if (memcmp(node->shard_id, shard_id, CLUSTER_NAMELEN) != 0) {
if (shard_id && memcmp(node->shard_id, shard_id, CLUSTER_NAMELEN) != 0) {
clusterRemoveNodeFromShard(node);
memcpy(node->shard_id, shard_id, CLUSTER_NAMELEN);
clusterAddNodeToShard(shard_id, node);
clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG);
}
if (myself != node && myself->slaveof == node) {
if (shard_id && myself != node && myself->slaveof == node) {
if (memcmp(myself->shard_id, shard_id, CLUSTER_NAMELEN) != 0) {
/* shard-id can diverge right after a rolling upgrade
* from pre-7.2 releases */
@@ -1120,6 +1134,9 @@ void clusterReset(int hard) {
}
dictReleaseIterator(di);
/* Empty the nodes blacklist. */
dictEmpty(server.cluster->nodes_black_list, NULL);
/* Hard reset only: set epochs to 0, change node ID. */
if (hard) {
sds oldname;
@@ -1670,6 +1687,7 @@ void clusterRenameNode(clusterNode *node, char *newname) {
serverAssert(retval == DICT_OK);
memcpy(node->name, newname, CLUSTER_NAMELEN);
clusterAddNode(node);
clusterAddNodeToShard(node->shard_id, node);
}
void clusterAddNodeToShard(const char *shard_id, clusterNode *node) {
@@ -2217,6 +2235,7 @@ void clusterProcessGossipSection(clusterMsg *hdr, clusterLink *link) {
node->tls_port = msg_tls_port;
node->cport = ntohs(g->cport);
clusterAddNode(node);
clusterAddNodeToShard(node->shard_id, node);
}
}
@@ -2394,7 +2413,6 @@ void clusterUpdateSlotsConfigWith(clusterNode *sender, uint64_t senderConfigEpoc
}
clusterDelSlot(j);
clusterAddSlot(sender,j);
bitmapClearBit(server.cluster->owner_not_claiming_slot, j);
clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG|
CLUSTER_TODO_UPDATE_STATE|
CLUSTER_TODO_FSYNC_CONFIG);
@@ -2641,8 +2659,7 @@ void clusterProcessPingExtensions(clusterMsg *hdr, clusterLink *link) {
clusterNode *n = clusterLookupNode(forgotten_node_ext->name, CLUSTER_NAMELEN);
if (n && n != myself && !(nodeIsSlave(myself) && myself->slaveof == n)) {
sds id = sdsnewlen(forgotten_node_ext->name, CLUSTER_NAMELEN);
dictEntry *de = dictAddRaw(server.cluster->nodes_black_list, id, NULL);
serverAssert(de != NULL);
dictEntry *de = dictAddOrFind(server.cluster->nodes_black_list, id);
uint64_t expire = server.unixtime + ntohu64(forgotten_node_ext->ttl);
dictSetUnsignedIntegerVal(de, expire);
clusterDelNode(n);
@@ -2660,11 +2677,24 @@ void clusterProcessPingExtensions(clusterMsg *hdr, clusterLink *link) {
/* We know this will be valid since we validated it ahead of time */
ext = getNextPingExt(ext);
}
/* If the node did not send us a hostname extension, assume
* they don't have an announced hostname. Otherwise, we'll
* set it now. */
updateAnnouncedHostname(sender, ext_hostname);
updateAnnouncedHumanNodename(sender, ext_humannodename);
/* If the node did not send us a shard-id extension, it means the sender
* does not support it (old version), node->shard_id is randomly generated.
* A cluster-wide consensus for the node's shard_id is not necessary.
* The key is maintaining consistency of the shard_id on each individual 7.2 node.
* As the cluster progressively upgrades to version 7.2, we can expect the shard_ids
* across all nodes to naturally converge and align.
*
* If sender is a replica, set the shard_id to the shard_id of its master.
* Otherwise, we'll set it now. */
if (ext_shardid == NULL) ext_shardid = clusterNodeGetMaster(sender)->shard_id;
updateShardId(sender, ext_shardid);
}
@@ -3008,6 +3038,10 @@ int clusterProcessPacket(clusterLink *link) {
clusterNodeAddSlave(master,sender);
sender->slaveof = master;
/* Update the shard_id when a replica is connected to its
* primary in the very first time. */
updateShardId(sender, master->shard_id);
/* Update config. */
clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG);
}
@@ -4922,14 +4956,12 @@ int clusterDelSlot(int slot) {
if (!n) return C_ERR;
/* Cleanup the channels in master/replica as part of slot deletion. */
list *nodes_for_slot = clusterGetNodesInMyShard(n);
serverAssert(nodes_for_slot != NULL);
listNode *ln = listSearchKey(nodes_for_slot, myself);
if (ln != NULL) {
removeChannelsInSlot(slot);
}
removeChannelsInSlot(slot);
/* Clear the slot bit. */
serverAssert(clusterNodeClearSlotBit(n,slot) == 1);
server.cluster->slots[slot] = NULL;
/* Make owner_not_claiming_slot flag consistent with slot ownership information. */
bitmapClearBit(server.cluster->owner_not_claiming_slot, slot);
return C_OK;
}
@@ -5570,7 +5602,7 @@ void addNodeToNodeReply(client *c, clusterNode *node) {
}
/* Report TLS ports to TLS client, and report non-TLS port to non-TLS client. */
addReplyLongLong(c, getNodeClientPort(node, connIsTLS(c->conn)));
addReplyLongLong(c, getNodeClientPort(node, shouldReturnTlsInfo()));
addReplyBulkCBuffer(c, node->name, CLUSTER_NAMELEN);
/* Add the additional endpoint information, this is all the known networking information
@@ -5697,13 +5729,13 @@ void addShardReplyForClusterShards(client *c, list *nodes) {
addReplyBulkCString(c, "slots");
/* Use slot_info_pairs from the primary only */
while (n->slaveof != NULL) n = n->slaveof;
n = clusterNodeGetMaster(n);
if (n->slot_info_pairs != NULL) {
serverAssert((n->slot_info_pairs_count % 2) == 0);
addReplyArrayLen(c, n->slot_info_pairs_count);
for (int i = 0; i < n->slot_info_pairs_count; i++)
addReplyBulkLongLong(c, (unsigned long)n->slot_info_pairs[i]);
addReplyLongLong(c, (unsigned long)n->slot_info_pairs[i]);
} else {
/* If no slot info pair is provided, the node owns no slots */
addReplyArrayLen(c, 0);
@@ -5946,7 +5978,7 @@ NULL
} else if (!strcasecmp(c->argv[1]->ptr,"nodes") && c->argc == 2) {
/* CLUSTER NODES */
/* Report TLS ports to TLS client, and report non-TLS port to non-TLS client. */
sds nodes = clusterGenNodesDescription(c, 0, connIsTLS(c->conn));
sds nodes = clusterGenNodesDescription(c, 0, shouldReturnTlsInfo());
addReplyVerbatim(c,nodes,sdslen(nodes),"txt");
sdsfree(nodes);
} else if (!strcasecmp(c->argv[1]->ptr,"myid") && c->argc == 2) {
@@ -6312,7 +6344,7 @@ NULL
/* Report TLS ports to TLS client, and report non-TLS port to non-TLS client. */
addReplyArrayLen(c,n->numslaves);
for (j = 0; j < n->numslaves; j++) {
sds ni = clusterGenNodeDescription(c, n->slaves[j], connIsTLS(c->conn));
sds ni = clusterGenNodeDescription(c, n->slaves[j], shouldReturnTlsInfo());
addReplyBulkCString(c,ni);
sdsfree(ni);
}
@@ -7438,7 +7470,7 @@ void clusterRedirectClient(client *c, clusterNode *n, int hashslot, int error_co
error_code == CLUSTER_REDIR_ASK)
{
/* Report TLS ports to TLS client, and report non-TLS port to non-TLS client. */
int port = getNodeClientPort(n, connIsTLS(c->conn));
int port = getNodeClientPort(n, shouldReturnTlsInfo());
addReplyErrorSds(c,sdscatprintf(sdsempty(),
"-%s %d %s:%d",
(error_code == CLUSTER_REDIR_ASK) ? "ASK" : "MOVED",
@@ -7631,6 +7663,11 @@ unsigned int countKeysInSlot(unsigned int hashslot) {
return (*server.db->slots_to_keys).by_slot[hashslot].count;
}
clusterNode *clusterNodeGetMaster(clusterNode *node) {
while (node->slaveof != NULL) node = node->slaveof;
return node;
}
/* -----------------------------------------------------------------------------
* Operation(s) on channel rax tree.
* -------------------------------------------------------------------------- */
+1
View File
@@ -413,6 +413,7 @@ void clusterInit(void);
void clusterInitListeners(void);
void clusterCron(void);
void clusterBeforeSleep(void);
clusterNode *clusterNodeGetMaster(clusterNode *node);
clusterNode *getNodeByQuery(client *c, struct redisCommand *cmd, robj **argv, int argc, int *hashslot, int *ask);
int verifyClusterNodeId(const char *name, int length);
clusterNode *clusterLookupNode(const char *name, int length);
+76 -21
View File
@@ -1391,7 +1391,10 @@ struct COMMAND_ARG CLIENT_REPLY_Args[] = {
#ifndef SKIP_CMD_TIPS_TABLE
/* CLIENT SETINFO tips */
#define CLIENT_SETINFO_Tips NULL
const char *CLIENT_SETINFO_Tips[] = {
"request_policy:all_nodes",
"response_policy:all_succeeded",
};
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
@@ -1419,7 +1422,10 @@ struct COMMAND_ARG CLIENT_SETINFO_Args[] = {
#ifndef SKIP_CMD_TIPS_TABLE
/* CLIENT SETNAME tips */
#define CLIENT_SETNAME_Tips NULL
const char *CLIENT_SETNAME_Tips[] = {
"request_policy:all_nodes",
"response_policy:all_succeeded",
};
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
@@ -1543,8 +1549,8 @@ struct COMMAND_STRUCT CLIENT_Subcommands[] = {
{MAKE_CMD("no-touch","Controls whether commands sent by the client affect the LRU/LFU of accessed keys.","O(1)","7.2.0",CMD_DOC_NONE,NULL,NULL,"connection",COMMAND_GROUP_CONNECTION,CLIENT_NO_TOUCH_History,0,CLIENT_NO_TOUCH_Tips,0,clientCommand,3,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,ACL_CATEGORY_CONNECTION,CLIENT_NO_TOUCH_Keyspecs,0,NULL,1),.args=CLIENT_NO_TOUCH_Args},
{MAKE_CMD("pause","Suspends commands processing.","O(1)","3.0.0",CMD_DOC_NONE,NULL,NULL,"connection",COMMAND_GROUP_CONNECTION,CLIENT_PAUSE_History,1,CLIENT_PAUSE_Tips,0,clientCommand,-3,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,ACL_CATEGORY_CONNECTION,CLIENT_PAUSE_Keyspecs,0,NULL,2),.args=CLIENT_PAUSE_Args},
{MAKE_CMD("reply","Instructs the server whether to reply to commands.","O(1)","3.2.0",CMD_DOC_NONE,NULL,NULL,"connection",COMMAND_GROUP_CONNECTION,CLIENT_REPLY_History,0,CLIENT_REPLY_Tips,0,clientCommand,3,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,ACL_CATEGORY_CONNECTION,CLIENT_REPLY_Keyspecs,0,NULL,1),.args=CLIENT_REPLY_Args},
{MAKE_CMD("setinfo","Sets information specific to the client or connection.","O(1)","7.2.0",CMD_DOC_NONE,NULL,NULL,"connection",COMMAND_GROUP_CONNECTION,CLIENT_SETINFO_History,0,CLIENT_SETINFO_Tips,0,clientSetinfoCommand,4,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,ACL_CATEGORY_CONNECTION,CLIENT_SETINFO_Keyspecs,0,NULL,1),.args=CLIENT_SETINFO_Args},
{MAKE_CMD("setname","Sets the connection name.","O(1)","2.6.9",CMD_DOC_NONE,NULL,NULL,"connection",COMMAND_GROUP_CONNECTION,CLIENT_SETNAME_History,0,CLIENT_SETNAME_Tips,0,clientCommand,3,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,ACL_CATEGORY_CONNECTION,CLIENT_SETNAME_Keyspecs,0,NULL,1),.args=CLIENT_SETNAME_Args},
{MAKE_CMD("setinfo","Sets information specific to the client or connection.","O(1)","7.2.0",CMD_DOC_NONE,NULL,NULL,"connection",COMMAND_GROUP_CONNECTION,CLIENT_SETINFO_History,0,CLIENT_SETINFO_Tips,2,clientSetinfoCommand,4,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,ACL_CATEGORY_CONNECTION,CLIENT_SETINFO_Keyspecs,0,NULL,1),.args=CLIENT_SETINFO_Args},
{MAKE_CMD("setname","Sets the connection name.","O(1)","2.6.9",CMD_DOC_NONE,NULL,NULL,"connection",COMMAND_GROUP_CONNECTION,CLIENT_SETNAME_History,0,CLIENT_SETNAME_Tips,2,clientCommand,3,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,ACL_CATEGORY_CONNECTION,CLIENT_SETNAME_Keyspecs,0,NULL,1),.args=CLIENT_SETNAME_Args},
{MAKE_CMD("tracking","Controls server-assisted client-side caching for the connection.","O(1). Some options may introduce additional complexity.","6.0.0",CMD_DOC_NONE,NULL,NULL,"connection",COMMAND_GROUP_CONNECTION,CLIENT_TRACKING_History,0,CLIENT_TRACKING_Tips,0,clientCommand,-3,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,ACL_CATEGORY_CONNECTION,CLIENT_TRACKING_Keyspecs,0,NULL,7),.args=CLIENT_TRACKING_Args},
{MAKE_CMD("trackinginfo","Returns information about server-assisted client-side caching for the connection.","O(1)","6.2.0",CMD_DOC_NONE,NULL,NULL,"connection",COMMAND_GROUP_CONNECTION,CLIENT_TRACKINGINFO_History,0,CLIENT_TRACKINGINFO_Tips,0,clientCommand,2,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,ACL_CATEGORY_CONNECTION,CLIENT_TRACKINGINFO_Keyspecs,0,NULL,0)},
{MAKE_CMD("unblock","Unblocks a client blocked by a blocking command from a different connection.","O(log N) where N is the number of client connections","5.0.0",CMD_DOC_NONE,NULL,NULL,"connection",COMMAND_GROUP_CONNECTION,CLIENT_UNBLOCK_History,0,CLIENT_UNBLOCK_Tips,0,clientCommand,-3,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,ACL_CATEGORY_CONNECTION,CLIENT_UNBLOCK_Keyspecs,0,NULL,2),.args=CLIENT_UNBLOCK_Args},
@@ -5886,7 +5892,10 @@ struct COMMAND_ARG ACL_CAT_Args[] = {
#ifndef SKIP_CMD_TIPS_TABLE
/* ACL DELUSER tips */
#define ACL_DELUSER_Tips NULL
const char *ACL_DELUSER_Tips[] = {
"request_policy:all_nodes",
"response_policy:all_succeeded",
};
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
@@ -6060,7 +6069,10 @@ struct COMMAND_ARG ACL_LOG_Args[] = {
#ifndef SKIP_CMD_TIPS_TABLE
/* ACL SAVE tips */
#define ACL_SAVE_Tips NULL
const char *ACL_SAVE_Tips[] = {
"request_policy:all_nodes",
"response_policy:all_succeeded",
};
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
@@ -6080,7 +6092,10 @@ commandHistory ACL_SETUSER_History[] = {
#ifndef SKIP_CMD_TIPS_TABLE
/* ACL SETUSER tips */
#define ACL_SETUSER_Tips NULL
const char *ACL_SETUSER_Tips[] = {
"request_policy:all_nodes",
"response_policy:all_succeeded",
};
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
@@ -6131,7 +6146,7 @@ struct COMMAND_ARG ACL_SETUSER_Args[] = {
/* ACL command table */
struct COMMAND_STRUCT ACL_Subcommands[] = {
{MAKE_CMD("cat","Lists the ACL categories, or the commands inside a category.","O(1) since the categories and commands are a fixed set.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_CAT_History,0,ACL_CAT_Tips,0,aclCommand,-2,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_CAT_Keyspecs,0,NULL,1),.args=ACL_CAT_Args},
{MAKE_CMD("deluser","Deletes ACL users, and terminates their connections.","O(1) amortized time considering the typical user.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_DELUSER_History,0,ACL_DELUSER_Tips,0,aclCommand,-3,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_DELUSER_Keyspecs,0,NULL,1),.args=ACL_DELUSER_Args},
{MAKE_CMD("deluser","Deletes ACL users, and terminates their connections.","O(1) amortized time considering the typical user.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_DELUSER_History,0,ACL_DELUSER_Tips,2,aclCommand,-3,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_DELUSER_Keyspecs,0,NULL,1),.args=ACL_DELUSER_Args},
{MAKE_CMD("dryrun","Simulates the execution of a command by a user, without executing the command.","O(1).","7.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_DRYRUN_History,0,ACL_DRYRUN_Tips,0,aclCommand,-4,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_DRYRUN_Keyspecs,0,NULL,3),.args=ACL_DRYRUN_Args},
{MAKE_CMD("genpass","Generates a pseudorandom, secure password that can be used to identify ACL users.","O(1)","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_GENPASS_History,0,ACL_GENPASS_Tips,0,aclCommand,-2,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_GENPASS_Keyspecs,0,NULL,1),.args=ACL_GENPASS_Args},
{MAKE_CMD("getuser","Lists the ACL rules of a user.","O(N). Where N is the number of password, command and pattern rules that the user has.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_GETUSER_History,2,ACL_GETUSER_Tips,0,aclCommand,3,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_GETUSER_Keyspecs,0,NULL,1),.args=ACL_GETUSER_Args},
@@ -6139,8 +6154,8 @@ struct COMMAND_STRUCT ACL_Subcommands[] = {
{MAKE_CMD("list","Dumps the effective rules in ACL file format.","O(N). Where N is the number of configured users.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_LIST_History,0,ACL_LIST_Tips,0,aclCommand,2,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_LIST_Keyspecs,0,NULL,0)},
{MAKE_CMD("load","Reloads the rules from the configured ACL file.","O(N). Where N is the number of configured users.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_LOAD_History,0,ACL_LOAD_Tips,0,aclCommand,2,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_LOAD_Keyspecs,0,NULL,0)},
{MAKE_CMD("log","Lists recent security events generated due to ACL rules.","O(N) with N being the number of entries shown.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_LOG_History,1,ACL_LOG_Tips,0,aclCommand,-2,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_LOG_Keyspecs,0,NULL,1),.args=ACL_LOG_Args},
{MAKE_CMD("save","Saves the effective ACL rules in the configured ACL file.","O(N). Where N is the number of configured users.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_SAVE_History,0,ACL_SAVE_Tips,0,aclCommand,2,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_SAVE_Keyspecs,0,NULL,0)},
{MAKE_CMD("setuser","Creates and modifies an ACL user and its rules.","O(N). Where N is the number of rules provided.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_SETUSER_History,2,ACL_SETUSER_Tips,0,aclCommand,-3,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_SETUSER_Keyspecs,0,NULL,2),.args=ACL_SETUSER_Args},
{MAKE_CMD("save","Saves the effective ACL rules in the configured ACL file.","O(N). Where N is the number of configured users.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_SAVE_History,0,ACL_SAVE_Tips,2,aclCommand,2,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_SAVE_Keyspecs,0,NULL,0)},
{MAKE_CMD("setuser","Creates and modifies an ACL user and its rules.","O(N). Where N is the number of rules provided.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_SETUSER_History,2,ACL_SETUSER_Tips,2,aclCommand,-3,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_SETUSER_Keyspecs,0,NULL,2),.args=ACL_SETUSER_Args},
{MAKE_CMD("users","Lists all ACL users.","O(N). Where N is the number of configured users.","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_USERS_History,0,ACL_USERS_Tips,0,aclCommand,2,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_USERS_Keyspecs,0,NULL,0)},
{MAKE_CMD("whoami","Returns the authenticated username of the current connection.","O(1)","6.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ACL_WHOAMI_History,0,ACL_WHOAMI_Tips,0,aclCommand,2,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,ACL_WHOAMI_Keyspecs,0,NULL,0)},
{0}
@@ -6446,7 +6461,10 @@ struct COMMAND_ARG CONFIG_GET_Args[] = {
#ifndef SKIP_CMD_TIPS_TABLE
/* CONFIG RESETSTAT tips */
#define CONFIG_RESETSTAT_Tips NULL
const char *CONFIG_RESETSTAT_Tips[] = {
"request_policy:all_nodes",
"response_policy:all_succeeded",
};
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
@@ -6463,7 +6481,10 @@ struct COMMAND_ARG CONFIG_GET_Args[] = {
#ifndef SKIP_CMD_TIPS_TABLE
/* CONFIG REWRITE tips */
#define CONFIG_REWRITE_Tips NULL
const char *CONFIG_REWRITE_Tips[] = {
"request_policy:all_nodes",
"response_policy:all_succeeded",
};
#endif
#ifndef SKIP_CMD_KEY_SPECS_TABLE
@@ -6508,8 +6529,8 @@ struct COMMAND_ARG CONFIG_SET_Args[] = {
struct COMMAND_STRUCT CONFIG_Subcommands[] = {
{MAKE_CMD("get","Returns the effective values of configuration parameters.","O(N) when N is the number of configuration parameters provided","2.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,CONFIG_GET_History,1,CONFIG_GET_Tips,0,configGetCommand,-3,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,0,CONFIG_GET_Keyspecs,0,NULL,1),.args=CONFIG_GET_Args},
{MAKE_CMD("help","Returns helpful text about the different subcommands.","O(1)","5.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,CONFIG_HELP_History,0,CONFIG_HELP_Tips,0,configHelpCommand,2,CMD_LOADING|CMD_STALE,0,CONFIG_HELP_Keyspecs,0,NULL,0)},
{MAKE_CMD("resetstat","Resets the server's statistics.","O(1)","2.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,CONFIG_RESETSTAT_History,0,CONFIG_RESETSTAT_Tips,0,configResetStatCommand,2,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,0,CONFIG_RESETSTAT_Keyspecs,0,NULL,0)},
{MAKE_CMD("rewrite","Persists the effective configuration to file.","O(1)","2.8.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,CONFIG_REWRITE_History,0,CONFIG_REWRITE_Tips,0,configRewriteCommand,2,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,0,CONFIG_REWRITE_Keyspecs,0,NULL,0)},
{MAKE_CMD("resetstat","Resets the server's statistics.","O(1)","2.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,CONFIG_RESETSTAT_History,0,CONFIG_RESETSTAT_Tips,2,configResetStatCommand,2,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,0,CONFIG_RESETSTAT_Keyspecs,0,NULL,0)},
{MAKE_CMD("rewrite","Persists the effective configuration to file.","O(1)","2.8.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,CONFIG_REWRITE_History,0,CONFIG_REWRITE_Tips,2,configRewriteCommand,2,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,0,CONFIG_REWRITE_Keyspecs,0,NULL,0)},
{MAKE_CMD("set","Sets configuration parameters in-flight.","O(N) when N is the number of configuration parameters provided","2.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,CONFIG_SET_History,1,CONFIG_SET_Tips,2,configSetCommand,-4,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,0,CONFIG_SET_Keyspecs,0,NULL,1),.args=CONFIG_SET_Args},
{0}
};
@@ -6862,7 +6883,7 @@ const char *LATENCY_LATEST_Tips[] = {
/* LATENCY RESET tips */
const char *LATENCY_RESET_Tips[] = {
"request_policy:all_nodes",
"response_policy:all_succeeded",
"response_policy:agg_sum",
};
#endif
@@ -7292,12 +7313,29 @@ struct COMMAND_ARG PSYNC_Args[] = {
#define REPLICAOF_Keyspecs NULL
#endif
/* REPLICAOF argument table */
struct COMMAND_ARG REPLICAOF_Args[] = {
/* REPLICAOF args host_port argument table */
struct COMMAND_ARG REPLICAOF_args_host_port_Subargs[] = {
{MAKE_ARG("host",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("port",ARG_TYPE_INTEGER,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* REPLICAOF args no_one argument table */
struct COMMAND_ARG REPLICAOF_args_no_one_Subargs[] = {
{MAKE_ARG("no",ARG_TYPE_PURE_TOKEN,-1,"NO",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("one",ARG_TYPE_PURE_TOKEN,-1,"ONE",NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* REPLICAOF args argument table */
struct COMMAND_ARG REPLICAOF_args_Subargs[] = {
{MAKE_ARG("host-port",ARG_TYPE_BLOCK,-1,NULL,NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=REPLICAOF_args_host_port_Subargs},
{MAKE_ARG("no-one",ARG_TYPE_BLOCK,-1,NULL,NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=REPLICAOF_args_no_one_Subargs},
};
/* REPLICAOF argument table */
struct COMMAND_ARG REPLICAOF_Args[] = {
{MAKE_ARG("args",ARG_TYPE_ONEOF,-1,NULL,NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=REPLICAOF_args_Subargs},
};
/********** RESTORE_ASKING ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
@@ -7416,12 +7454,29 @@ struct COMMAND_ARG SHUTDOWN_Args[] = {
#define SLAVEOF_Keyspecs NULL
#endif
/* SLAVEOF argument table */
struct COMMAND_ARG SLAVEOF_Args[] = {
/* SLAVEOF args host_port argument table */
struct COMMAND_ARG SLAVEOF_args_host_port_Subargs[] = {
{MAKE_ARG("host",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("port",ARG_TYPE_INTEGER,-1,NULL,NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* SLAVEOF args no_one argument table */
struct COMMAND_ARG SLAVEOF_args_no_one_Subargs[] = {
{MAKE_ARG("no",ARG_TYPE_PURE_TOKEN,-1,"NO",NULL,NULL,CMD_ARG_NONE,0,NULL)},
{MAKE_ARG("one",ARG_TYPE_PURE_TOKEN,-1,"ONE",NULL,NULL,CMD_ARG_NONE,0,NULL)},
};
/* SLAVEOF args argument table */
struct COMMAND_ARG SLAVEOF_args_Subargs[] = {
{MAKE_ARG("host-port",ARG_TYPE_BLOCK,-1,NULL,NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=SLAVEOF_args_host_port_Subargs},
{MAKE_ARG("no-one",ARG_TYPE_BLOCK,-1,NULL,NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=SLAVEOF_args_no_one_Subargs},
};
/* SLAVEOF argument table */
struct COMMAND_ARG SLAVEOF_Args[] = {
{MAKE_ARG("args",ARG_TYPE_ONEOF,-1,NULL,NULL,NULL,CMD_ARG_NONE,2,NULL),.subargs=SLAVEOF_args_Subargs},
};
/********** SLOWLOG GET ********************/
#ifndef SKIP_CMD_HISTORY_TABLE
@@ -10731,12 +10786,12 @@ struct COMMAND_STRUCT redisCommandTable[] = {
{MAKE_CMD("monitor","Listens for all requests received by the server in real-time.",NULL,"1.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,MONITOR_History,0,MONITOR_Tips,0,monitorCommand,1,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,0,MONITOR_Keyspecs,0,NULL,0)},
{MAKE_CMD("psync","An internal command used in replication.",NULL,"2.8.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,PSYNC_History,0,PSYNC_Tips,0,syncCommand,-3,CMD_NO_ASYNC_LOADING|CMD_ADMIN|CMD_NO_MULTI|CMD_NOSCRIPT,0,PSYNC_Keyspecs,0,NULL,2),.args=PSYNC_Args},
{MAKE_CMD("replconf","An internal command for configuring the replication stream.","O(1)","3.0.0",CMD_DOC_SYSCMD,NULL,NULL,"server",COMMAND_GROUP_SERVER,REPLCONF_History,0,REPLCONF_Tips,0,replconfCommand,-1,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_ALLOW_BUSY,0,REPLCONF_Keyspecs,0,NULL,0)},
{MAKE_CMD("replicaof","Configures a server as replica of another, or promotes it to a master.","O(1)","5.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,REPLICAOF_History,0,REPLICAOF_Tips,0,replicaofCommand,3,CMD_NO_ASYNC_LOADING|CMD_ADMIN|CMD_NOSCRIPT|CMD_STALE,0,REPLICAOF_Keyspecs,0,NULL,2),.args=REPLICAOF_Args},
{MAKE_CMD("replicaof","Configures a server as replica of another, or promotes it to a master.","O(1)","5.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,REPLICAOF_History,0,REPLICAOF_Tips,0,replicaofCommand,3,CMD_NO_ASYNC_LOADING|CMD_ADMIN|CMD_NOSCRIPT|CMD_STALE,0,REPLICAOF_Keyspecs,0,NULL,1),.args=REPLICAOF_Args},
{MAKE_CMD("restore-asking","An internal command for migrating keys in a cluster.","O(1) to create the new key and additional O(N*M) to reconstruct the serialized value, where N is the number of Redis objects composing the value and M their average size. For small string values the time complexity is thus O(1)+O(1*M) where M is small, so simply O(1). However for sorted set values the complexity is O(N*M*log(N)) because inserting values into sorted sets is O(log(N)).","3.0.0",CMD_DOC_SYSCMD,NULL,NULL,"server",COMMAND_GROUP_SERVER,RESTORE_ASKING_History,3,RESTORE_ASKING_Tips,0,restoreCommand,-4,CMD_WRITE|CMD_DENYOOM|CMD_ASKING,ACL_CATEGORY_KEYSPACE|ACL_CATEGORY_DANGEROUS,RESTORE_ASKING_Keyspecs,1,NULL,7),.args=RESTORE_ASKING_Args},
{MAKE_CMD("role","Returns the replication role.","O(1)","2.8.12",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,ROLE_History,0,ROLE_Tips,0,roleCommand,1,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_FAST|CMD_SENTINEL,ACL_CATEGORY_ADMIN|ACL_CATEGORY_DANGEROUS,ROLE_Keyspecs,0,NULL,0)},
{MAKE_CMD("save","Synchronously saves the database(s) to disk.","O(N) where N is the total number of keys in all databases","1.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,SAVE_History,0,SAVE_Tips,0,saveCommand,1,CMD_NO_ASYNC_LOADING|CMD_ADMIN|CMD_NOSCRIPT|CMD_NO_MULTI,0,SAVE_Keyspecs,0,NULL,0)},
{MAKE_CMD("shutdown","Synchronously saves the database(s) to disk and shuts down the Redis server.","O(N) when saving, where N is the total number of keys in all databases when saving data, otherwise O(1)","1.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,SHUTDOWN_History,1,SHUTDOWN_Tips,0,shutdownCommand,-1,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_NO_MULTI|CMD_SENTINEL|CMD_ALLOW_BUSY,0,SHUTDOWN_Keyspecs,0,NULL,4),.args=SHUTDOWN_Args},
{MAKE_CMD("slaveof","Sets a Redis server as a replica of another, or promotes it to being a master.","O(1)","1.0.0",CMD_DOC_DEPRECATED,"`REPLICAOF`","5.0.0","server",COMMAND_GROUP_SERVER,SLAVEOF_History,0,SLAVEOF_Tips,0,replicaofCommand,3,CMD_NO_ASYNC_LOADING|CMD_ADMIN|CMD_NOSCRIPT|CMD_STALE,0,SLAVEOF_Keyspecs,0,NULL,2),.args=SLAVEOF_Args},
{MAKE_CMD("slaveof","Sets a Redis server as a replica of another, or promotes it to being a master.","O(1)","1.0.0",CMD_DOC_DEPRECATED,"`REPLICAOF`","5.0.0","server",COMMAND_GROUP_SERVER,SLAVEOF_History,0,SLAVEOF_Tips,0,replicaofCommand,3,CMD_NO_ASYNC_LOADING|CMD_ADMIN|CMD_NOSCRIPT|CMD_STALE,0,SLAVEOF_Keyspecs,0,NULL,1),.args=SLAVEOF_Args},
{MAKE_CMD("slowlog","A container for slow log commands.","Depends on subcommand.","2.2.12",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,SLOWLOG_History,0,SLOWLOG_Tips,0,NULL,-2,0,0,SLOWLOG_Keyspecs,0,NULL,0),.subcommands=SLOWLOG_Subcommands},
{MAKE_CMD("swapdb","Swaps two Redis databases.","O(N) where N is the count of clients watching or blocking on keys from both databases.","4.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,SWAPDB_History,0,SWAPDB_Tips,0,swapdbCommand,3,CMD_WRITE|CMD_FAST,ACL_CATEGORY_KEYSPACE|ACL_CATEGORY_DANGEROUS,SWAPDB_Keyspecs,0,NULL,2),.args=SWAPDB_Args},
{MAKE_CMD("sync","An internal command used in replication.",NULL,"1.0.0",CMD_DOC_NONE,NULL,NULL,"server",COMMAND_GROUP_SERVER,SYNC_History,0,SYNC_Tips,0,syncCommand,1,CMD_NO_ASYNC_LOADING|CMD_ADMIN|CMD_NO_MULTI|CMD_NOSCRIPT,0,SYNC_Keyspecs,0,NULL,0)},
+4
View File
@@ -14,6 +14,10 @@
"STALE",
"SENTINEL"
],
"command_tips": [
"REQUEST_POLICY:ALL_NODES",
"RESPONSE_POLICY:ALL_SUCCEEDED"
],
"reply_schema": {
"type": "integer",
"description": "The number of users that were deleted"
+4
View File
@@ -14,6 +14,10 @@
"STALE",
"SENTINEL"
],
"command_tips": [
"REQUEST_POLICY:ALL_NODES",
"RESPONSE_POLICY:ALL_SUCCEEDED"
],
"reply_schema": {
"const": "OK"
}
+4
View File
@@ -24,6 +24,10 @@
"STALE",
"SENTINEL"
],
"command_tips": [
"REQUEST_POLICY:ALL_NODES",
"RESPONSE_POLICY:ALL_SUCCEEDED"
],
"reply_schema": {
"const": "OK"
},
+4
View File
@@ -13,6 +13,10 @@
"STALE",
"SENTINEL"
],
"command_tips": [
"REQUEST_POLICY:ALL_NODES",
"RESPONSE_POLICY:ALL_SUCCEEDED"
],
"acl_categories": [
"CONNECTION"
],
+4
View File
@@ -13,6 +13,10 @@
"STALE",
"SENTINEL"
],
"command_tips": [
"REQUEST_POLICY:ALL_NODES",
"RESPONSE_POLICY:ALL_SUCCEEDED"
],
"acl_categories": [
"CONNECTION"
],
+1 -1
View File
@@ -26,7 +26,7 @@
"description": "an even number element array specifying the start and end slot numbers for slot ranges owned by this shard",
"type": "array",
"items": {
"type": "string"
"type": "integer"
}
},
"nodes": {
+4
View File
@@ -13,6 +13,10 @@
"LOADING",
"STALE"
],
"command_tips": [
"REQUEST_POLICY:ALL_NODES",
"RESPONSE_POLICY:ALL_SUCCEEDED"
],
"reply_schema": {
"const": "OK"
}
+4
View File
@@ -13,6 +13,10 @@
"LOADING",
"STALE"
],
"command_tips": [
"REQUEST_POLICY:ALL_NODES",
"RESPONSE_POLICY:ALL_SUCCEEDED"
],
"reply_schema": {
"const": "OK"
}
+1 -1
View File
@@ -15,7 +15,7 @@
],
"command_tips": [
"REQUEST_POLICY:ALL_NODES",
"RESPONSE_POLICY:ALL_SUCCEEDED"
"RESPONSE_POLICY:AGG_SUM"
],
"reply_schema": {
"type": "integer",
+34 -6
View File
@@ -14,12 +14,40 @@
],
"arguments": [
{
"name": "host",
"type": "string"
},
{
"name": "port",
"type": "integer"
"name": "args",
"type": "oneof",
"arguments": [
{
"name": "host-port",
"type": "block",
"arguments": [
{
"name": "host",
"type": "string"
},
{
"name": "port",
"type": "integer"
}
]
},
{
"name": "no-one",
"type": "block",
"arguments": [
{
"name": "no",
"type": "pure-token",
"token": "NO"
},
{
"name": "one",
"type": "pure-token",
"token": "ONE"
}
]
}
]
}
],
"reply_schema": {
+34 -6
View File
@@ -19,12 +19,40 @@
],
"arguments": [
{
"name": "host",
"type": "string"
},
{
"name": "port",
"type": "integer"
"name": "args",
"type": "oneof",
"arguments": [
{
"name": "host-port",
"type": "block",
"arguments": [
{
"name": "host",
"type": "string"
},
{
"name": "port",
"type": "integer"
}
]
},
{
"name": "no-one",
"type": "block",
"arguments": [
{
"name": "no",
"type": "pure-token",
"token": "NO"
},
{
"name": "one",
"type": "pure-token",
"token": "ONE"
}
]
}
]
}
],
"reply_schema": {
+1 -1
View File
@@ -150,7 +150,7 @@
"type": "string"
},
{
"description": "GET option is specified, but no object was found ",
"description": "GET option is specified, but no object was found",
"type": "null"
}
]
+9 -1
View File
@@ -117,7 +117,15 @@
"description": "a list of sorted elements",
"type": "array",
"items": {
"type": "string"
"oneOf": [
{
"type": "string"
},
{
"description": "GET option is specified, but no object was found",
"type": "null"
}
]
}
}
}
+7 -3
View File
@@ -40,8 +40,12 @@
#include <fcntl.h>
#endif
#if defined(__APPLE__) && defined(__MAC_OS_X_VERSION_MAX_ALLOWED) && __MAC_OS_X_VERSION_MAX_ALLOWED >= 1060
#define MAC_OS_10_6_DETECTED
#endif
/* Define redis_fstat to fstat or fstat64() */
#if defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
#if defined(__APPLE__) && !defined(MAC_OS_10_6_DETECTED)
#define redis_fstat fstat64
#define redis_stat stat64
#else
@@ -96,7 +100,7 @@
#define HAVE_ACCEPT4 1
#endif
#if (defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)) || defined(__FreeBSD__) || defined(__OpenBSD__) || defined (__NetBSD__)
#if (defined(__APPLE__) && defined(MAC_OS_10_6_DETECTED)) || defined(__FreeBSD__) || defined(__OpenBSD__) || defined (__NetBSD__)
#define HAVE_KQUEUE 1
#endif
@@ -293,7 +297,7 @@ void setproctitle(const char *fmt, ...);
#include <kernel/OS.h>
#define redis_set_thread_title(name) rename_thread(find_thread(0), name)
#else
#if (defined __APPLE__ && defined(MAC_OS_X_VERSION_10_7))
#if (defined __APPLE__ && defined(__MAC_OS_X_VERSION_MAX_ALLOWED) && __MAC_OS_X_VERSION_MAX_ALLOWED >= 1070)
int pthread_setname_np(const char *name);
#include <pthread.h>
#define redis_set_thread_title(name) pthread_setname_np(name)
+2 -1
View File
@@ -2294,7 +2294,8 @@ int sortROGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult
keys = getKeysPrepareResult(result, 1);
keys[0].pos = 1; /* <sort-key> is always present. */
keys[0].flags = CMD_KEY_RO | CMD_KEY_ACCESS;
return 1;
result->numkeys = 1;
return result->numkeys;
}
/* Helper function to extract keys from the SORT command.
+3 -3
View File
@@ -1190,7 +1190,7 @@ static void* getAndSetMcontextEip(ucontext_t *uc, void *eip) {
} \
return old_val; \
} while(0)
#if defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
#if defined(__APPLE__) && !defined(MAC_OS_10_6_DETECTED)
/* OSX < 10.6 */
#if defined(__x86_64__)
GET_SET_RETURN(uc->uc_mcontext->__ss.__rip, eip);
@@ -1199,7 +1199,7 @@ static void* getAndSetMcontextEip(ucontext_t *uc, void *eip) {
#else
GET_SET_RETURN(uc->uc_mcontext->__ss.__srr0, eip);
#endif
#elif defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)
#elif defined(__APPLE__) && defined(MAC_OS_10_6_DETECTED)
/* OSX >= 10.6 */
#if defined(_STRUCT_X86_THREAD_STATE64) && !defined(__i386__)
GET_SET_RETURN(uc->uc_mcontext->__ss.__rip, eip);
@@ -1290,7 +1290,7 @@ void logRegisters(ucontext_t *uc) {
} while(0)
/* OSX */
#if defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)
#if defined(__APPLE__) && defined(MAC_OS_10_6_DETECTED)
/* OSX AMD64 */
#if defined(_STRUCT_X86_THREAD_STATE64) && !defined(__i386__)
serverLog(LL_WARNING,
+5 -10
View File
@@ -1414,30 +1414,25 @@ static int _dictExpandIfNeeded(dict *d)
* table (global setting) or we should avoid it but the ratio between
* elements/buckets is over the "safe" threshold, we resize doubling
* the number of buckets. */
if (!dictTypeExpandAllowed(d))
return DICT_OK;
if ((dict_can_resize == DICT_RESIZE_ENABLE &&
d->ht_used[0] >= DICTHT_SIZE(d->ht_size_exp[0])) ||
(dict_can_resize != DICT_RESIZE_FORBID &&
d->ht_used[0] / DICTHT_SIZE(d->ht_size_exp[0]) > dict_force_resize_ratio))
{
if (!dictTypeExpandAllowed(d))
return DICT_OK;
return dictExpand(d, d->ht_used[0] + 1);
}
return DICT_OK;
}
/* TODO: clz optimization */
/* Our hash table capability is a power of two */
static signed char _dictNextExp(unsigned long size)
{
unsigned char e = DICT_HT_INITIAL_EXP;
if (size <= DICT_HT_INITIAL_SIZE) return DICT_HT_INITIAL_EXP;
if (size >= LONG_MAX) return (8*sizeof(long)-1);
while(1) {
if (((unsigned long)1<<e) >= size)
return e;
e++;
}
return 8*sizeof(long) - __builtin_clzl(size-1);
}
/* Finds and returns the position within the dict where the provided key should
+2
View File
@@ -667,6 +667,7 @@ int performEvictions(void) {
*
* AOF and Output buffer memory will be freed eventually so
* we only care about memory used by the key space. */
enterExecutionUnit(1, 0);
delta = (long long) zmalloc_used_memory();
latencyStartMonitor(eviction_latency);
dbGenericDelete(db,keyobj,server.lazyfree_lazy_eviction,DB_FLAG_KEY_EVICTED);
@@ -679,6 +680,7 @@ int performEvictions(void) {
notifyKeyspaceEvent(NOTIFY_EVICTED, "evicted",
keyobj, db->id);
propagateDeletion(db,keyobj,server.lazyfree_lazy_eviction);
exitExecutionUnit();
postExecutionUnitOperations();
decrRefCount(keyobj);
keys_freed++;
+2
View File
@@ -54,10 +54,12 @@
int activeExpireCycleTryExpire(redisDb *db, dictEntry *de, long long now) {
long long t = dictGetSignedIntegerVal(de);
if (now > t) {
enterExecutionUnit(1, 0);
sds key = dictGetKey(de);
robj *keyobj = createStringObject(key,sdslen(key));
deleteExpiredKeyAndPropagate(db,keyobj);
decrRefCount(keyobj);
exitExecutionUnit();
return 1;
} else {
return 0;
+15 -18
View File
@@ -12115,6 +12115,19 @@ int parseLoadexArguments(RedisModuleString ***module_argv, int *module_argc) {
return REDISMODULE_OK;
}
/* Unregister module-related things, called when moduleLoad fails or moduleUnload. */
void moduleUnregisterCleanup(RedisModule *module) {
moduleFreeAuthenticatedClients(module);
moduleUnregisterCommands(module);
moduleUnsubscribeNotifications(module);
moduleUnregisterSharedAPI(module);
moduleUnregisterUsedAPI(module);
moduleUnregisterFilters(module);
moduleUnsubscribeAllServerEvents(module);
moduleRemoveConfigs(module);
moduleUnregisterAuthCBs(module);
}
/* Load a module and initialize it. On success C_OK is returned, otherwise
* C_ERR is returned. */
int moduleLoad(const char *path, void **module_argv, int module_argc, int is_loadex) {
@@ -12149,11 +12162,7 @@ int moduleLoad(const char *path, void **module_argv, int module_argc, int is_loa
serverLog(LL_WARNING,
"Module %s initialization failed. Module not loaded",path);
if (ctx.module) {
moduleUnregisterCommands(ctx.module);
moduleUnregisterSharedAPI(ctx.module);
moduleUnregisterUsedAPI(ctx.module);
moduleRemoveConfigs(ctx.module);
moduleUnregisterAuthCBs(ctx.module);
moduleUnregisterCleanup(ctx.module);
moduleFreeModuleStructure(ctx.module);
}
moduleFreeContext(&ctx);
@@ -12194,8 +12203,6 @@ int moduleLoad(const char *path, void **module_argv, int module_argc, int is_loa
}
if (post_load_err) {
/* Unregister module auth callbacks (if any exist) that this Module registered onload. */
moduleUnregisterAuthCBs(ctx.module);
moduleUnload(ctx.module->name, NULL);
moduleFreeContext(&ctx);
return C_ERR;
@@ -12253,17 +12260,7 @@ int moduleUnload(sds name, const char **errmsg) {
}
}
moduleFreeAuthenticatedClients(module);
moduleUnregisterCommands(module);
moduleUnregisterSharedAPI(module);
moduleUnregisterUsedAPI(module);
moduleUnregisterFilters(module);
moduleUnregisterAuthCBs(module);
moduleRemoveConfigs(module);
/* Remove any notification subscribers this module might have */
moduleUnsubscribeNotifications(module);
moduleUnsubscribeAllServerEvents(module);
moduleUnregisterCleanup(module);
/* Unload the dynamic library. */
if (dlclose(module->handle) == -1) {
+1
View File
@@ -1657,6 +1657,7 @@ static int cliConnect(int flags) {
redisFree(context);
config.dbnum = 0;
config.in_multi = 0;
config.pubsub_mode = 0;
cliRefreshPrompt();
}
+1
View File
@@ -2250,6 +2250,7 @@ void readSyncBulkPayload(connection *conn) {
}
zfree(server.repl_transfer_tmpfile);
close(server.repl_transfer_fd);
server.repl_transfer_fd = -1;
server.repl_transfer_tmpfile = NULL;
}
+15 -13
View File
@@ -349,20 +349,22 @@ sds sdsResize(sds s, size_t size, int would_regrow) {
* type. */
int use_realloc = (oldtype==type || (type < oldtype && type > SDS_TYPE_8));
size_t newlen = use_realloc ? oldhdrlen+size+1 : hdrlen+size+1;
int alloc_already_optimal = 0;
#if defined(USE_JEMALLOC)
/* je_nallocx returns the expected allocation size for the newlen.
* We aim to avoid calling realloc() when using Jemalloc if there is no
* change in the allocation size, as it incurs a cost even if the
* allocation size stays the same. */
alloc_already_optimal = (je_nallocx(newlen, 0) == zmalloc_size(sh));
#endif
if (use_realloc && !alloc_already_optimal) {
newsh = s_realloc(sh, newlen);
if (newsh == NULL) return NULL;
s = (char*)newsh+oldhdrlen;
} else if (!alloc_already_optimal) {
if (use_realloc) {
int alloc_already_optimal = 0;
#if defined(USE_JEMALLOC)
/* je_nallocx returns the expected allocation size for the newlen.
* We aim to avoid calling realloc() when using Jemalloc if there is no
* change in the allocation size, as it incurs a cost even if the
* allocation size stays the same. */
alloc_already_optimal = (je_nallocx(newlen, 0) == zmalloc_size(sh));
#endif
if (!alloc_already_optimal) {
newsh = s_realloc(sh, newlen);
if (newsh == NULL) return NULL;
s = (char*)newsh+oldhdrlen;
}
} else {
newsh = s_malloc(newlen);
if (newsh == NULL) return NULL;
memcpy((char*)newsh+hdrlen, s, len);
+2 -2
View File
@@ -1,2 +1,2 @@
#define REDIS_VERSION "255.255.255"
#define REDIS_VERSION_NUM 0x00ffffff
#define REDIS_VERSION "7.2.4"
#define REDIS_VERSION_NUM 0x00070204
+14 -3
View File
@@ -1,6 +1,7 @@
#include "redismodule.h"
#include <string.h>
#include <strings.h>
static RedisModuleString *log_key_name;
@@ -92,7 +93,7 @@ int CommandFilter_LogCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int
return REDISMODULE_OK;
}
int CommandFilter_UnfilteredClientdId(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
int CommandFilter_UnfilteredClientId(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
{
if (argc < 2)
return RedisModule_WrongArity(ctx);
@@ -192,7 +193,7 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
if (RedisModule_Init(ctx,"commandfilter",1,REDISMODULE_APIVER_1)
== REDISMODULE_ERR) return REDISMODULE_ERR;
if (argc != 2) {
if (argc != 2 && argc != 3) {
RedisModule_Log(ctx, "warning", "Log key name not specified");
return REDISMODULE_ERR;
}
@@ -219,7 +220,7 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx, unfiltered_clientid_name,
CommandFilter_UnfilteredClientdId, "admin", 1,1,1) == REDISMODULE_ERR)
CommandFilter_UnfilteredClientId, "admin", 1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if ((filter = RedisModule_RegisterCommandFilter(ctx, CommandFilter_CommandFilter,
@@ -229,6 +230,16 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
if ((filter1 = RedisModule_RegisterCommandFilter(ctx, CommandFilter_BlmoveSwap, 0)) == NULL)
return REDISMODULE_ERR;
if (argc == 3) {
const char *ptr = RedisModule_StringPtrLen(argv[2], NULL);
if (!strcasecmp(ptr, "noload")) {
/* This is a hint that we return ERR at the last moment of OnLoad. */
RedisModule_FreeString(ctx, log_key_name);
if (retained) RedisModule_FreeString(NULL, retained);
return REDISMODULE_ERR;
}
}
return REDISMODULE_OK;
}
+13 -3
View File
@@ -33,6 +33,7 @@
#include "redismodule.h"
#include <stdio.h>
#include <string.h>
#include <strings.h>
#include <assert.h>
/* We need to store events to be able to test and see what we got, and we can't
@@ -407,9 +408,6 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_ERR; \
}
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
if (RedisModule_Init(ctx,"testhook",1,REDISMODULE_APIVER_1)
== REDISMODULE_ERR) return REDISMODULE_ERR;
@@ -471,6 +469,18 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
if (RedisModule_CreateCommand(ctx,"hooks.pexpireat", cmdKeyExpiry,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (argc == 1) {
const char *ptr = RedisModule_StringPtrLen(argv[0], NULL);
if (!strcasecmp(ptr, "noload")) {
/* This is a hint that we return ERR at the last moment of OnLoad. */
RedisModule_FreeDict(ctx, event_log);
RedisModule_FreeDict(ctx, removed_event_log);
RedisModule_FreeDict(ctx, removed_subevent_type);
RedisModule_FreeDict(ctx, removed_expiry_log);
return REDISMODULE_ERR;
}
}
return REDISMODULE_OK;
}
+11 -3
View File
@@ -36,6 +36,7 @@
#include "redismodule.h"
#include <stdio.h>
#include <string.h>
#include <strings.h>
#include <unistd.h>
ustime_t cached_time = 0;
@@ -318,9 +319,6 @@ static int cmdGetDels(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
/* This function must be present on each Redis module. It is used in order to
* register the commands into the Redis server. */
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
if (RedisModule_Init(ctx,"testkeyspace",1,REDISMODULE_APIVER_1) == REDISMODULE_ERR){
return REDISMODULE_ERR;
}
@@ -405,6 +403,16 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_ERR;
}
if (argc == 1) {
const char *ptr = RedisModule_StringPtrLen(argv[0], NULL);
if (!strcasecmp(ptr, "noload")) {
/* This is a hint that we return ERR at the last moment of OnLoad. */
RedisModule_FreeDict(ctx, loaded_event_log);
RedisModule_FreeDict(ctx, module_event_log);
return REDISMODULE_ERR;
}
}
return REDISMODULE_OK;
}
+67
View File
@@ -90,6 +90,10 @@ static int KeySpace_NotificationEvicted(RedisModuleCtx *ctx, int type, const cha
return REDISMODULE_OK; /* do not count the evicted key */
}
if (strncmp(key_str, "before_evicted", 14) == 0) {
return REDISMODULE_OK; /* do not count the before_evicted key */
}
RedisModuleString *new_key = RedisModule_CreateString(NULL, "evicted", 7);
RedisModule_AddPostNotificationJob(ctx, KeySpace_PostNotificationString, new_key, KeySpace_PostNotificationStringFreePD);
return REDISMODULE_OK;
@@ -186,6 +190,55 @@ static int KeySpace_PostNotificationsAsyncSet(RedisModuleCtx *ctx, RedisModuleSt
return REDISMODULE_OK;
}
typedef struct KeySpace_EventPostNotificationCtx {
RedisModuleString *triggered_on;
RedisModuleString *new_key;
} KeySpace_EventPostNotificationCtx;
static void KeySpace_ServerEventPostNotificationFree(void *pd) {
KeySpace_EventPostNotificationCtx *pn_ctx = pd;
RedisModule_FreeString(NULL, pn_ctx->new_key);
RedisModule_FreeString(NULL, pn_ctx->triggered_on);
RedisModule_Free(pn_ctx);
}
static void KeySpace_ServerEventPostNotification(RedisModuleCtx *ctx, void *pd) {
REDISMODULE_NOT_USED(ctx);
KeySpace_EventPostNotificationCtx *pn_ctx = pd;
RedisModuleCallReply* rep = RedisModule_Call(ctx, "lpush", "!ss", pn_ctx->new_key, pn_ctx->triggered_on);
RedisModule_FreeCallReply(rep);
}
static void KeySpace_ServerEventCallback(RedisModuleCtx *ctx, RedisModuleEvent eid, uint64_t subevent, void *data) {
REDISMODULE_NOT_USED(eid);
REDISMODULE_NOT_USED(data);
if (subevent > 3) {
RedisModule_Log(ctx, "warning", "Got an unexpected subevent '%ld'", subevent);
return;
}
static const char* events[] = {
"before_deleted",
"before_expired",
"before_evicted",
"before_overwritten",
};
const RedisModuleString *key_name = RedisModule_GetKeyNameFromModuleKey(((RedisModuleKeyInfo*)data)->key);
const char *key_str = RedisModule_StringPtrLen(key_name, NULL);
for (int i = 0 ; i < 4 ; ++i) {
const char *event = events[i];
if (strncmp(key_str, event , strlen(event)) == 0) {
return; /* don't log any event on our tracking keys */
}
}
KeySpace_EventPostNotificationCtx *pn_ctx = RedisModule_Alloc(sizeof(*pn_ctx));
pn_ctx->triggered_on = RedisModule_HoldString(NULL, (RedisModuleString*)key_name);
pn_ctx->new_key = RedisModule_CreateString(NULL, events[subevent], strlen(events[subevent]));
RedisModule_AddPostNotificationJob(ctx, KeySpace_ServerEventPostNotification, pn_ctx, KeySpace_ServerEventPostNotificationFree);
}
/* This function must be present on each Redis module. It is used in order to
* register the commands into the Redis server. */
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
@@ -200,6 +253,14 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_ERR;
}
int with_key_events = 0;
if (argc >= 1) {
const char *arg = RedisModule_StringPtrLen(argv[0], 0);
if (strcmp(arg, "with_key_events") == 0) {
with_key_events = 1;
}
}
RedisModule_SetModuleOptions(ctx, REDISMODULE_OPTIONS_ALLOW_NESTED_KEYSPACE_NOTIFICATIONS);
if(RedisModule_SubscribeToKeyspaceEvents(ctx, REDISMODULE_NOTIFY_STRING, KeySpace_NotificationString) != REDISMODULE_OK){
@@ -222,6 +283,12 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_ERR;
}
if (with_key_events) {
if(RedisModule_SubscribeToServerEvent(ctx, RedisModuleEvent_Key, KeySpace_ServerEventCallback) != REDISMODULE_OK){
return REDISMODULE_ERR;
}
}
if (RedisModule_CreateCommand(ctx, "postnotification.async_set", KeySpace_PostNotificationsAsyncSet,
"write", 0, 0, 0) == REDISMODULE_ERR){
return REDISMODULE_ERR;
+27
View File
@@ -314,6 +314,14 @@ start_server {tags {"modules"}} {
r lpush l a
assert_equal [$rd read] {OK}
# Explanation of the first multi exec block:
# {lpop l} - pop the value by our blocking command 'blpop_and_set_multiple_keys'
# {set string_foo 1} - the action of our blocking command 'blpop_and_set_multiple_keys'
# {set string_bar 2} - the action of our blocking command 'blpop_and_set_multiple_keys'
# {incr string_changed{string_foo}} - post notification job that was registered when 'string_foo' changed
# {incr string_changed{string_bar}} - post notification job that was registered when 'string_bar' changed
# {incr string_total} - post notification job that was registered when string_changed{string_foo} changed
# {incr string_total} - post notification job that was registered when string_changed{string_bar} changed
assert_replication_stream $repl {
{select *}
{lpush l a}
@@ -355,6 +363,25 @@ start_server {tags {"modules"}} {
r lpush l a
assert_equal [$rd read] {OK}
# Explanation of the first multi exec block:
# {lpop l} - pop the value by our blocking command 'blpop_and_set_multiple_keys'
# {set string_foo 1} - the action of our blocking command 'blpop_and_set_multiple_keys'
# {set string_bar 2} - the action of our blocking command 'blpop_and_set_multiple_keys'
# {incr string_changed{string_foo}} - post notification job that was registered when 'string_foo' changed
# {incr string_changed{string_bar}} - post notification job that was registered when 'string_bar' changed
# {incr string_total} - post notification job that was registered when string_changed{string_foo} changed
# {incr string_total} - post notification job that was registered when string_changed{string_bar} changed
#
# Explanation of the second multi exec block:
# {lpop l} - pop the value by our blocking command 'blpop_and_set_multiple_keys'
# {del string_foo} - lazy expiration of string_foo when 'blpop_and_set_multiple_keys' tries to write to it.
# {set string_foo 1} - the action of our blocking command 'blpop_and_set_multiple_keys'
# {set string_bar 2} - the action of our blocking command 'blpop_and_set_multiple_keys'
# {incr expired} - the post notification job, registered after string_foo got expired
# {incr string_changed{string_foo}} - post notification job triggered when we set string_foo
# {incr string_changed{string_bar}} - post notification job triggered when we set string_bar
# {incr string_total} - post notification job triggered when we incr 'string_changed{string_foo}'
# {incr string_total} - post notification job triggered when we incr 'string_changed{string_bar}'
assert_replication_stream $repl {
{select *}
{lpush l a}
+12 -2
View File
@@ -95,7 +95,7 @@ start_server {tags {"modules"}} {
test "Unload the module - commandfilter" {
assert_equal {OK} [r module unload commandfilter]
}
}
}
test {RM_CommandFilterArgInsert and script argv caching} {
# coverage for scripts calling commands that expand the argv array
@@ -162,4 +162,14 @@ test {Filtering based on client id} {
$rr close
}
}
}
start_server {} {
test {OnLoad failure will handle un-registration} {
catch {r module load $testmodule log-key 0 noload}
r set mykey @log
assert_equal [r lrange log-key 0 -1] {}
r rpush mylist elem1 @delme elem2
assert_equal [r lrange mylist 0 -1] {elem1 @delme elem2}
}
}
+8
View File
@@ -310,4 +310,12 @@ tags "modules" {
assert_equal [string match {*module-event-shutdown*} [exec tail -5 < $replica_stdout]] 1
}
}
start_server {} {
test {OnLoad failure will handle un-registration} {
catch {r module load $testmodule noload}
r flushall
r ping
}
}
}
+13
View File
@@ -102,4 +102,17 @@ tags "modules" {
assert_equal {OK} [r set x 1 EX 1]
}
}
start_server {} {
test {OnLoad failure will handle un-registration} {
catch {r module load $testmodule noload}
r set x 1
r hset y f v
r lpush z 1 2 3
r sadd p 1 2 3
r zadd t 1 f1 2 f2
r xadd s * f v
r ping
}
}
}
+21 -7
View File
@@ -1,7 +1,8 @@
set testmodule [file normalize tests/modules/postnotifications.so]
tags "modules" {
start_server [list overrides [list loadmodule "$testmodule"]] {
start_server {} {
r module load $testmodule with_key_events
test {Test write on post notification callback} {
set repl [attach_to_replication_stream]
@@ -9,11 +10,12 @@ tags "modules" {
r set string_x 1
assert_equal {1} [r get string_changed{string_x}]
assert_equal {1} [r get string_total]
r set string_x 2
assert_equal {2} [r get string_changed{string_x}]
assert_equal {2} [r get string_total]
# the {lpush before_overwritten string_x} is a post notification job registered when 'string_x' was overwritten
assert_replication_stream $repl {
{multi}
{select *}
@@ -23,6 +25,7 @@ tags "modules" {
{exec}
{multi}
{set string_x 2}
{lpush before_overwritten string_x}
{incr string_changed{string_x}}
{incr string_total}
{exec}
@@ -37,7 +40,7 @@ tags "modules" {
assert_equal {OK} [r postnotification.async_set]
assert_equal {1} [r get string_changed{string_x}]
assert_equal {1} [r get string_total]
assert_replication_stream $repl {
{multi}
{select *}
@@ -63,12 +66,14 @@ tags "modules" {
fail "Failed waiting for x to expired"
}
# the {lpush before_expired x} is a post notification job registered before 'x' got expired
assert_replication_stream $repl {
{select *}
{set x 1}
{pexpireat x *}
{multi}
{del x}
{lpush before_expired x}
{incr expired}
{exec}
}
@@ -85,12 +90,14 @@ tags "modules" {
after 10
assert_equal {} [r get x]
# the {lpush before_expired x} is a post notification job registered before 'x' got expired
assert_replication_stream $repl {
{select *}
{set x 1}
{pexpireat x *}
{multi}
{del x}
{lpush before_expired x}
{incr expired}
{exec}
}
@@ -108,6 +115,7 @@ tags "modules" {
after 10
assert_equal {OK} [r set read_x 1]
# the {lpush before_expired x} is a post notification job registered before 'x' got expired
assert_replication_stream $repl {
{select *}
{set x 1}
@@ -115,6 +123,7 @@ tags "modules" {
{multi}
{set read_x 1}
{del x}
{lpush before_expired x}
{incr expired}
{exec}
}
@@ -143,16 +152,18 @@ tags "modules" {
r flushall
set repl [attach_to_replication_stream]
r set x 1
r config set maxmemory-policy allkeys-random
r config set maxmemory-policy allkeys-random
r config set maxmemory 1
assert_error {OOM *} {r set y 1}
# the {lpush before_evicted x} is a post notification job registered before 'x' got evicted
assert_replication_stream $repl {
{select *}
{set x 1}
{multi}
{del x}
{lpush before_evicted x}
{incr evicted}
{exec}
}
@@ -164,7 +175,8 @@ tags "modules" {
set testmodule2 [file normalize tests/modules/keyspace_events.so]
tags "modules" {
start_server [list overrides [list loadmodule "$testmodule"]] {
start_server {} {
r module load $testmodule with_key_events
r module load $testmodule2
test {Test write on post notification callback} {
set repl [attach_to_replication_stream]
@@ -172,7 +184,7 @@ tags "modules" {
r set string_x 1
assert_equal {1} [r get string_changed{string_x}]
assert_equal {1} [r get string_total]
r set string_x 2
assert_equal {2} [r get string_changed{string_x}]
assert_equal {2} [r get string_total]
@@ -181,6 +193,7 @@ tags "modules" {
assert_equal {1} [r get string_changed{string1_x}]
assert_equal {3} [r get string_total]
# the {lpush before_overwritten string_x} is a post notification job registered before 'string_x' got overwritten
assert_replication_stream $repl {
{multi}
{select *}
@@ -190,6 +203,7 @@ tags "modules" {
{exec}
{multi}
{set string_x 2}
{lpush before_overwritten string_x}
{incr string_changed{string_x}}
{incr string_total}
{exec}
@@ -202,4 +216,4 @@ tags "modules" {
close_replication_stream $repl
}
}
}
}
+8 -2
View File
@@ -75,12 +75,14 @@ start_server {
assert_equal [lsort -integer $result] [r sort tosort GET #]
} {} {cluster:skip}
test "SORT GET <const>" {
foreach command {SORT SORT_RO} {
test "$command GET <const>" {
r del foo
set res [r sort tosort GET foo]
set res [r $command tosort GET foo]
assert_equal 16 [llength $res]
foreach item $res { assert_equal {} $item }
} {} {cluster:skip}
}
test "SORT GET (key and hash) with sanity check" {
set l1 [r sort tosort GET # GET weight_*]
@@ -109,6 +111,10 @@ start_server {
test "SORT extracts STORE correctly" {
r command getkeys sort abc store def
} {abc def}
test "SORT_RO get keys" {
r command getkeys sort_ro abc
} {abc}
test "SORT extracts multiple STORE correctly" {
r command getkeys sort abc store invalid store stillbad store def
+31
View File
@@ -140,6 +140,37 @@ tags {"wait aof network external:skip"} {
assert_error {ERR WAITAOF cannot be used when numlocal is set but appendonly is disabled.} {$master waitaof 1 0 0}
}
test {WAITAOF local if AOFRW was postponed} {
r config set appendfsync everysec
# turn off AOF
r config set appendonly no
# create an RDB child that takes a lot of time to run
r set x y
r config set rdb-key-save-delay 100000000 ;# 100 seconds
r bgsave
assert_equal [s rdb_bgsave_in_progress] 1
# turn on AOF
r config set appendonly yes
assert_equal [s aof_rewrite_scheduled] 1
# create a write command (to increment master_repl_offset)
r set x y
# reset save_delay and kill RDB child
r config set rdb-key-save-delay 0
catch {exec kill -9 [get_child_pid 0]}
# wait for AOF (will unblock after AOFRW finishes)
assert_equal [r waitaof 1 0 10000] {1 0}
# make sure AOFRW finished
assert_equal [s aof_rewrite_in_progress] 0
assert_equal [s aof_rewrite_scheduled] 0
}
$master config set appendonly yes
waitForBgrewriteaof $master