Compare commits

...
72 Commits
Author SHA1 Message Date
YaacovHazan 4b03ddfdac Redis 7.4.6 2025-10-02 23:29:48 +03:00
Ozan TezcanandYaacovHazan 0475e953b9 Lua script may lead to integer overflow and potential RCE (CVE-2025-46817) 2025-10-02 23:17:11 +03:00
Ozan TezcanandYaacovHazan cbde20410c LUA out-of-bound read (CVE-2025-46819) 2025-10-02 23:17:11 +03:00
Ozan TezcanandYaacovHazan 9b73e7dbcd Lua script can be executed in the context of another user (CVE-2025-46818) 2025-10-02 23:03:38 +03:00
Mincho PaskalevandYaacovHazan 155519b195 Lua script may lead to remote code execution (CVE-2025-49844) 2025-10-02 23:03:33 +03:00
debing.sun 663a471ba7 Fix defrag issues for pubsub and lua (#14330)
This PR fixes two defrag issues.

1. Fix a use-after-free issue caused by updating dictionary keys after a
pubsub channel is reallocated.
This issue was introduced by https://github.com/redis/redis/pull/13058

2. Fix potential use-after-free for lua during AOF loading with defrag
This issue was introduced by https://github.com/redis/redis/issues/13058
   This fix follows https://github.com/redis/redis/pull/14319
This PR updates the LuaScript LRU list before script execution to
prevent accessing a potentially invalidated pointer after long-running
scripts.
2025-09-30 22:09:56 +08:00
debing.sun 7a51095925 Fix crash during lua script defrag (#14319)
This PR fixes two crashes due to the defragmentation of the Lua script,
which were by https://github.com/redis/redis/pull/13108

1. During long-running Lua script execution, active defragmentation may
be triggered, causing the luaScript structure to be reallocated to a new
memory location, then we access `l->node`(may be reallocatedd) after
script execution to update the Lua LRU list.
In this PR, we don't defrag during blocked scripts, so we don't mess up
the LRU update when the script ends.
   Note that defrag is now only permitted during loading.
This PR also reverts the changes made by
https://github.com/redis/redis/pull/14274.

2. Forgot to update the Lua LUR list node's value.
Since `lua_scripts_lru_list` node stores a pointer to the `lua_script`'s
key, we also need to update `node->value` when the key is reallocated.
In this PR, after performing defragmentation on a Lua script, if the
script is in the LRU list, its reference in the LRU list will be
unconditionally updated.
2025-09-30 22:09:56 +08:00
0201cadf9e Fix 'Client output buffer hard limit is enforced' test causing infinite loop (#13934)
This PR fixes an issue in the CI test for client-output-buffer-limit,
which was causing an infinite loop when running on macOS 15.4.

### Problem

This test start two clients, R and R1:
```c
R1 subscribe foo
R publish foo bar
```

When R executes `PUBLISH foo bar`, the server first stores the message
`bar` in R1‘s buf. Only when the space in buf is insufficient does it
call `_addReplyProtoToList`.
Inside this function, `closeClientOnOutputBufferLimitReached` is invoked
to check whether the client’s R1 output buffer has reached its
configured limit.
On macOS 15.4, because the server writes to the client at a high speed,
R1’s buf never gets full. As a result,
`closeClientOnOutputBufferLimitReached` in the test is never triggered,
causing the test to never exit and fall into an infinite loop.

---------

Co-authored-by: debing.sun <debing.sun@redis.com>
2025-09-30 16:40:53 +03:00
debing.sunandYaacovHazan 5e6a000136 Fix timeout issues in memefficiency.tcl (#14231)
Follow https://github.com/redis/redis/pull/14217
Fix https://github.com/redis/redis/issues/14196

Fix two other issues that might cause timeouts due to command writing
via pipe.
2025-09-30 15:13:23 +03:00
debing.sunandYaacovHazan 2ea088f3f3 Fix some daily CI issues (#14217)
1) Fix the timeout of `Active defrag big keys: standalone`
Using a pipe to write commands may cause the write to block if the read
buffer becomes full.

2) Fix the failure of `Main db not affected when fail to diskless load`
test
If the master was killed in slow environment, then after
`cluster-node-timeout` (3s in our test), running keyspace commands on
the replica will get a CLUSTERDOWN error.

3) Fix the failure of `Test shutdown hook` test
ASAN can intercept a signal, so I guess that when we send SIGCONT after
SIGTERM to kill the server, it might start doing some work again,
causing the process to close very slowly.
2025-09-30 15:13:17 +03:00
a8e2a0386f CLIENT UNBLOCK should't be able to unpause paused clients (#14164)
This PR is based on https://github.com/valkey-io/valkey/pull/2117

When a client is blocked by something like `CLIENT PAUSE`, we should not
allow `CLIENT UNBLOCK timeout` to unblock it, since some blocking types
does not has the timeout callback, it will trigger a panic in the core,
people should use `CLIENT UNPAUSE` to unblock it.

Also using `CLIENT UNBLOCK error` is not right, it will return a
UNBLOCKED error to the command, people don't expect a `SET` command to
get an error.

So in this commit, in these cases, we will return 0 to `CLIENT UNBLOCK`
to indicate the unblock is fail. The reason is that we assume that if a
command doesn't expect to be timedout, it also doesn't expect to be
unblocked by `CLIENT UNBLOCK`.

The old behavior of the following command will trigger panic in timeout
and get UNBLOCKED error in error. Under the new behavior, client unblock
will get the result of 0.
```
client 1> client pause 100000 write
client 2> set x x

client 1> client unblock 2 timeout
or
client 1> client unblock 2 error
```

---------

Signed-off-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
2025-09-30 15:13:12 +03:00
9ec175af3d Only mark the client reprocessing flag when unblocked on keys (#14165)
This PR is based on https://github.com/valkey-io/valkey/pull/2109

When we refactored the blocking framework we introduced the client
reprocessing infrastructure. In cases the client was blocked on keys, it
will attempt to reprocess the command. One challenge was to keep track
of the command timeout, since we are reprocessing and do not want to
re-register the client with a fresh timeout each time. The solution was
to consider the client reprocessing flag when the client is
blockedOnKeys:

```c
    if (!(c->flags & CLIENT_REPROCESSING_COMMAND)) {
        /* If the client is re-processing the command, we do not set the timeout
         * because we need to retain the client's original timeout. */
        c->bstate.timeout = timeout;
    }
```

However, this introduced a new issue. There are cases where the client
will consecutive blocking of different types for example:
```
CLIENT PAUSE 10000 ALL
BZPOPMAX zset 1
```
would have the client blocked on the zset endlessly if nothing will be
written to it.

**Credits to @uriyage for locating this with his fuzzer testing**

The suggested solution is to only flag the client when it is
specifically unblocked on keys.

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
2025-09-30 15:13:00 +03:00
debing.sunandYaacovHazan b9771292e2 Update debian buster sources to archive.debian.org (#14197)
http://deb.debian.org/debian no longer serves Debian 10 (buster); it has
been moved to http://archive.debian.org/debian.
2025-09-30 15:11:15 +03:00
803685a41b Ensure empty error tables in scripts don't crash (#14163)
This PR is based on: https://github.com/valkey-io/valkey/pull/2229

When calling the command `EVAL error{} 0`, Redis crashes with the
following stack trace. This patch ensures we never leave the
`err_info.msg` field null when we fail to extract a proper error
message.

---------

Signed-off-by: Fusl <fusl@meo.ws>
Signed-off-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Fusl <fusl@meo.ws>
Co-authored-by: Binbin <binloveplay1314@qq.com>
2025-09-30 15:10:54 +03:00
Oran AgraandYaacovHazan 4b55309a5f Make Active defrag big list test much faster (#14157)
it aims to create listpacks of 500k, but did that with 5 insertions of
100k each, instead do that in one insertion, reducing the need for
listpack gradual growth, and reducing the number of commands we send.
apparently there are some stalls reading the replies of the commands,
specifically in GH actions, reducing the number of commands seems to
eliminate that.
2025-09-30 15:04:44 +03:00
Yuan WangandGitHub 1d96a95d75 Fix HINCRBYFLOAT removes field expiration on replica (#14227)
Fixes https://github.com/redis/redis/issues/14218

Before, we replicate HINCRBYFLOAT as an HSET command with the final
value in order to make sure that differences in float precision or
formatting will not create differences in replicas or after an AOF
restart.
However, on the replica side, if the field has an expiration time, HSET
will remove it, even though the master retains it. This leads to
inconsistencies between the master and the replica.

For Redis 8.0 and above, [PR
#14224](https://github.com/redis/redis/pull/14224) has fixed this issue.
However, Redis 7.4 does not support the HSETEX command. To address this,
HINCRBYFLOAT will be replicated as a simple HSET if no expiration is
set. If an expiration is specified, it will be replicated as a
MULTI/EXEC containing HSET and HPEXPIREAT commands.
2025-07-30 19:02:19 +08:00
YaacovHazanandYaacovHazan 7e0f533932 Redis 7.4.5 2025-07-06 14:59:57 +03:00
Mincho PaskalevandYaacovHazan dfa5b12627 Remove string cat usage in tcl tests in order to support tcl8.5 2025-07-06 14:59:57 +03:00
Ozan TezcanandYaacovHazan 39ea429d03 Retry accept() even if accepted connection reports an error (CVE-2025-48367)
In case of accept4() returns an error, we should check errno value and decide if we should retry accept4() without waiting next event loop iteration.
2025-07-06 14:59:57 +03:00
8658e2aeb9 Fix out of bounds write in hyperloglog commands (CVE-2025-32023)
Co-authored-by: oranagra <oran@redislabs.com>
2025-07-06 14:59:57 +03:00
yzc-yzcandYaacovHazan 4c6ffd5125 Fix negative offset issue for ZRANGEBY[SCORE|LEX] command (#14043)
Fix #13952

This PR ensures that ZRANGE_SCORE/LEX command with a negative offset
will return empty.
2025-07-06 14:59:57 +03:00
YaacovHazanandYaacovHazan 08961e16db Redis 7.4.4 2025-05-27 15:38:26 +03:00
YaacovHazanandYaacovHazan e93df78b8b Check length of AOF file name in redis-check-aof (CVE-2025-27151)
Ensure that the length of the input file name does not exceed PATH_MAX
2025-05-27 15:38:26 +03:00
f9530e77d1 Correctly update kvstore overhead after emptying or releasing dict (#13984)
Close #13973

This PR fixed two bugs.
1)  `overhead_hashtable_lut` isn't updated correctly
    This bug was introduced by https://github.com/redis/redis/pull/12913
We only update `overhead_hashtable_lut` at the beginning and end of
rehashing, but we forgot to update it when a dict is emptied or
released.

This PR introduces a new `bucketChanged` callback to track the change
changes in the bucket size.
Now, `rehashingStarted` and `rehashingCompleted` callbacks are no longer
responsible for bucket changes, but are entirely handled by
`bucketChanged`, this can also avoid that we need to register three
callbacks to track the change of bucket size, now only one is needed.

In most cases, it will be triggered together with `rehashingStarted` or
`rehashingCompleted`,
except when a dict is being emptied or released, in these cases, even if
the dict is not rehashing, we still need to subtract its current size.

On the other hand, `overhead_hashtable_lut` was duplicated with
`bucket_count`, so we remove `overhead_hashtable_lut` and use
`bucket_count` instead.

Note that this bug only happens with cluster mode, because we don't use
KVSTORE_FREE_EMPTY_DICTS without cluster.

2) The size of `dict_size_index` repeatedly counted in terms of memory
usage.
`dict_size_index` is created at startup, so its memory usage has been
counted into `used_memory_startup`.
However, when we want to count the overhead, we repeat the calculation,
which may cause the overhead to exceed the total memory usage.

---------

Co-authored-by: Yuan Wang <yuan.wang@redis.com>
2025-05-27 15:38:26 +03:00
Vitah LinandYaacovHazan 9d11a4f862 Fix tls port update not reflected in CLUSTER SLOTS (#13966)
### Problem 

A previous PR (https://github.com/redis/redis/pull/13932) fixed the TCP
port issue in CLUSTER SLOTS, but it seems the handling of the TLS port
was overlooked.

There is this comment in the `addNodeToNodeReply` function in the
`cluster.c` file:
```c
    /* Report TLS ports to TLS client, and report non-TLS port to non-TLS client. */
    addReplyLongLong(c, clusterNodeClientPort(node, shouldReturnTlsInfo()));
    addReplyBulkCBuffer(c, clusterNodeGetName(node), CLUSTER_NAMELEN);
```

### Fixed 

This PR fixes the TLS port issue and adds relevant tests.
2025-05-27 15:38:26 +03:00
nesty92andYaacovHazan 5f317a632c Fix incorrect lag due to trimming stream via XTRIM or XADD command (#13958)
This PR fix the lag calculation by ensuring that when consumer group's last_id
is behind the first entry, the consumer group's entries read is considered
invalid and recalculated from the start of the stream

Supplement to PR #13473 

Close #13957

Signed-off-by: Ernesto Alejandro Santana Hidalgo <ernesto.alejandrosantana@gmail.com>
2025-05-27 15:38:26 +03:00
Stav-LeviandYaacovHazan 38e9e8ed38 Fix port update not reflected in CLUSTER SLOTS (#13932)
Close https://github.com/redis/redis/issues/13892 
config set port cmd updates server.port. cluster slot retrieves
information about cluster slots and their associated nodes. the fix
updates this info when config set port cmd is done, so cluster slots cmd
returns the right value.
2025-05-27 15:38:26 +03:00
YaacovHazanandGitHub 24080114cc Redis 7.4.3 (#13963) 2025-04-23 14:54:51 +03:00
YaacovHazan b9405a7d9d Redis 7.4.3 2025-04-23 10:26:24 +03:00
YaacovHazan a93a319ba2 Limiting output buffer for unauthenticated client (CVE-2025-21605)
For unauthenticated clients the output buffer is limited to prevent
them from abusing it by not reading the replies
2025-04-23 10:18:06 +03:00
YaacovHazan c214e0f1eb Avoid sanitizer warning for stable CI 2025-04-22 14:53:32 +03:00
Vitah LinandYaacovHazan c2289ea402 Fix oldTC CI dk.archive.ubuntu.com could not connect (#13961) 2025-04-22 12:16:02 +03:00
JasonandYaacovHazan 3138dbac1b Ignore shardId updates from replica nodes (#13877)
Close https://github.com/redis/redis/issues/13868

This bug was introduced by https://github.com/redis/redis/pull/13468

## Issue
To maintain compatibility with older versions that do not support
shardid, when a replica passes a shardid, we also update the master’s
shardid accordingly.

However, when both the master and replica support shardid, an issue
arises: in one moment, the master may pass a shardid, causing us to
update both the master and all its replicas to match the master’s
shardid. But if the replica later passes a different shardid, we would
then update the master’s shardid again, leading to continuous changes in
shardid.

## Solution
Regardless of the situation, we always ensure that the replica’s shardid
remains consistent with the master’s shardid.
2025-04-22 10:50:48 +03:00
3415d7ae63 Fix defrag when type/encoding changes during scan (#13883)
This PR is based on: https://github.com/valkey-io/valkey/pull/1801

[SoftlyRaining](https://github.com/SoftlyRaining) was hunting for defrag
bugs with Jim and found a couple of improvements to make. Jim pointed
out that in several of the callbacks, if the encoding were to change it
simply returns without doing anything to `cursor` to make it reach 0,
meaning that it would continue no-op working on that item without making
any progress. Type and encoding can change while the defrag scan is in
progress if the value is mutated or replaced by something else with the
same key.

---------

Signed-off-by: Rain Valentine <rsg000@gmail.com>
Co-authored-by: Rain Valentine <rsg000@gmail.com>
2025-04-22 10:50:37 +03:00
6717df1388 Fix potential infinite loop of RANDOMKEY during client pause (#13863)
The bug mentioned in this
[#13862](https://github.com/redis/redis/issues/13862) has been fixed.

---------

Signed-off-by: li-benson <1260437731@qq.com>
Signed-off-by: youngmore1024 <youngmore1024@outlook.com>
Co-authored-by: youngmore1024 <youngmore1024@outlook.com>
2025-04-22 10:50:21 +03:00
949d4efa36 Fix crash during SLAVEOF when clients are blocked on lazyfree (#13853)
After https://github.com/redis/redis/pull/13167, when a client calls
`FLUSHDB` command, we still async empty database, and the client was
blocked until the lazyfree completes.

1) If another client calls `SLAVEOF` command during this time, the
server will unblock all blocked clients, including those blocked by the
lazyfree. However, when unblocking a lazyfree blocked client, we forgot
to call `updateStatsOnUnblock()`, which ultimately triggered the
following assertion.

2) If a client blocked by Lazyfree is unblocked midway, and at this
point the `bio_comp_list` has already received the completion
notification for the bio, we might end up processing a client that has
already been unblocked in `flushallSyncBgDone()`. Therefore, we need to
filter it out.

---------

Co-authored-by: oranagra <oran@redislabs.com>
2025-04-22 10:29:35 +03:00
9b45a148f2 Fix wrong behavior of XREAD + after last entry of stream have been removed (#13632)
Close #13628

This PR changes behavior of special `+` id of XREAD command. Now it uses
`streamLastValidID` to find last entry instead of `last_id` field of
stream object.
This PR adds test for the issue.

**Notes**

Initial idea to update `last_id` while executing XDEL seems to be wrong.
`last_id` is used to strore last generated id and not id of last entry.

---------

Co-authored-by: debing.sun <debing.sun@redis.com>
Co-authored-by: guybe7 <guy.benoish@redislabs.com>
2025-04-22 10:25:22 +03:00
Yuan WangandYaacovHazan 5d7b60e117 Fix wrongly updating fsynced_reploff_pending when appendfsync=everysecond (#13793)
```
if (server.aof_fsync == AOF_FSYNC_EVERYSEC &&
    server.aof_last_incr_fsync_offset != server.aof_last_incr_size &&
    server.mstime - server.aof_last_fsync >= 1000 &&
    !(sync_in_progress = aofFsyncInProgress())) {
    goto try_fsync;
```
In https://github.com/redis/redis/pull/12622, when when
appendfsync=everysecond, if redis has written some data to AOF but not
`fsync`, and less than 1 second has passed since the last `fsync `,
redis will won't fsync AOF, but we will update `
fsynced_reploff_pending`, so it cause the `WAITAOF` to return
prematurely.

this bug is introduced in https://github.com/redis/redis/pull/12622,
from 7.4

The bug fix
https://github.com/redis/redis/pull/13793/commits/1bd6688bcae4add51dc829d96776359dfa39b100
is just as follows:
```diff
diff --git a/src/aof.c b/src/aof.c
index 8ccd8d8f8..521b30449 100644
--- a/src/aof.c
+++ b/src/aof.c
@@ -1096,8 +1096,11 @@ void flushAppendOnlyFile(int force) {
              * in which case master_repl_offset will increase but fsynced_reploff_pending won't be updated
              * (because there's no reason, from the AOF POV, to call fsync) and then WAITAOF may wait on
              * the higher offset (which contains data that was only propagated to replicas, and not to AOF) */
-            if (!sync_in_progress && server.aof_fsync != AOF_FSYNC_NO)
+            if (server.aof_last_incr_fsync_offset == server.aof_last_incr_size &&
+                !(sync_in_progress = aofFsyncInProgress()))
+            {
                 atomicSet(server.fsynced_reploff_pending, server.master_repl_offset);
+            }
             return;
```
Additionally, we slightly refactored fsync AOF to make it simpler, as
https://github.com/redis/redis/pull/13793/commits/584f008d1c39b90ae1d4862a313e04e3b426b136
2025-04-22 10:25:15 +03:00
Mingyi KangandYaacovHazan 7a9584b15d Bump actions/upload-artifact from 3 to 4 (#13780)
Update `upload-artifact` from v3 to v4 to avoid the failure of `External
Server Tests` (I encountered this error when opening
[#13779](https://github.com/redis/redis/pull/13779)):

> Error: This request has been automatically failed because it uses a
deprecated version of `actions/upload-artifact: v3`. Learn more:
https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/
2025-04-22 10:25:07 +03:00
debing.sunandYaacovHazan 98077651bf Update info.tcl test to revert client output limits sooner (#13738)
This PR is based on: https://github.com/valkey-io/valkey/pull/1462

We set the client output buffer limits to 10 bytes, and then execute
info stats which produces more than 10 bytes of output, which can cause
that command to throw an error.

I'm not sure why it wasn't consistently erroring before, might have been
some change related to the ubuntu upgrade though.

failed CI:
https://github.com/redis/redis/actions/runs/12738281720/job/35500381299

------
Co-authored-by: Madelyn Olson
[madelyneolson@gmail.com](mailto:madelyneolson@gmail.com)
2025-04-22 10:25:00 +03:00
b2a8c675a7 Fix memory leak of jemalloc tcache on function flush command (#13661)
Starting from https://github.com/redis/redis/pull/13133, we allocate a
jemalloc thread cache and use it for lua vm.
On certain cases, like `script flush` or `function flush` command, we
free the existing thread cache and create a new one.

Though, for `function flush`, we were not actually destroying the
existing thread cache itself. Each call creates a new thread cache on
jemalloc and we leak the previous thread cache instances. Jemalloc
allows maximum 4096 thread cache instances. If we reach this limit,
Redis prints "Failed creating the lua jemalloc tcache" log and abort.

There are other cases that can cause this memory leak, including
replication scenarios when emptyData() is called.

The implication is that it looks like redis `used_memory` is low, but
`allocator_allocated` and RSS remain high.

Co-authored-by: debing.sun <debing.sun@redis.com>
2025-04-22 10:24:30 +03:00
a96216bb9b Fix module loadex command crash due to invalid config (#13653)
Fix to https://github.com/redis/redis/issues/13650

providing an invalid config to a module with datatype crashes when redis
tries to unload the module due to the invalid config

---------

Co-authored-by: debing.sun <debing.sun@redis.com>
2025-04-22 10:16:24 +03:00
2619411a4c Avoid cluster.nodes load corruption due to shard-id generation (#13468)
PR #13428 doesn't fully resolve an issue where corruption errors can
still occur on loading of cluster.nodes file - seen on upgrade where
there were no shard_ids (from old Redis), 7.2.5 loading generated new
random ones, and persisted them to the file before gossip/handshake
could propagate the correct ones (or some other nodes unreachable).
This results in a primary/replica having differing shard_id in the
cluster.nodes and then the server cannot startup - reports corruption.

This PR builds on #13428 by simply ignoring the replica's shard_id in
cluster.nodes (if it exists), and uses the replica's primary's shard_id.
Additional handling was necessary to cover the case where the replica
appears before the primary in cluster.nodes, where it will first use a
generated shard_id for the primary, and then correct after it loads the
primary cluster.nodes entry.

---------

Co-authored-by: debing.sun <debing.sun@redis.com>
2025-04-22 10:16:14 +03:00
YaacovHazanandYaacovHazan a0a6f23d99 Redis 7.4.2 2025-01-06 16:00:10 +02:00
8b3cc8d6c6 Deprecate ubuntu lunar and macos-12 in workflows (#13669)
1. Ubuntu Lunar reached End of Life on January 25, 2024, so upgrade the
ubuntu version to plucky in action `test-ubuntu-jemalloc-fortify` to
pass the daily CI
2. The macOS-12 environment is deprecated so upgrade macos-12 to
macos-13 in daily CI

---------

Co-authored-by: debing.sun <debing.sun@redis.com>
2025-01-06 16:00:10 +02:00
YaacovHazanandYaacovHazan 781658e44e Fix LUA garbage collector (CVE-2024-46981)
Reset GC state before closing the lua VM to prevent user data
to be wrongly freed while still might be used on destructor callbacks.
2025-01-06 16:00:10 +02:00
YaacovHazanandYaacovHazan 67a442258c Fix Read/Write key pattern selector (CVE-2024-51741)
The '%' rule must contain one or both of R/W
2025-01-06 16:00:10 +02:00
guybe7andYaacovHazan f097bef7c6 Modules: defrag CB should take robj, not sds (#13627)
Added a log of the keyname in the test modules to reproduce the problem
(tests crash without the fix)
2025-01-06 16:00:10 +02:00
Moti CohenandYaacovHazan 2e473e9067 Fix memory leak on rdbload error (#13626)
On RDB load error, if an invalid `expireAt` value is read,
`dupSearchDict` is not released.
2025-01-06 16:00:10 +02:00
cae8201754 Fix get # option in sort command (#13608)
From 7.4, Redis allows `GET` options in cluster mode when the pattern maps to
the same slot as the key, but GET # pattern that represents key itself is missed.
This commit resolves it, bug report #13607.

---------

Co-authored-by: Yuan Wang <yuan.wang@redis.com>
2025-01-06 16:00:10 +02:00
Moti CohenandYaacovHazan d427a2618e Fix race in HFE tests (#13563)
Test 1 - give more time for expiration
Test 2 - Evaluate expiration time boundaries [+1,+2] before setting expiration [+1]
Test 3 - Avoid race on test HFEs propagated to replica
2025-01-06 16:00:10 +02:00
Moti CohenandYaacovHazan 232f15f6aa Correct spelling error at t_hash.c comment (#13540)
spell check error : ./src/t_hash.c:1141: RESOTRE ==> RESTORE
2025-01-06 16:00:10 +02:00
Moti CohenandYaacovHazan 0daa3dde60 HFE - Fix key ref by the hash on RENAME/MOVE/SWAPDB/RESTORE (#13539)
If the hash previously had HFEs (hash-fields with expiration) but later no longer
does, the key ref in the hash might become outdated after a MOVE, COPY,
RENAME or RESTORE operation. These commands maintain the key ref only
if HFEs are present. That is, we can only be sure that key ref is valid as long as the
hash has HFEs.
2025-01-06 16:00:10 +02:00
dd1bbaad26 Increment kvstore's non_empty_dicts only on first insert (#13528)
Found by @oranagra 

Currently, when the size of dict becomes 1, we do not check whether
`delta` is positive or negative.
As a result, `non_empty_dicts` is still incremented when the size of
dict changes from 2 to 1.
We should only increment `non_empty_dicts` when `delta` is positive, as
this indicates the first time an element is inserted into the dict.

---------

Co-authored-by: oranagra <oran@redislabs.com>
2025-01-06 16:00:10 +02:00
debing.sunandYaacovHazan 7145dc09b2 Fix a race condition issue in the cache_memory of functionsLibCtx (#13476)
This is a missing of the PR https://github.com/redis/redis/pull/13383.
We will call `functionsLibCtxClear()` in bio, so we shouldn't touch
`curr_functions_lib_ctx` in it.
2025-01-06 16:00:10 +02:00
debing.sunandYaacovHazan 9de665c613 Fix incorrect lag due to trimming stream via XTRIM command (#13473)
## Describe
When using the `XTRIM` command to trim a stream, it does not update the
maximal tombstone (`max_deleted_entry_id`). This leads to an issue where
the lag calculation incorrectly assumes that there are no tombstones
after the consumer group's last_id, resulting in an inaccurate lag.

The reason XTRIM doesn't need to update the maximal tombstone is that it
always trims from the beginning of the stream. This means that it
consistently changes the position of the first entry, leading to the
following scenarios:

1) First entry trimmed after maximal tombstone:
If the first entry is trimmed to a position after the maximal tombstone,
all tombstones will be before the first entry, so they won't affect the
consumer group's lag.

2) First entry trimmed before maximal tombstone:
If the first entry is trimmed to a position before the maximal
tombstone, the maximal tombstone will not be updated.

## Solution
Therefore, this PR optimizes the lag calculation by ensuring that when
both the consumer group's last_id and the maximal tombstone are behind
the first entry, the consumer group's lag is always equal to the number
of remaining elements in the stream.

Supplement to PR https://github.com/redis/redis/pull/13338
2025-01-06 16:00:10 +02:00
Moti CohenandYaacovHazan 344bac9b44 On HDEL last field with expiry, update global HFE DS (#13470)
Hash field expiration is optimized to avoid frequent update global HFE DS for
each field deletion. Eventually active-expiration will run and update or remove
the hash from global HFE DS gracefully. Nevertheless, statistic "subexpiry"
might reflect wrong number of hashes with HFE to the user if HDEL deletes
the last field with expiration in hash (yet there are more fields without expiration).

Following this change, if HDEL the last field with expiration in the hash then
take care to remove the hash from global HFE DS as well.
2025-01-06 16:00:10 +02:00
1bce7ef755 Pass extensions to node if extension processing is handled by it (#13465)
This PR is based on the commits from PR
https://github.com/valkey-io/valkey/pull/52.
Ref: https://github.com/redis/redis/pull/12760
Close https://github.com/redis/redis/issues/13401
This PR will replace https://github.com/redis/redis/pull/13449

Fixes compatibilty of Redis cluster (7.2 - extensions enabled by
default) with older Redis cluster (< 7.0 - extensions not handled) .

With some of the extensions enabled by default in 7.2 version, new nodes
running 7.2 and above start sending out larger clusterbus message
payload including the ping extensions. This caused an incompatibility
with node running engine versions < 7.0. Old nodes (< 7.0) would receive
the payload from new nodes (> 7.2) would observe a payload length
(totlen) > (estlen) and would perform an early exit and won't process
the message.

This fix does the following things:
1. Always set `CLUSTERMSG_FLAG0_EXT_DATA`, because during the meet
phase, we do not know whether the connected node supports ext data, we
need to make sure that it knows and send back its ext data if it has.
2. If another node does not support ext data, we will not send it ext
data to avoid the handshake failure due to the incorrect payload length.

Note: A successful `PING`/`PONG` is required as a sender for a given
node to be marked as `CLUSTERMSG_FLAG0_EXT_DATA` and then extensions
message
will be sent to it. This could cause a slight delay in receiving the
extensions message(s).

---------

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
Co-authored-by: Harkrishn Patro <harkrisp@amazon.com>

---------

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
Co-authored-by: Harkrishn Patro <harkrisp@amazon.com>
2025-01-06 16:00:10 +02:00
debing.sunandYaacovHazan b0d5f813f9 Ensure validity of myself as master or replica when loading cluster config (#13443)
First, we need to ensure that `curmaster` in
`clusterUpdateSlotsConfigWith()` is not NULL in the line
https://github.com/redis/redis/blob/82f00f5179720c8cee6cd650763d184ba943be92/src/cluster_legacy.c#L2320
otherwise, it will crash in the
https://github.com/redis/redis/blob/82f00f5179720c8cee6cd650763d184ba943be92/src/cluster_legacy.c#L2395

So when loading cluster node config, we need to ensure that the
following conditions are met:
1. A node must be at least one of the master or replica.
2. If a node is a replica, its master can't be NULL.
2025-01-06 16:00:10 +02:00
debing.sunandYaacovHazan fe7f73f1d6 Fix CLUSTER SHARDS command returns empty array (#13422)
Close https://github.com/redis/redis/issues/13414

When the cluster's master node fails and is switched to another node,
the first node in the shard node list (the old master) is no longer
valid.
Add a new method clusterGetMasterFromShard() to obtain the current
master.
2025-01-06 16:00:10 +02:00
debing.sunandYaacovHazan 9768e45413 Fix incorrect lag field in XINFO when tombstone is after the last_id of consume group (#13338)
Fix #13337

Ths PR fixes fixed two bugs that caused lag calculation errors.
1. When the latest tombstone is before the first entry, the tombstone
may stil be after the last id of consume group.
2. When a tombstone is after the last id of consume group, the group's
counter will be invalid, we should caculate the entries_read by using
estimates.
2025-01-06 16:00:10 +02:00
Oran Agra 74b289a0e1 Release Redis 7.4.1 2024-10-02 22:04:05 +03:00
Oran Agra 9a931df94f Prevent pattern matching abuse (CVE-2024-31228) 2024-10-02 22:04:05 +03:00
Oran Agra c5d0936b64 Fix ACL SETUSER Read/Write key pattern selector (CVE-2024-31227)
The '%' rule must contain one or both of R/W
2024-10-02 22:04:05 +03:00
Oran Agra b483cc5071 Fix lua bit.tohex (CVE-2024-31449)
INT_MIN value must be explicitly checked, and cannot be negated.
2024-10-02 22:04:05 +03:00
YaacovHazan c9d29f6a91 Redis 7.4.0 2024-07-29 08:31:59 +03:00
YaacovHazan 2e5b3f0829 Merge remote-tracking branch 'upstream/unstable' into HEAD 2024-07-28 20:59:59 +03:00
YaacovHazanandGitHub 0637b4ea45 Redis 7.4 RC2 (#13371)
Upgrade urgency LOW: This is the second Release Candidate for Redis 7.4.

Performance and resource utilization improvements
=================================================
* #13296 Optimize CPU cache efficiency

Changes to new 7.4 new features (compared to 7.4 RC1)
=====================================================
* #13343 Hash - expiration of individual fields: when key does not exist
- reply with an array (nonexisting code for each field)
* #13329 Hash - expiration of individual fields: new keyspace event:
`hexpired`

Modules API - Potentially breaking changes to new 7.4 features (compared
to 7.4 RC1)

====================================================================================
* #13326 Hash - expiration of individual fields: avoid lazy expire when
called from a Modules API function
2024-06-27 10:35:35 +03:00
YaacovHazan 4000bb2ee9 Redis 7.4 RC2 2024-06-27 10:02:04 +03:00
YaacovHazan 4e9acd4b82 Merge remote-tracking branch 'upstream/unstable' into HEAD 2024-06-27 09:30:10 +03:00
4606f91d5e Redis 7.4 RC1 (#13328)
Co-authored-by: YaacovHazan <yaacov.hazan@redislabs.com>
2024-06-06 14:03:20 +03:00
YaacovHazan 884c346797 Redis 7.4 RC1 2024-06-06 10:55:19 +03:00
71 changed files with 1863 additions and 311 deletions
+4 -2
View File
@@ -46,6 +46,8 @@ jobs:
- uses: actions/checkout@v4
- name: make
run: |
sed -i 's|http://deb.debian.org/debian|http://archive.debian.org/debian|g' /etc/apt/sources.list
sed -i 's|http://security.debian.org|http://archive.debian.org/debian-security|g' /etc/apt/sources.list
apt-get update && apt-get install -y build-essential
make REDIS_CFLAGS='-Werror'
@@ -91,8 +93,8 @@ jobs:
run: |
apt-get update
apt-get install -y gnupg2
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial main" >> /etc/apt/sources.list
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe" >> /etc/apt/sources.list
echo "deb http://archive.ubuntu.com/ubuntu/ xenial main" >> /etc/apt/sources.list
echo "deb http://archive.ubuntu.com/ubuntu/ xenial universe" >> /etc/apt/sources.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 40976EAF437D05B5
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3B4FE6ACC0B21F32
apt-get update
+10 -12
View File
@@ -76,7 +76,6 @@ jobs:
if: |
(github.event_name == 'workflow_dispatch' || (github.event_name != 'workflow_dispatch' && github.repository == 'redis/redis')) &&
!contains(github.event.inputs.skipjobs, 'fortify')
container: ubuntu:lunar
timeout-minutes: 14400
steps:
- name: prep
@@ -94,11 +93,10 @@ jobs:
ref: ${{ env.GITHUB_HEAD_REF }}
- name: make
run: |
apt-get update && apt-get install -y make gcc-13
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-13 100
apt-get update && apt-get install -y make gcc
make CC=gcc REDIS_CFLAGS='-Werror -DREDIS_TEST -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3'
- name: testprep
run: apt-get install -y tcl8.6 tclx procps
run: sudo apt-get install -y tcl8.6 tclx procps
- name: test
if: true && !contains(github.event.inputs.skiptests, 'redis')
run: ./runtest --accurate --verbose --dump-logs ${{github.event.inputs.test_args}}
@@ -875,7 +873,7 @@ jobs:
build-macos:
strategy:
matrix:
os: [macos-12, macos-14]
os: [macos-13, macos-15]
runs-on: ${{ matrix.os }}
if: |
(github.event_name == 'workflow_dispatch' || (github.event_name != 'workflow_dispatch' && github.repository == 'redis/redis')) &&
@@ -902,7 +900,7 @@ jobs:
run: make REDIS_CFLAGS='-Werror -DREDIS_TEST'
test-freebsd:
runs-on: macos-12
runs-on: macos-13
if: |
(github.event_name == 'workflow_dispatch' || (github.event_name != 'workflow_dispatch' && github.repository == 'redis/redis')) &&
!contains(github.event.inputs.skipjobs, 'freebsd')
@@ -1075,8 +1073,8 @@ jobs:
run: |
apt-get update
apt-get install -y gnupg2
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial main" >> /etc/apt/sources.list
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe" >> /etc/apt/sources.list
echo "deb http://archive.ubuntu.com/ubuntu/ xenial main" >> /etc/apt/sources.list
echo "deb http://archive.ubuntu.com/ubuntu/ xenial universe" >> /etc/apt/sources.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 40976EAF437D05B5
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3B4FE6ACC0B21F32
apt-get update
@@ -1123,8 +1121,8 @@ jobs:
run: |
apt-get update
apt-get install -y gnupg2
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial main" >> /etc/apt/sources.list
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe" >> /etc/apt/sources.list
echo "deb http://archive.ubuntu.com/ubuntu/ xenial main" >> /etc/apt/sources.list
echo "deb http://archive.ubuntu.com/ubuntu/ xenial universe" >> /etc/apt/sources.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 40976EAF437D05B5
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3B4FE6ACC0B21F32
apt-get update
@@ -1177,8 +1175,8 @@ jobs:
run: |
apt-get update
apt-get install -y gnupg2
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial main" >> /etc/apt/sources.list
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe" >> /etc/apt/sources.list
echo "deb http://archive.ubuntu.com/ubuntu/ xenial main" >> /etc/apt/sources.list
echo "deb http://archive.ubuntu.com/ubuntu/ xenial universe" >> /etc/apt/sources.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 40976EAF437D05B5
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3B4FE6ACC0B21F32
apt-get update
+3 -3
View File
@@ -27,7 +27,7 @@ jobs:
--tags -slow
- name: Archive redis log
if: ${{ failure() }}
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: test-external-redis-log
path: external-redis.log
@@ -55,7 +55,7 @@ jobs:
--tags -slow
- name: Archive redis log
if: ${{ failure() }}
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: test-external-cluster-log
path: external-redis-cluster.log
@@ -79,7 +79,7 @@ jobs:
--tags "-slow -needs:debug"
- name: Archive redis log
if: ${{ failure() }}
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: test-external-redis-nodebug-log
path: external-redis-nodebug.log
+268 -11
View File
@@ -1,16 +1,273 @@
Hello! This file is just a placeholder, since this is the "unstable" branch
of Redis, the place where all the development happens.
Redis Community Edition 7.4 release notes
=========================================
There is no release notes for this branch, it gets forked into another branch
every time there is a partial feature freeze in order to eventually create
a new stable release.
--------------------------------------------------------------------------------
Upgrade urgency levels:
Usually "unstable" is stable enough for you to use it in development environments
however you should never use it in production environments. It is possible
to download the latest stable release here:
LOW: No need to upgrade unless there are new features you want to use.
MODERATE: Program an upgrade of the server, but it's not urgent.
HIGH: There is a critical bug that may affect a subset of users. Upgrade!
CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
SECURITY: There are security fixes in the release.
--------------------------------------------------------------------------------
https://download.redis.io/redis-stable.tar.gz
More information is available at https://redis.io
================================================================================
Redis 7.4.6 Released Fri 3 Oct 2025 10:00:00 IST
================================================================================
Update urgency: `SECURITY`: There are security fixes in the release.
### Security fixes
- (CVE-2025-49844) A Lua script may lead to remote code execution
- (CVE-2025-46817) A Lua script may lead to integer overflow and potential RCE
- (CVE-2025-46818) A Lua script can be executed in the context of another user
- (CVE-2025-46819) LUA out-of-bound read
### Bug fixes
- #14330 Potential use-after-free after pubsub and Lua defrag
- #14319 Potential crash on Lua script defrag
- #14164 Prevent `CLIENT UNBLOCK` from unblocking `CLIENT PAUSE`
- #14165 Endless client blocking for blocking commands
- #14163 `EVAL` crash when error table is empty
- #14227 `HINCRBYFLOAT` removes field expiration on replica
================================================================================
Redis 7.4.5 Released Sun 6 Jul 2025 12:00:00 IST
================================================================================
Update urgency: `SECURITY`: There are security fixes in the release.
### Security fixes
- (CVE-2025-32023) Fix out-of-bounds write in `HyperLogLog` commands
- (CVE-2025-48367) Retry accepting other connections even if the accepted connection reports an error
================================================================================
Redis 7.4.4 Released Tue 27 May 2025 12:00:00 IST
================================================================================
Update urgency: `SECURITY`: There are security fixes in the release.
### Security fixes
- (CVE-2025-27151) redis-check-aof may lead to stack overflow and potential RCE
### Bug fixes
- #13966, #13932 `CLUSTER SLOTS` - TLS port update not reflected in CLUSTER SLOTS
- #13958 `XTRIM`, `XADD` - incorrect lag due to trimming stream
================================================================================
Redis 7.4.3 Released Wed 23 Apr 2025 12:00:00 IST
================================================================================
Update urgency: `SECURITY`: There are security fixes in the release.
### Security fixes
- (CVE-2025-21605) An unauthenticated client can cause an unlimited growth of output buffers
### Bug fixes
- #13661 `FUNCTION FLUSH` - memory leak when using jemalloc
- #13793 `WAITAOF` returns prematurely
- #13853 `SLAVEOF` - crash when clients are blocked on lazy free
- #13863 `RANDOMKEY` - infinite loop during client pause
- #13877 ShardID inconsistency when both primary and replica support it
================================================================================
Redis Community Edition 7.4.2 Released Mon 6 Jan 2025 12:30:00 IDT
================================================================================
Upgrade urgency SECURITY: See security fixes below.
### Security fixes
- (CVE-2024-46981) Lua script commands may lead to remote code execution
- (CVE-2024-51741) Denial-of-service due to malformed ACL selectors
### Bug fixes
- #13627 Crash on module memory defragmentation
- #13338 Streams: `XINFO` lag field is wrong when tombstone is after the `last_id` of the consume group
- #13473 Streams: `XTRIM` does not update the maximal tombstone, leading to an incorrect lag
- #13470 `INFO` after `HDEL` show wrong number of hash keys with expiration
- #13476 Fix a race condition in the `cache_memory` of `functionsLibCtx`
- #13626 Memory leak on failed RDB loading
- #13539 Hash: fix key ref for a hash that no longer has fields with expiration on `RENAME`/`MOVE`/`SWAPDB`/`RESTORE`
- #13443 Cluster: crash when loading cluster config
- #13422 Cluster: `CLUSTER SHARDS` returns empty array
- #13465 Cluster: incompatibility with older node versions
- #13608 Cluster: `SORT ... GET #`: incorrect error message
================================================================================
Redis Community Edition 7.4.1 Released Wed 02 Oct 2024 20:17:04 IDT
================================================================================
Upgrade urgency SECURITY: See security fixes below.
Security fixes
==============
* (CVE-2024-31449) Lua library commands may lead to stack overflow and potential RCE.
* (CVE-2024-31227) Potential Denial-of-service due to malformed ACL selectors.
* (CVE-2024-31228) Potential Denial-of-service due to unbounded pattern matching.
================================================================================
Redis Community Edition 7.4.0 GA Released Mon Jul 29 2024 9:00:00 IDT
================================================================================
This is the General Availability release of Redis Community Edition 7.4.
Changes to new 7.4 features (compared to 7.4 RC2)
=================================================
* #13391,#13438 Hash - expiration of individual fields: RDB file format changes
* #13372 Hash - expiration of individual fields: rename and fix counting of `expired_subkeys` metric
* #13372 Hash - expiration of individual fields: rename `INFO` keyspace field to `subexpiry`
Configuration parameters
========================
* #13400 Add hide-user-data-from-log - allows hiding user data from the log file
Bug fixes
=========
* #13407 trigger Lua GC after `SCRIPT LOAD`
* #13380 Fix possible crash due to OOM panic on invalid command
* #13383 `FUNCTION FLUSH` - improve Lua GC behavior and fix thread race in ASYNC mode
* #13408 `HEXPIRE`-like commands should emit HDEL keyspace notification if expire time is in the past
================================================================================
Redis Community Edition 7.4 RC2 Released Thu 27 Jun 2024 10:00:00 IST
================================================================================
Upgrade urgency LOW: This is the second Release Candidate for Redis Community Edition 7.4.
Performance and resource utilization improvements
=================================================
* #13296 Optimize CPU cache efficiency
Changes to new 7.4 new features (compared to 7.4 RC1)
=====================================================
* #13343 Hash - expiration of individual fields: when key does not exist - reply with an array (nonexisting code for each field)
* #13329 Hash - expiration of individual fields: new keyspace event: `hexpired`
Modules API - Potentially breaking changes to new 7.4 features (compared to 7.4 RC1)
====================================================================================
* #13326 Hash - expiration of individual fields: avoid lazy expire when called from a Modules API function
================================================================================
Redis Community Edition 7.4 RC1 Released Thu 6 Jun 2024 10:00:00 IST
================================================================================
Note: License changed - see LICENSE.txt
Upgrade urgency LOW: This is the first Release Candidate for Redis Community Edition 7.4.
Here is a comprehensive list of changes in this release compared to 7.2.5.
New Features
============
* #13303 Hash - expiration of individual fields. 9 commands were introduced:
- `HEXPIRE` and `HPEXPIRE` set the remaining time to live for specific fields
- `HEXPIREAT` and `HPEXPIREAT` set the expiration time to a UNIX timestamp for specific fields
- `HPERSIST` removes the expiration for specific fields
- `HEXPIRETIME` and `HPEXPIRETIME` get the expiration time for specific fields
- `HTTL` and `HPTTL` get the remaining time to live for specific fields
* #13117 `XREAD`: new id value `+` to start reading from the last message
* #12765 `HSCAN`: new [NOVALUES] flag to report only field names
* #12728 `SORT`, `SORT_RO`: allow `BY` and `GET` options in cluster mode when the pattern maps to the same slot as the key
* #12299 `CLIENT KILL`: new optional filter: `MAXAGE maxage` - retain connections older than `maxage` seconds
* #12971 Lua: expose `os.clock()` API for getting the elapsed time of Lua code execution
* #13276 Allow `SPUBLISH` command within `MULTI ... EXEC` transactions on replica
Bug fixes
=========
* #12898 `XREADGROUP`: fix entries-read inconsistency between master and replicas
* #13042 `SORT ... STORE`: fix created lists to respect list compression and packing configs
* #12817, #12905 Fix race condition issues between the main thread and module threads
* #12577 Unsubscribe all clients from replica for shard channel if the master ownership changes
* #12622 `WAITAOF` could timeout or hang if used after a module command that propagated effects only to replicas and not to AOF
* #11734 `BITCOUNT` and `BITPOS` with nonexistent key and illegal arguments return an error, not 0
* #12394 `BITCOUNT`: check for wrong argument before checking if key exists
* #12961 Allow execution of read-only transactions when out of memory
* #13274 Fix crash when a client performs ACL change that disconnects itself
* #13311 Cluster: Fix crash due to unblocking client during slot migration
Security improvements
=====================
* #13108 Lua: LRU eviction for scripts generated with `EVAL` *** BEHAVIOR CHANGE ***
* #12961 Restrict the total request size of `MULTI ... EXEC` transactions
* #12860 Redact ACL username information and mark '*-key-file-pass configs' as sensitive
Performance and resource utilization improvements
=================================================
* #12838 Improve performance when many clients call `PUNSUBSCRIBE` / `SUNSUBSCRIBE` simultaneously
* #12627 Reduce lag when waking `WAITAOF` clients and there is not much traffic
* #12754 Optimize `KEYS` when pattern includes hashtag and implies a single slot
* #11695 Reduce memory and improve performance by replacing cluster metadata with slot specific dictionaries
* #13087 `SCRIPT FLUSH ASYNC` now does not block the main thread
* #12996 Active memory defragmentation efficiency improvements
* #12899 Improve performance of read/update operation during rehashing
* #12536 `SCAN ... MATCH`: Improve performance when the pattern implies cluster slot
* #12450 `ZRANGE ... LIMIT`: improved performance
Other general improvements
==========================
* #13133 Lua: allocate VM code with jemalloc instead of libc and count it as used memory *** BEHAVIOR CHANGE ***
* #12171 `ACL LOAD`: do not disconnect all clients *** BEHAVIOR CHANGE ***
* #13020 Allow adjusting defrag configurations while active defragmentation is running
* #12949 Increase the accuracy of avg_ttl (the average keyspace keys TTL)
* #12977 Allow running `WAITAOF` in scripts
* #12782 Implement TCP keep-alive across most Unix-like systems
* #12707 Improved error codes when rejecting scripts in cluster mode
* #12596 Support `XREAD ... BLOCK` in scripts; rejected only if it ends up blocking
New metrics
===========
* #12849 `INFO`: `pubsub_clients` - number of clients in Pub/Sub mode
* #12966 `INFO`: `watching_clients` - number of clients that are watching keys
* #12966 `INFO`: `total_watched_keys` - number of watched keys
* #12476 `INFO`: `client_query_buffer_limit_disconnections` - count client input buffer OOM events
* #12476 `INFO`: `client_output_buffer_limit_disconnections` - count client output buffer OOM events
* #12996 `INFO`: `allocator_muzzy` - memory returned to the OS but still shows as RSS until the OS reclaims it
* #13108 `INFO`: `evicted_scripts` - number of evicted eval scripts. Users can check it to see if they are abusing EVAL
* #12996 `MEMORY STATS`: `allocator.muzzy` - memory returned to the OS but still shows as RSS until the OS reclaims it
* #12913 `INFO MEMORY` `mem_overhead_db_hashtable_rehashing` - memory resharding overhead (only the memory that will be released soon)
* #12913 `MEMORY STATS': `overhead.db.hashtable.lut` - total overhead of dictionary buckets in databases
* #12913 `MEMORY STATS': `overhead.db.hashtable.rehashing` - temporary memory overhead of database dictionaries currently being rehashed
* #12913 `MEMORY STATS': `db.dict.rehashing.count` - number of top level dictionaries currently being rehashed
* #12966 `CLIENT LIST`: `watch` - number of keys each client is currently watching
Modules API
===========
* #12985 New API calls: `RM_TryCalloc` and `RM_TryRealloc` - allow modules to handle memory allocation failures gracefully
* #13069 New API call: `RM_ClusterKeySlot` - which slot a key will hash to
* #13069 New API call: `RM_ClusterCanonicalKeyNameInSlot` - get a consistent key that will map to a slot
* #12486 New API call: `RM_AddACLCategory` - allow modules to declare new ACL categories
Configuration parameters
========================
* #12178 New configuration parameters: `max-new-connections-per-cycle` and `max-new-tls-connections-per-cycle` to limit the number of new client connections per event-loop cycle
* #7351 Rename some CPU configuration parameters for style alignment. Added alias to the old names to avoid breaking change
CLI tools
=========
* #10609 redis-cli: new `-t <timeout>` argument: specify server connection timeout in seconds
* #11315 redis-cli: new `-4` and `-6` flags to prefer IPV4 or IPV6 on DNS lookup
* #12862 redis-cli: allows pressing up arrow to return any command (including sensitive commands which are still not persisted)
* #12543 redis-cli: add reverse history search (like Ctrl+R in terminals)
* #12826 redis-cli: add `--keystats` and `--keystats-samples` to combines `--memkeys` and `--bigkeys` with additional distribution data
* #12735 redis-cli: fix: `--bigkeys` and `--memkeys` now work on cluster replicas
* #9411 redis-benchmark: add support for binary strings
* #12986 redis-benchmark: fix: pick random slot for a node to distribute operation across slots
Happy hacking!
+4 -3
View File
@@ -340,13 +340,14 @@ static int luaB_assert (lua_State *L) {
static int luaB_unpack (lua_State *L) {
int i, e, n;
int i, e;
unsigned int n;
luaL_checktype(L, 1, LUA_TTABLE);
i = luaL_optint(L, 2, 1);
e = luaL_opt(L, luaL_checkint, 3, luaL_getn(L, 1));
if (i > e) return 0; /* empty range */
n = e - i + 1; /* number of elements */
if (n <= 0 || !lua_checkstack(L, n)) /* n <= 0 means arith. overflow */
n = (unsigned int)e - (unsigned int)i; /* number of elements minus 1 */
if (n >= INT_MAX || !lua_checkstack(L, ++n))
return luaL_error(L, "too many results to unpack");
lua_rawgeti(L, 1, i); /* push arg[i] (avoiding overflow problems) */
while (i++ < e) /* push arg[i + 1...e] */
+21 -13
View File
@@ -138,6 +138,7 @@ static void inclinenumber (LexState *ls) {
void luaX_setinput (lua_State *L, LexState *ls, ZIO *z, TString *source) {
ls->t.token = 0;
ls->decpoint = '.';
ls->L = L;
ls->lookahead.token = TK_EOS; /* no look-ahead token */
@@ -206,9 +207,13 @@ static void read_numeral (LexState *ls, SemInfo *seminfo) {
trydecpoint(ls, seminfo); /* try to update decimal point separator */
}
static int skip_sep (LexState *ls) {
int count = 0;
/*
** reads a sequence '[=*[' or ']=*]', leaving the last bracket.
** If a sequence is well-formed, return its number of '='s + 2; otherwise,
** return 1 if there is no '='s or 0 otherwise (an unfinished '[==...').
*/
static size_t skip_sep (LexState *ls) {
size_t count = 0;
int s = ls->current;
lua_assert(s == '[' || s == ']');
save_and_next(ls);
@@ -216,11 +221,13 @@ static int skip_sep (LexState *ls) {
save_and_next(ls);
count++;
}
return (ls->current == s) ? count : (-count) - 1;
return (ls->current == s) ? count + 2
: (count == 0) ? 1
: 0;
}
static void read_long_string (LexState *ls, SemInfo *seminfo, int sep) {
static void read_long_string (LexState *ls, SemInfo *seminfo, size_t sep) {
int cont = 0;
(void)(cont); /* avoid warnings when `cont' is not used */
save_and_next(ls); /* skip 2nd `[' */
@@ -270,8 +277,8 @@ static void read_long_string (LexState *ls, SemInfo *seminfo, int sep) {
}
} endloop:
if (seminfo)
seminfo->ts = luaX_newstring(ls, luaZ_buffer(ls->buff) + (2 + sep),
luaZ_bufflen(ls->buff) - 2*(2 + sep));
seminfo->ts = luaX_newstring(ls, luaZ_buffer(ls->buff) + sep,
luaZ_bufflen(ls->buff) - 2 * sep);
}
@@ -346,9 +353,9 @@ static int llex (LexState *ls, SemInfo *seminfo) {
/* else is a comment */
next(ls);
if (ls->current == '[') {
int sep = skip_sep(ls);
size_t sep = skip_sep(ls);
luaZ_resetbuffer(ls->buff); /* `skip_sep' may dirty the buffer */
if (sep >= 0) {
if (sep >= 2) {
read_long_string(ls, NULL, sep); /* long comment */
luaZ_resetbuffer(ls->buff);
continue;
@@ -360,13 +367,14 @@ static int llex (LexState *ls, SemInfo *seminfo) {
continue;
}
case '[': {
int sep = skip_sep(ls);
if (sep >= 0) {
size_t sep = skip_sep(ls);
if (sep >= 2) {
read_long_string(ls, seminfo, sep);
return TK_STRING;
}
else if (sep == -1) return '[';
else luaX_lexerror(ls, "invalid long string delimiter", TK_STRING);
else if (sep == 0) /* '[=...' missing second bracket */
luaX_lexerror(ls, "invalid long string delimiter", TK_STRING);
return '[';
}
case '=': {
next(ls);
+5 -1
View File
@@ -384,13 +384,17 @@ Proto *luaY_parser (lua_State *L, ZIO *z, Mbuffer *buff, const char *name) {
struct LexState lexstate;
struct FuncState funcstate;
lexstate.buff = buff;
luaX_setinput(L, &lexstate, z, luaS_new(L, name));
TString *tname = luaS_new(L, name);
setsvalue2s(L, L->top, tname);
incr_top(L);
luaX_setinput(L, &lexstate, z, tname);
open_func(&lexstate, &funcstate);
funcstate.f->is_vararg = VARARG_ISVARARG; /* main func. is always vararg */
luaX_next(&lexstate); /* read first token */
chunk(&lexstate);
check(&lexstate, TK_EOS);
close_func(&lexstate);
--L->top;
lua_assert(funcstate.prev == NULL);
lua_assert(funcstate.f->nups == 0);
lua_assert(lexstate.fs == NULL);
+1 -2
View File
@@ -434,8 +434,7 @@ static TValue *newkey (lua_State *L, Table *t, const TValue *key) {
** search function for integers
*/
const TValue *luaH_getnum (Table *t, int key) {
/* (1 <= key && key <= t->sizearray) */
if (cast(unsigned int, key-1) < cast(unsigned int, t->sizearray))
if (1 <= key && key <= t->sizearray)
return &t->array[key-1];
else {
lua_Number nk = cast_num(key);
+1
View File
@@ -132,6 +132,7 @@ static int bit_tohex(lua_State *L)
const char *hexdigits = "0123456789abcdef";
char buf[8];
int i;
if (n == INT32_MIN) n = INT32_MIN+1;
if (n < 0) { n = -n; hexdigits = "0123456789ABCDEF"; }
if (n > 8) n = 8;
for (i = (int)n; --i >= 0; ) { buf[i] = hexdigits[b & 15]; b >>= 4; }
+3
View File
@@ -110,6 +110,9 @@ else
ifeq ($(SANITIZER),undefined)
MALLOC=libc
CFLAGS+=-fsanitize=undefined -fno-sanitize-recover=all -fno-omit-frame-pointer
ifeq (clang,$(CLANG))
CFLAGS+=-fno-sanitize=function
endif
LDFLAGS+=-fsanitize=undefined
else
ifeq ($(SANITIZER),thread)
+7 -2
View File
@@ -1061,6 +1061,7 @@ int ACLSetSelector(aclSelector *selector, const char* op, size_t oplen) {
int flags = 0;
size_t offset = 1;
if (op[0] == '%') {
int perm_ok = 1;
for (; offset < oplen; offset++) {
if (toupper(op[offset]) == 'R' && !(flags & ACL_READ_PERMISSION)) {
flags |= ACL_READ_PERMISSION;
@@ -1070,10 +1071,14 @@ int ACLSetSelector(aclSelector *selector, const char* op, size_t oplen) {
offset++;
break;
} else {
errno = EINVAL;
return C_ERR;
perm_ok = 0;
break;
}
}
if (!flags || !perm_ok) {
errno = EINVAL;
return C_ERR;
}
} else {
flags = ACL_ALL_PERMISSION;
}
+24
View File
@@ -786,3 +786,27 @@ int anetIsFifo(char *filepath) {
if (stat(filepath, &sb) == -1) return 0;
return S_ISFIFO(sb.st_mode);
}
/* This function must be called after accept4() fails. It returns 1 if 'err'
* indicates accepted connection faced an error, and it's okay to continue
* accepting next connection by calling accept4() again. Other errors either
* indicate programming errors, e.g. calling accept() on a closed fd or indicate
* a resource limit has been reached, e.g. -EMFILE, open fd limit has been
* reached. In the latter case, caller might wait until resources are available.
* See accept4() documentation for details. */
int anetAcceptFailureNeedsRetry(int err) {
if (err == ECONNABORTED)
return 1;
#if defined(__linux__)
/* For details, see 'Error Handling' section on
* https://man7.org/linux/man-pages/man2/accept.2.html */
if (err == ENETDOWN || err == EPROTO || err == ENOPROTOOPT ||
err == EHOSTDOWN || err == ENONET || err == EHOSTUNREACH ||
err == EOPNOTSUPP || err == ENETUNREACH)
{
return 1;
}
#endif
return 0;
}
+1
View File
@@ -52,5 +52,6 @@ int anetPipe(int fds[2], int read_flags, int write_flags);
int anetSetSockMarkId(char *err, int fd, uint32_t id);
int anetGetError(int fd);
int anetIsFifo(char *filepath);
int anetAcceptFailureNeedsRetry(int err);
#endif
+25 -26
View File
@@ -1048,35 +1048,34 @@ void flushAppendOnlyFile(int force) {
mstime_t latency;
if (sdslen(server.aof_buf) == 0) {
/* Check if we need to do fsync even the aof buffer is empty,
* because previously in AOF_FSYNC_EVERYSEC mode, fsync is
* called only when aof buffer is not empty, so if users
* stop write commands before fsync called in one second,
* the data in page cache cannot be flushed in time. */
if (server.aof_fsync == AOF_FSYNC_EVERYSEC &&
server.aof_last_incr_fsync_offset != server.aof_last_incr_size &&
server.mstime - server.aof_last_fsync >= 1000 &&
!(sync_in_progress = aofFsyncInProgress())) {
goto try_fsync;
/* Check if we need to do fsync even the aof buffer is empty,
* the reason is described in the previous AOF_FSYNC_EVERYSEC block,
* and AOF_FSYNC_ALWAYS is also checked here to handle a case where
* aof_fsync is changed from everysec to always. */
} else if (server.aof_fsync == AOF_FSYNC_ALWAYS &&
server.aof_last_incr_fsync_offset != server.aof_last_incr_size)
{
goto try_fsync;
} else {
if (server.aof_last_incr_fsync_offset == server.aof_last_incr_size) {
/* All data is fsync'd already: Update fsynced_reploff_pending just in case.
* This is needed to avoid a WAITAOF hang in case a module used RM_Call with the NO_AOF flag,
* in which case master_repl_offset will increase but fsynced_reploff_pending won't be updated
* (because there's no reason, from the AOF POV, to call fsync) and then WAITAOF may wait on
* the higher offset (which contains data that was only propagated to replicas, and not to AOF) */
if (!sync_in_progress && server.aof_fsync != AOF_FSYNC_NO)
* This is needed to avoid a WAITAOF hang in case a module used RM_Call
* with the NO_AOF flag, in which case master_repl_offset will increase but
* fsynced_reploff_pending won't be updated (because there's no reason, from
* the AOF POV, to call fsync) and then WAITAOF may wait on the higher offset
* (which contains data that was only propagated to replicas, and not to AOF) */
if (!aofFsyncInProgress())
atomicSet(server.fsynced_reploff_pending, server.master_repl_offset);
return;
} else {
/* Check if we need to do fsync even the aof buffer is empty,
* because previously in AOF_FSYNC_EVERYSEC mode, fsync is
* called only when aof buffer is not empty, so if users
* stop write commands before fsync called in one second,
* the data in page cache cannot be flushed in time. */
if (server.aof_fsync == AOF_FSYNC_EVERYSEC &&
server.mstime - server.aof_last_fsync >= 1000 &&
!(sync_in_progress = aofFsyncInProgress()))
goto try_fsync;
/* Check if we need to do fsync even the aof buffer is empty,
* the reason is described in the previous AOF_FSYNC_EVERYSEC block,
* and AOF_FSYNC_ALWAYS is also checked here to handle a case where
* aof_fsync is changed from everysec to always. */
if (server.aof_fsync == AOF_FSYNC_ALWAYS)
goto try_fsync;
}
return;
}
if (server.aof_fsync == AOF_FSYNC_EVERYSEC)
+23 -1
View File
@@ -205,6 +205,24 @@ void unblockClient(client *c, int queue_for_reprocessing) {
if (queue_for_reprocessing) queueClientForReprocessing(c);
}
/* Check if the specified client can be safely timed out using
* unblockClientOnTimeout(). */
int blockedClientMayTimeout(client *c) {
if (c->bstate.btype == BLOCKED_MODULE) {
return moduleBlockedClientMayTimeout(c);
}
if (c->bstate.btype == BLOCKED_LIST ||
c->bstate.btype == BLOCKED_ZSET ||
c->bstate.btype == BLOCKED_STREAM ||
c->bstate.btype == BLOCKED_WAIT ||
c->bstate.btype == BLOCKED_WAITAOF)
{
return 1;
}
return 0;
}
/* This function gets called when a blocked client timed out in order to
* send it a reply of some kind. After this function is called,
* unblockClient() will be called with the same client as argument. */
@@ -270,6 +288,7 @@ void disconnectAllBlockedClients(void) {
if (c->bstate.btype == BLOCKED_LAZYFREE) {
addReply(c, shared.ok); /* No reason lazy-free to fail */
updateStatsOnUnblock(c, 0, 0, 0);
c->flags &= ~CLIENT_PENDING_COMMAND;
unblockClient(c, 1);
} else {
@@ -361,7 +380,7 @@ void blockForKeys(client *c, int btype, robj **keys, int numkeys, mstime_t timeo
list *l;
int j;
if (!(c->flags & CLIENT_REPROCESSING_COMMAND)) {
if (!(c->flags & CLIENT_REEXECUTING_COMMAND)) {
/* If the client is re-processing the command, we do not set the timeout
* because we need to retain the client's original timeout. */
c->bstate.timeout = timeout;
@@ -647,6 +666,7 @@ static void unblockClientOnKey(client *c, robj *key) {
* we need to re process the command again */
if (c->flags & CLIENT_PENDING_COMMAND) {
c->flags &= ~CLIENT_PENDING_COMMAND;
c->flags |= CLIENT_REEXECUTING_COMMAND;
/* We want the command processing and the unblock handler (see RM_Call 'K' option)
* to run atomically, this is why we must enter the execution unit here before
* running the command, and exit the execution unit after calling the unblock handler (if exists).
@@ -665,6 +685,8 @@ static void unblockClientOnKey(client *c, robj *key) {
}
exitExecutionUnit();
afterCommand(c);
/* Clear the CLIENT_REEXECUTING_COMMAND flag after the proc is executed. */
c->flags &= ~CLIENT_REEXECUTING_COMMAND;
server.current_client = old_client;
}
}
+74 -39
View File
@@ -2,8 +2,13 @@
* Copyright (c) 2009-Present, Redis Ltd.
* All rights reserved.
*
* Copyright (c) 2024-present, Valkey contributors.
* All rights reserved.
*
* Licensed under your choice of the Redis Source Available License 2.0
* (RSALv2) or the Server Side Public License v1 (SSPLv1).
*
* Portions of this file are available under BSD3 terms; see REDISCONTRIBUTIONS for more information.
*/
/*
@@ -88,6 +93,7 @@ int auxTlsPortPresent(clusterNode *n);
static void clusterBuildMessageHdr(clusterMsg *hdr, int type, size_t msglen);
void freeClusterLink(clusterLink *link);
int verifyClusterNodeId(const char *name, int length);
static void updateShardId(clusterNode *node, const char *shard_id);
int getNodeDefaultClientPort(clusterNode *n) {
return server.tls_cluster ? n->tls_port : n->tcp_port;
@@ -198,12 +204,11 @@ int auxShardIdSetter(clusterNode *n, void *value, int length) {
return C_ERR;
}
memcpy(n->shard_id, value, CLUSTER_NAMELEN);
/* if n already has replicas, make sure they all agree
* on the shard id */
/* if n already has replicas, make sure they all use
* the primary shard id */
for (int i = 0; i < n->numslaves; i++) {
if (memcmp(n->slaves[i]->shard_id, n->shard_id, CLUSTER_NAMELEN) != 0) {
return C_ERR;
}
if (memcmp(n->slaves[i]->shard_id, n->shard_id, CLUSTER_NAMELEN) != 0)
updateShardId(n->slaves[i], n->shard_id);
}
clusterAddNodeToShard(value, n);
return C_OK;
@@ -545,18 +550,12 @@ int clusterLoadConfig(char *filename) {
clusterAddNode(master);
}
/* shard_id can be absent if we are loading a nodes.conf generated
* by an older version of Redis; we should follow the primary's
* shard_id in this case */
if (auxFieldHandlers[af_shard_id].isPresent(n) == 0) {
memcpy(n->shard_id, master->shard_id, CLUSTER_NAMELEN);
clusterAddNodeToShard(master->shard_id, n);
} else if (clusterGetNodesInMyShard(master) != NULL &&
memcmp(master->shard_id, n->shard_id, CLUSTER_NAMELEN) != 0)
{
/* If the primary has been added to a shard, make sure this
* node has the same persisted shard id as the primary. */
goto fmterr;
}
* by an older version of Redis;
* ignore replica's shard_id in the file, only use the primary's.
* If replica precedes primary in file, it will be corrected
* later by the auxShardIdSetter */
memcpy(n->shard_id, master->shard_id, CLUSTER_NAMELEN);
clusterAddNodeToShard(master->shard_id, n);
n->slaveof = master;
clusterNodeAddSlave(master,n);
} else if (auxFieldHandlers[af_shard_id].isPresent(n) == 0) {
@@ -634,6 +633,8 @@ int clusterLoadConfig(char *filename) {
}
/* Config sanity check */
if (server.cluster->myself == NULL) goto fmterr;
if (!(myself->flags & (CLUSTER_NODE_MASTER | CLUSTER_NODE_SLAVE))) goto fmterr;
if (nodeIsSlave(myself) && myself->slaveof == NULL) goto fmterr;
zfree(line);
fclose(fp);
@@ -901,22 +902,33 @@ static void updateAnnouncedHumanNodename(clusterNode *node, char *new) {
clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG);
}
static void assignShardIdToNode(clusterNode *node, const char *shard_id, int flag) {
clusterRemoveNodeFromShard(node);
memcpy(node->shard_id, shard_id, CLUSTER_NAMELEN);
clusterAddNodeToShard(shard_id, node);
clusterDoBeforeSleep(flag);
}
static void updateShardId(clusterNode *node, const char *shard_id) {
if (shard_id && memcmp(node->shard_id, shard_id, CLUSTER_NAMELEN) != 0) {
clusterRemoveNodeFromShard(node);
memcpy(node->shard_id, shard_id, CLUSTER_NAMELEN);
clusterAddNodeToShard(shard_id, node);
clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG);
}
if (shard_id && myself != node && myself->slaveof == node) {
if (memcmp(myself->shard_id, shard_id, CLUSTER_NAMELEN) != 0) {
/* shard-id can diverge right after a rolling upgrade
* from pre-7.2 releases */
clusterRemoveNodeFromShard(myself);
memcpy(myself->shard_id, shard_id, CLUSTER_NAMELEN);
clusterAddNodeToShard(shard_id, myself);
clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG|CLUSTER_TODO_FSYNC_CONFIG);
/* We always make our best effort to keep the shard-id consistent
* between the master and its replicas:
*
* 1. When updating the master's shard-id, we simultaneously update the
* shard-id of all its replicas to ensure consistency.
* 2. When updating replica's shard-id, if it differs from its master's shard-id,
* we discard this replica's shard-id and continue using master's shard-id.
* This applies even if the master does not support shard-id, in which
* case we rely on the master's randomly generated shard-id. */
if (node->slaveof == NULL) {
assignShardIdToNode(node, shard_id, CLUSTER_TODO_SAVE_CONFIG);
for (int i = 0; i < clusterNodeNumSlaves(node); i++) {
clusterNode *slavenode = clusterNodeGetSlave(node, i);
if (memcmp(slavenode->shard_id, shard_id, CLUSTER_NAMELEN) != 0)
assignShardIdToNode(slavenode, shard_id, CLUSTER_TODO_SAVE_CONFIG|CLUSTER_TODO_FSYNC_CONFIG);
}
} else if (memcmp(node->slaveof->shard_id, shard_id, CLUSTER_NAMELEN) == 0) {
assignShardIdToNode(node, shard_id, CLUSTER_TODO_SAVE_CONFIG);
}
}
}
@@ -1238,6 +1250,8 @@ void clusterAcceptHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
while(max--) {
cfd = anetTcpAccept(server.neterr, fd, cip, sizeof(cip), &cport);
if (cfd == ANET_ERR) {
if (anetAcceptFailureNeedsRetry(errno))
continue;
if (errno != EWOULDBLOCK)
serverLog(LL_VERBOSE,
"Error accepting cluster node: %s", server.neterr);
@@ -1624,6 +1638,22 @@ void clusterRemoveNodeFromShard(clusterNode *node) {
sdsfree(s);
}
static clusterNode *clusterGetMasterFromShard(list *nodes) {
clusterNode *n = NULL;
listIter li;
listNode *ln;
listRewind(nodes,&li);
while ((ln = listNext(&li)) != NULL) {
clusterNode *node = listNodeValue(ln);
if (!nodeFailed(node)) {
n = node;
break;
}
}
if (!n) return NULL;
return clusterNodeGetMaster(n);
}
/* -----------------------------------------------------------------------------
* CLUSTER config epoch handling
* -------------------------------------------------------------------------- */
@@ -2577,9 +2607,6 @@ uint32_t writePingExt(clusterMsg *hdr, int gossipcount) {
extensions++;
if (hdr != NULL) {
if (extensions != 0) {
hdr->mflags[0] |= CLUSTERMSG_FLAG0_EXT_DATA;
}
hdr->extensions = htons(extensions);
}
@@ -2769,6 +2796,9 @@ int clusterProcessPacket(clusterLink *link) {
}
sender = getNodeFromLinkAndMsg(link, hdr);
if (sender && (hdr->mflags[0] & CLUSTERMSG_FLAG0_EXT_DATA)) {
sender->flags |= CLUSTER_NODE_EXTENSIONS_SUPPORTED;
}
/* Update the last time we saw any data from this node. We
* use this in order to avoid detecting a timeout from a node that
@@ -3534,6 +3564,8 @@ static void clusterBuildMessageHdr(clusterMsg *hdr, int type, size_t msglen) {
/* Set the message flags. */
if (clusterNodeIsMaster(myself) && server.cluster->mf_end)
hdr->mflags[0] |= CLUSTERMSG_FLAG0_PAUSED;
hdr->mflags[0] |= CLUSTERMSG_FLAG0_EXT_DATA; /* Always make other nodes know that
* this node supports extension data. */
hdr->totlen = htonl(msglen);
}
@@ -3612,7 +3644,9 @@ void clusterSendPing(clusterLink *link, int type) {
* to put inside the packet. */
estlen = sizeof(clusterMsg) - sizeof(union clusterMsgData);
estlen += (sizeof(clusterMsgDataGossip)*(wanted + pfail_wanted));
estlen += writePingExt(NULL, 0);
if (link->node && nodeSupportsExtensions(link->node)) {
estlen += writePingExt(NULL, 0);
}
/* Note: clusterBuildMessageHdr() expects the buffer to be always at least
* sizeof(clusterMsg) or more. */
if (estlen < (int)sizeof(clusterMsg)) estlen = sizeof(clusterMsg);
@@ -3682,7 +3716,9 @@ void clusterSendPing(clusterLink *link, int type) {
/* Compute the actual total length and send! */
uint32_t totlen = 0;
totlen += writePingExt(hdr, gossipcount);
if (link->node && nodeSupportsExtensions(link->node)) {
totlen += writePingExt(hdr, gossipcount);
}
totlen += sizeof(clusterMsg)-sizeof(union clusterMsgData);
totlen += (sizeof(clusterMsgDataGossip)*gossipcount);
serverAssert(gossipcount < USHRT_MAX);
@@ -5649,14 +5685,13 @@ void addNodeDetailsToShardReply(client *c, clusterNode *node) {
/* Add the shard reply of a single shard based off the given primary node. */
void addShardReplyForClusterShards(client *c, list *nodes) {
serverAssert(listLength(nodes) > 0);
clusterNode *n = listNodeValue(listFirst(nodes));
addReplyMapLen(c, 2);
addReplyBulkCString(c, "slots");
/* Use slot_info_pairs from the primary only */
n = clusterNodeGetMaster(n);
if (n->slot_info_pairs != NULL) {
clusterNode *n = clusterGetMasterFromShard(nodes);
if (n && n->slot_info_pairs != NULL) {
serverAssert((n->slot_info_pairs_count % 2) == 0);
addReplyArrayLen(c, n->slot_info_pairs_count);
for (int i = 0; i < n->slot_info_pairs_count; i++)
+15
View File
@@ -1,3 +1,16 @@
/*
* Copyright (c) 2009-Present, Redis Ltd.
* All rights reserved.
*
* Copyright (c) 2024-present, Valkey contributors.
* All rights reserved.
*
* Licensed under your choice of the Redis Source Available License 2.0
* (RSALv2) or the Server Side Public License v1 (SSPLv1).
*
* Portions of this file are available under BSD3 terms; see REDISCONTRIBUTIONS for more information.
*/
#ifndef CLUSTER_LEGACY_H
#define CLUSTER_LEGACY_H
@@ -51,6 +64,7 @@ typedef struct clusterLink {
#define CLUSTER_NODE_MEET 128 /* Send a MEET message to this node */
#define CLUSTER_NODE_MIGRATE_TO 256 /* Master eligible for replica migration. */
#define CLUSTER_NODE_NOFAILOVER 512 /* Slave will not try to failover. */
#define CLUSTER_NODE_EXTENSIONS_SUPPORTED 1024 /* This node supports extensions. */
#define CLUSTER_NODE_NULL_NAME "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"
#define nodeIsSlave(n) ((n)->flags & CLUSTER_NODE_SLAVE)
@@ -59,6 +73,7 @@ typedef struct clusterLink {
#define nodeTimedOut(n) ((n)->flags & CLUSTER_NODE_PFAIL)
#define nodeFailed(n) ((n)->flags & CLUSTER_NODE_FAIL)
#define nodeCantFailover(n) ((n)->flags & CLUSTER_NODE_NOFAILOVER)
#define nodeSupportsExtensions(n) ((n)->flags & CLUSTER_NODE_EXTENSIONS_SUPPORTED)
/* This structure represent elements of node->fail_reports. */
typedef struct clusterNodeFailReport {
+3
View File
@@ -2450,6 +2450,7 @@ static int updatePort(const char **err) {
listener->bindaddr = server.bindaddr;
listener->bindaddr_count = server.bindaddr_count;
listener->port = server.port;
clusterUpdateMyselfAnnouncedPorts();
listener->ct = connectionByType(CONN_TYPE_SOCKET);
if (changeListener(listener) == C_ERR) {
*err = "Unable to listen on this port. Check server logs.";
@@ -2671,6 +2672,7 @@ static int applyTLSPort(const char **err) {
listener->bindaddr_count = server.bindaddr_count;
listener->port = server.tls_port;
listener->ct = connectionByType(CONN_TYPE_TLS);
clusterUpdateMyselfAnnouncedPorts();
if (changeListener(listener) == C_ERR) {
*err = "Unable to listen on this port. Check server logs.";
return 0;
@@ -3093,6 +3095,7 @@ standardConfig static_configs[] = {
createBoolConfig("aof-disable-auto-gc", NULL, MODIFIABLE_CONFIG | HIDDEN_CONFIG, server.aof_disable_auto_gc, 0, NULL, updateAofAutoGCEnabled),
createBoolConfig("replica-ignore-disk-write-errors", NULL, MODIFIABLE_CONFIG, server.repl_ignore_disk_write_error, 0, NULL, NULL),
createBoolConfig("hide-user-data-from-log", NULL, MODIFIABLE_CONFIG, server.hide_user_data_from_log, 0, NULL, NULL),
createBoolConfig("lua-enable-deprecated-api", NULL, IMMUTABLE_CONFIG | HIDDEN_CONFIG, server.lua_enable_deprecated_api, 0, NULL, NULL),
/* String Configs */
createStringConfig("aclfile", NULL, IMMUTABLE_CONFIG, ALLOW_EMPTY_STRING, server.acl_filename, "", NULL, NULL),
+2 -2
View File
@@ -348,7 +348,7 @@ robj *dbRandomKey(redisDb *db) {
key = dictGetKey(de);
keyobj = createStringObject(key,sdslen(key));
if (dbFindExpires(db, key)) {
if (allvolatile && server.masterhost && --maxtries == 0) {
if (allvolatile && (server.masterhost || isPausedActions(PAUSE_ACTION_EXPIRE)) && --maxtries == 0) {
/* If the DB is composed only of keys with an expire set,
* it could happen that all the keys are already logically
* expired in the slave, so the function cannot stop because
@@ -702,7 +702,7 @@ void flushallSyncBgDone(uint64_t client_id) {
client *c = lookupClientByID(client_id);
/* Verify that client still exists */
if (!c) return;
if (!(c && c->flags & CLIENT_BLOCKED)) return;
/* Update current_client (Called functions might rely on it) */
client *old_client = server.current_client;
+33 -23
View File
@@ -266,6 +266,16 @@ void activeDefragSdsDictCallback(void *privdata, const dictEntry *de) {
UNUSED(de);
}
void activeDefragLuaScriptDictCallback(void *privdata, const dictEntry *de) {
UNUSED(privdata);
/* If this luaScript is in the LRU list, unconditionally update the node's
* value pointer to the current dict key (regardless of reallocation). */
luaScript *script = dictGetVal(de);
if (script->node)
script->node->value = dictGetKey(de);
}
void activeDefragHfieldDictCallback(void *privdata, const dictEntry *de) {
dict *d = privdata;
hfield newhf, hf = dictGetKey(de);
@@ -303,8 +313,10 @@ void activeDefragSdsDict(dict* d, int val_type) {
val_type == DEFRAG_SDS_DICT_VAL_LUA_SCRIPT ? (dictDefragAllocFunction *)activeDefragLuaScript :
NULL)
};
dictScanFunction *fn = (val_type == DEFRAG_SDS_DICT_VAL_LUA_SCRIPT ?
activeDefragLuaScriptDictCallback : activeDefragSdsDictCallback);
do {
cursor = dictScanDefrag(d, cursor, activeDefragSdsDictCallback,
cursor = dictScanDefrag(d, cursor, fn,
&defragfns, NULL);
} while (cursor != 0);
}
@@ -394,8 +406,7 @@ long scanLaterList(robj *ob, unsigned long *cursor, long long endtime) {
quicklistNode *node;
long iterations = 0;
int bookmark_failed = 0;
if (ob->type != OBJ_LIST || ob->encoding != OBJ_ENCODING_QUICKLIST)
return 0;
serverAssert(ob->type == OBJ_LIST && ob->encoding == OBJ_ENCODING_QUICKLIST);
if (*cursor == 0) {
/* if cursor is 0, we start new iteration */
@@ -444,8 +455,7 @@ void scanLaterZsetCallback(void *privdata, const dictEntry *_de) {
}
void scanLaterZset(robj *ob, unsigned long *cursor) {
if (ob->type != OBJ_ZSET || ob->encoding != OBJ_ENCODING_SKIPLIST)
return;
serverAssert(ob->type == OBJ_ZSET && ob->encoding == OBJ_ENCODING_SKIPLIST);
zset *zs = (zset*)ob->ptr;
dict *d = zs->dict;
scanLaterZsetData data = {zs};
@@ -461,8 +471,7 @@ void scanCallbackCountScanned(void *privdata, const dictEntry *de) {
}
void scanLaterSet(robj *ob, unsigned long *cursor) {
if (ob->type != OBJ_SET || ob->encoding != OBJ_ENCODING_HT)
return;
serverAssert(ob->type == OBJ_SET && ob->encoding == OBJ_ENCODING_HT);
dict *d = ob->ptr;
dictDefragFunctions defragfns = {
.defragAlloc = activeDefragAlloc,
@@ -472,8 +481,7 @@ void scanLaterSet(robj *ob, unsigned long *cursor) {
}
void scanLaterHash(robj *ob, unsigned long *cursor) {
if (ob->type != OBJ_HASH || ob->encoding != OBJ_ENCODING_HT)
return;
serverAssert(ob->type == OBJ_HASH && ob->encoding == OBJ_ENCODING_HT);
dict *d = ob->ptr;
dictDefragFunctions defragfns = {
.defragAlloc = activeDefragAlloc,
@@ -568,10 +576,7 @@ int scanLaterStreamListpacks(robj *ob, unsigned long *cursor, long long endtime)
static unsigned char last[sizeof(streamID)];
raxIterator ri;
long iterations = 0;
if (ob->type != OBJ_STREAM || ob->encoding != OBJ_ENCODING_STREAM) {
*cursor = 0;
return 0;
}
serverAssert(ob->type == OBJ_STREAM && ob->encoding == OBJ_ENCODING_STREAM);
stream *s = ob->ptr;
raxStart(&ri,s->rax);
@@ -718,8 +723,9 @@ void defragStream(redisDb *db, dictEntry *kde) {
void defragModule(redisDb *db, dictEntry *kde) {
robj *obj = dictGetVal(kde);
serverAssert(obj->type == OBJ_MODULE);
if (!moduleDefragValue(dictGetKey(kde), obj, db->id))
robj keyobj;
initStaticStringObject(keyobj, dictGetKey(kde));
if (!moduleDefragValue(&keyobj, obj, db->id))
defragLater(db, kde);
}
@@ -886,7 +892,9 @@ void defragPubsubScanCallback(void *privdata, const dictEntry *de) {
dictEntry *clientde;
while((clientde = dictNext(di)) != NULL) {
client *c = dictGetKey(clientde);
dictEntry *pubsub_channel = dictFind(pubsub_ctx->clientPubSubChannels(c), newchannel);
dict *client_channels = pubsub_ctx->clientPubSubChannels(c);
uint64_t hash = dictGetHash(client_channels, newchannel);
dictEntry *pubsub_channel = dictFindByHashAndPtr(client_channels, channel, hash);
serverAssert(pubsub_channel);
dictSetKey(pubsub_ctx->clientPubSubChannels(c), pubsub_channel, newchannel);
}
@@ -918,20 +926,22 @@ void defragOtherGlobals(void) {
int defragLaterItem(dictEntry *de, unsigned long *cursor, long long endtime, int dbid) {
if (de) {
robj *ob = dictGetVal(de);
if (ob->type == OBJ_LIST) {
if (ob->type == OBJ_LIST && ob->encoding == OBJ_ENCODING_QUICKLIST) {
return scanLaterList(ob, cursor, endtime);
} else if (ob->type == OBJ_SET) {
} else if (ob->type == OBJ_SET && ob->encoding == OBJ_ENCODING_HT) {
scanLaterSet(ob, cursor);
} else if (ob->type == OBJ_ZSET) {
} else if (ob->type == OBJ_ZSET && ob->encoding == OBJ_ENCODING_SKIPLIST) {
scanLaterZset(ob, cursor);
} else if (ob->type == OBJ_HASH) {
} else if (ob->type == OBJ_HASH && ob->encoding == OBJ_ENCODING_HT) {
scanLaterHash(ob, cursor);
} else if (ob->type == OBJ_STREAM) {
} else if (ob->type == OBJ_STREAM && ob->encoding == OBJ_ENCODING_STREAM) {
return scanLaterStreamListpacks(ob, cursor, endtime);
} else if (ob->type == OBJ_MODULE) {
return moduleLateDefrag(dictGetKey(de), ob, cursor, endtime, dbid);
robj keyobj;
initStaticStringObject(keyobj, dictGetKey(de));
return moduleLateDefrag(&keyobj, ob, cursor, endtime, dbid);
} else {
*cursor = 0; /* object type may have changed since we schedule it for later */
*cursor = 0; /* object type/encoding may have changed since we schedule it for later */
}
} else {
*cursor = 0; /* object may have been deleted already */
+39
View File
@@ -260,12 +260,16 @@ int _dictResize(dict *d, unsigned long size, int* malloc_failed)
d->ht_table[1] = new_ht_table;
d->rehashidx = 0;
if (d->type->rehashingStarted) d->type->rehashingStarted(d);
if (d->type->bucketChanged)
d->type->bucketChanged(d, DICTHT_SIZE(d->ht_size_exp[1]));
/* Is this the first initialization or is the first hash table empty? If so
* it's not really a rehashing, we can just set the first hash table so that
* it can accept keys. */
if (d->ht_table[0] == NULL || d->ht_used[0] == 0) {
if (d->type->rehashingCompleted) d->type->rehashingCompleted(d);
if (d->type->bucketChanged)
d->type->bucketChanged(d, -(long long)DICTHT_SIZE(d->ht_size_exp[0]));
if (d->ht_table[0]) zfree(d->ht_table[0]);
d->ht_size_exp[0] = new_ht_size_exp;
d->ht_used[0] = new_ht_used;
@@ -363,6 +367,8 @@ static int dictCheckRehashingCompleted(dict *d) {
if (d->ht_used[0] != 0) return 0;
if (d->type->rehashingCompleted) d->type->rehashingCompleted(d);
if (d->type->bucketChanged)
d->type->bucketChanged(d, -(long long)DICTHT_SIZE(d->ht_size_exp[0]));
zfree(d->ht_table[0]);
/* Copy the new ht onto the old one */
d->ht_table[0] = d->ht_table[1];
@@ -725,6 +731,10 @@ void dictRelease(dict *d)
if (dictIsRehashing(d) && d->type->rehashingCompleted)
d->type->rehashingCompleted(d);
/* Subtract the size of all buckets. */
if (d->type->bucketChanged)
d->type->bucketChanged(d, -(long long)dictBuckets(d));
if (d->type->onDictRelease)
d->type->onDictRelease(d);
@@ -733,6 +743,30 @@ void dictRelease(dict *d)
zfree(d);
}
/* Finds the dictEntry using pointer and pre-calculated hash.
* oldkey is a dead pointer and should not be accessed.
* the hash value should be provided using dictGetHash.
* no string / key comparison is performed.
* return value is a pointer to the dictEntry if found, or NULL if not found. */
dictEntry *dictFindByHashAndPtr(dict *d, const void *oldptr, const uint64_t hash) {
dictEntry *he;
unsigned long idx, table;
if (dictSize(d) == 0) return NULL; /* dict is empty */
for (table = 0; table <= 1; table++) {
idx = hash & DICTHT_SIZE_MASK(d->ht_size_exp[table]);
if (table == 0 && (long)idx < d->rehashidx) continue;
he = d->ht_table[table][idx];
while(he) {
if (oldptr == dictGetKey(he))
return he;
he = dictGetNext(he);
}
if (!dictIsRehashing(d)) return NULL;
}
return NULL;
}
dictEntry *dictFind(dict *d, const void *key)
{
dictEntry *he;
@@ -1619,6 +1653,11 @@ void dictEmpty(dict *d, void(callback)(dict*)) {
* destroying the dict fake completion. */
if (dictIsRehashing(d) && d->type->rehashingCompleted)
d->type->rehashingCompleted(d);
/* Subtract the size of all buckets. */
if (d->type->bucketChanged)
d->type->bucketChanged(d, -(long long)dictBuckets(d));
_dictClear(d,0,callback);
_dictClear(d,1,callback);
d->rehashidx = -1;
+4
View File
@@ -43,6 +43,9 @@ typedef struct dictType {
/* Invoked at the end of dict initialization/rehashing of all the entries from old to new ht. Both ht still exists
* and are cleaned up after this callback. */
void (*rehashingCompleted)(dict *d);
/* Invoked when the size of the dictionary changes.
* The `delta` parameter can be positive (size increase) or negative (size decrease). */
void (*bucketChanged)(dict *d, long long delta);
/* Allow a dict to carry extra caller-defined metadata. The
* extra memory is initialized to 0 when a dict is allocated. */
size_t (*dictMetadataBytes)(dict *d);
@@ -206,6 +209,7 @@ void dictFreeUnlinkedEntry(dict *d, dictEntry *he);
dictEntry *dictTwoPhaseUnlinkFind(dict *d, const void *key, dictEntry ***plink, int *table_index);
void dictTwoPhaseUnlinkFree(dict *d, dictEntry *he, dictEntry **plink, int table_index);
void dictRelease(dict *d);
dictEntry *dictFindByHashAndPtr(dict *d, const void *oldptr, const uint64_t hash);
dictEntry * dictFind(dict *d, const void *key);
void *dictFetchValue(dict *d, const void *key);
int dictShrinkIfNeeded(dict *d);
+17 -6
View File
@@ -251,6 +251,8 @@ void scriptingInit(int setup) {
/* Recursively lock all tables that can be reached from the global table */
luaSetTableProtectionRecursively(lua);
lua_pop(lua, 1);
/* Set metatables of basic types (string, number, nil etc.) readonly. */
luaSetTableProtectionForBasicTypes(lua);
lctx.lua = lua;
}
@@ -259,12 +261,17 @@ void scriptingInit(int setup) {
void freeLuaScriptsSync(dict *lua_scripts, list *lua_scripts_lru_list, lua_State *lua) {
dictRelease(lua_scripts);
listRelease(lua_scripts_lru_list);
lua_close(lua);
#if defined(USE_JEMALLOC)
/* When lua is closed, destroy the previously used private tcache. */
void *ud = (global_State*)G(lua)->ud;
unsigned int lua_tcache = (unsigned int)(uintptr_t)ud;
#endif
lua_gc(lua, LUA_GCCOLLECT, 0);
lua_close(lua);
#if defined(USE_JEMALLOC)
je_mallctl("tcache.destroy", NULL, NULL, (void *)&lua_tcache, sizeof(unsigned int));
#endif
}
@@ -605,17 +612,21 @@ void evalGenericCommand(client *c, int evalsha) {
rctx.flags |= SCRIPT_EVAL_MODE; /* mark the current run as EVAL (as opposed to FCALL) so we'll
get appropriate error messages and logs */
luaCallFunction(&rctx, lua, c->argv+3, numkeys, c->argv+3+numkeys, c->argc-3-numkeys, ldb.active);
lua_pop(lua,1); /* Remove the error handler. */
scriptResetRun(&rctx);
luaGC(lua, &gc_count);
if (l->node) {
/* Quick removal and re-insertion after the script is called to
* maintain the LRU list. */
listUnlinkNode(lctx.lua_scripts_lru_list, l->node);
listLinkNodeTail(lctx.lua_scripts_lru_list, l->node);
}
luaCallFunction(&rctx, lua, c->argv+3, numkeys, c->argv+3+numkeys, c->argc-3-numkeys, ldb.active);
lua_pop(lua,1); /* Remove the error handler. */
scriptResetRun(&rctx);
luaGC(lua, &gc_count);
/* We can no longer touch 'l' here, as it may have been reallocated by activedefrag
* during AOF loading of long-running scripts. This issue is not with newly generated
* AOF files, in which scripts propagate effects rather than scripts. */
}
void evalCommand(client *c) {
+16
View File
@@ -23,6 +23,9 @@
#include <lua.h>
#include <lauxlib.h>
#include <lualib.h>
#if defined(USE_JEMALLOC)
#include <lstate.h>
#endif
#define LUA_ENGINE_NAME "LUA"
#define REGISTRY_ENGINE_CTX_NAME "__ENGINE_CTX__"
@@ -189,8 +192,19 @@ static void luaEngineFreeFunction(void *engine_ctx, void *compiled_function) {
static void luaEngineFreeCtx(void *engine_ctx) {
luaEngineCtx *lua_engine_ctx = engine_ctx;
#if defined(USE_JEMALLOC)
/* When lua is closed, destroy the previously used private tcache. */
void *ud = (global_State*)G(lua_engine_ctx->lua)->ud;
unsigned int lua_tcache = (unsigned int)(uintptr_t)ud;
#endif
lua_gc(lua_engine_ctx->lua, LUA_GCCOLLECT, 0);
lua_close(lua_engine_ctx->lua);
zfree(lua_engine_ctx);
#if defined(USE_JEMALLOC)
je_mallctl("tcache.destroy", NULL, NULL, (void *)&lua_tcache, sizeof(unsigned int));
#endif
}
static void luaRegisterFunctionArgsInitialize(registerFunctionArgs *register_f_args,
@@ -480,6 +494,8 @@ int luaEngineInitEngine(void) {
lua_enablereadonlytable(lua_engine_ctx->lua, -1, 1); /* protect the new global table */
lua_replace(lua_engine_ctx->lua, LUA_GLOBALSINDEX); /* set new global table as the new globals */
/* Set metatables of basic types (string, number, nil etc.) readonly. */
luaSetTableProtectionForBasicTypes(lua_engine_ctx->lua);
engine *lua_engine = zmalloc(sizeof(*lua_engine));
*lua_engine = (engine) {
+1 -1
View File
@@ -171,7 +171,7 @@ void functionsLibCtxClear(functionsLibCtx *lib_ctx) {
stats->n_lib = 0;
}
dictReleaseIterator(iter);
curr_functions_lib_ctx->cache_memory = 0;
lib_ctx->cache_memory = 0;
}
void functionsLibCtxClearCurrent(int async) {
+42 -5
View File
@@ -566,6 +566,7 @@ int hllSparseToDense(robj *o) {
struct hllhdr *hdr, *oldhdr = (struct hllhdr*)sparse;
int idx = 0, runlen, regval;
uint8_t *p = (uint8_t*)sparse, *end = p+sdslen(sparse);
int valid = 1;
/* If the representation is already the right one return ASAP. */
hdr = (struct hllhdr*) sparse;
@@ -585,16 +586,27 @@ int hllSparseToDense(robj *o) {
while(p < end) {
if (HLL_SPARSE_IS_ZERO(p)) {
runlen = HLL_SPARSE_ZERO_LEN(p);
if ((runlen + idx) > HLL_REGISTERS) { /* Overflow. */
valid = 0;
break;
}
idx += runlen;
p++;
} else if (HLL_SPARSE_IS_XZERO(p)) {
runlen = HLL_SPARSE_XZERO_LEN(p);
if ((runlen + idx) > HLL_REGISTERS) { /* Overflow. */
valid = 0;
break;
}
idx += runlen;
p += 2;
} else {
runlen = HLL_SPARSE_VAL_LEN(p);
regval = HLL_SPARSE_VAL_VALUE(p);
if ((runlen + idx) > HLL_REGISTERS) break; /* Overflow. */
if ((runlen + idx) > HLL_REGISTERS) { /* Overflow. */
valid = 0;
break;
}
while(runlen--) {
HLL_DENSE_SET_REGISTER(hdr->registers,idx,regval);
idx++;
@@ -605,7 +617,7 @@ int hllSparseToDense(robj *o) {
/* If the sparse representation was valid, we expect to find idx
* set to HLL_REGISTERS. */
if (idx != HLL_REGISTERS) {
if (!valid || idx != HLL_REGISTERS) {
sdsfree(dense);
return C_ERR;
}
@@ -902,27 +914,40 @@ int hllSparseAdd(robj *o, unsigned char *ele, size_t elesize) {
void hllSparseRegHisto(uint8_t *sparse, int sparselen, int *invalid, int* reghisto) {
int idx = 0, runlen, regval;
uint8_t *end = sparse+sparselen, *p = sparse;
int valid = 1;
while(p < end) {
if (HLL_SPARSE_IS_ZERO(p)) {
runlen = HLL_SPARSE_ZERO_LEN(p);
if ((runlen + idx) > HLL_REGISTERS) { /* Overflow. */
valid = 0;
break;
}
idx += runlen;
reghisto[0] += runlen;
p++;
} else if (HLL_SPARSE_IS_XZERO(p)) {
runlen = HLL_SPARSE_XZERO_LEN(p);
if ((runlen + idx) > HLL_REGISTERS) { /* Overflow. */
valid = 0;
break;
}
idx += runlen;
reghisto[0] += runlen;
p += 2;
} else {
runlen = HLL_SPARSE_VAL_LEN(p);
regval = HLL_SPARSE_VAL_VALUE(p);
if ((runlen + idx) > HLL_REGISTERS) { /* Overflow. */
valid = 0;
break;
}
idx += runlen;
reghisto[regval] += runlen;
p++;
}
}
if (idx != HLL_REGISTERS && invalid) *invalid = 1;
if ((!valid || idx != HLL_REGISTERS) && invalid) *invalid = 1;
}
/* ========================= HyperLogLog Count ==============================
@@ -1070,22 +1095,34 @@ int hllMerge(uint8_t *max, robj *hll) {
} else {
uint8_t *p = hll->ptr, *end = p + sdslen(hll->ptr);
long runlen, regval;
int valid = 1;
p += HLL_HDR_SIZE;
i = 0;
while(p < end) {
if (HLL_SPARSE_IS_ZERO(p)) {
runlen = HLL_SPARSE_ZERO_LEN(p);
if ((runlen + i) > HLL_REGISTERS) { /* Overflow. */
valid = 0;
break;
}
i += runlen;
p++;
} else if (HLL_SPARSE_IS_XZERO(p)) {
runlen = HLL_SPARSE_XZERO_LEN(p);
if ((runlen + i) > HLL_REGISTERS) { /* Overflow. */
valid = 0;
break;
}
i += runlen;
p += 2;
} else {
runlen = HLL_SPARSE_VAL_LEN(p);
regval = HLL_SPARSE_VAL_VALUE(p);
if ((runlen + i) > HLL_REGISTERS) break; /* Overflow. */
if ((runlen + i) > HLL_REGISTERS) { /* Overflow. */
valid = 0;
break;
}
while(runlen--) {
if (regval > max[i]) max[i] = regval;
i++;
@@ -1093,7 +1130,7 @@ int hllMerge(uint8_t *max, robj *hll) {
p++;
}
}
if (i != HLL_REGISTERS) return C_ERR;
if (!valid || i != HLL_REGISTERS) return C_ERR;
}
return C_OK;
}
+38 -12
View File
@@ -40,7 +40,6 @@ struct _kvstore {
unsigned long long key_count; /* Total number of keys in this kvstore. */
unsigned long long bucket_count; /* Total number of buckets in this kvstore across dictionaries. */
unsigned long long *dict_size_index; /* Binary indexed tree (BIT) that describes cumulative key frequencies up until given dict-index. */
size_t overhead_hashtable_lut; /* The overhead of all dictionaries. */
size_t overhead_hashtable_rehashing; /* The overhead of dictionaries rehashing. */
};
@@ -124,7 +123,9 @@ static void cumulativeKeyCountAdd(kvstore *kvs, int didx, long delta) {
dict *d = kvstoreGetDict(kvs, didx);
size_t dsize = dictSize(d);
int non_empty_dicts_delta = dsize == 1? 1 : dsize == 0? -1 : 0;
/* Increment if dsize is 1 and delta is positive (first element inserted, dict becomes non-empty).
* Decrement if dsize is 0 (dict becomes empty). */
int non_empty_dicts_delta = (dsize == 1 && delta > 0) ? 1 : (dsize == 0) ? -1 : 0;
kvs->non_empty_dicts += non_empty_dicts_delta;
/* BIT does not need to be calculated when there's only one dict. */
@@ -188,8 +189,6 @@ static void kvstoreDictRehashingStarted(dict *d) {
unsigned long long from, to;
dictRehashingInfo(d, &from, &to);
kvs->bucket_count += to; /* Started rehashing (Add the new ht size) */
kvs->overhead_hashtable_lut += to;
kvs->overhead_hashtable_rehashing += from;
}
@@ -207,11 +206,17 @@ static void kvstoreDictRehashingCompleted(dict *d) {
unsigned long long from, to;
dictRehashingInfo(d, &from, &to);
kvs->bucket_count -= from; /* Finished rehashing (Remove the old ht size) */
kvs->overhead_hashtable_lut -= from;
kvs->overhead_hashtable_rehashing -= from;
}
/* Updates the bucket count for the given dictionary in a DB. It adds the new ht size
* of the dictionary or removes the old ht size of the dictionary from the total
* sum of buckets for a DB. */
static void kvstoreDictBucketChanged(dict *d, long long delta) {
kvstore *kvs = d->type->userdata;
kvs->bucket_count += delta;
}
/* Returns the size of the DB dict metadata in bytes. */
static size_t kvstoreDictMetadataSize(dict *d) {
UNUSED(d);
@@ -244,6 +249,7 @@ kvstore *kvstoreCreate(dictType *type, int num_dicts_bits, int flags) {
kvs->dtype.dictMetadataBytes = kvstoreDictMetadataSize;
kvs->dtype.rehashingStarted = kvstoreDictRehashingStarted;
kvs->dtype.rehashingCompleted = kvstoreDictRehashingCompleted;
kvs->dtype.bucketChanged = kvstoreDictBucketChanged;
kvs->num_dicts_bits = num_dicts_bits;
kvs->num_dicts = 1 << kvs->num_dicts_bits;
@@ -259,7 +265,6 @@ kvstore *kvstoreCreate(dictType *type, int num_dicts_bits, int flags) {
kvs->resize_cursor = 0;
kvs->dict_size_index = kvs->num_dicts > 1? zcalloc(sizeof(unsigned long long) * (kvs->num_dicts + 1)) : NULL;
kvs->bucket_count = 0;
kvs->overhead_hashtable_lut = 0;
kvs->overhead_hashtable_rehashing = 0;
return kvs;
@@ -285,7 +290,6 @@ void kvstoreEmpty(kvstore *kvs, void(callback)(dict*)) {
kvs->bucket_count = 0;
if (kvs->dict_size_index)
memset(kvs->dict_size_index, 0, sizeof(unsigned long long) * (kvs->num_dicts + 1));
kvs->overhead_hashtable_lut = 0;
kvs->overhead_hashtable_rehashing = 0;
}
@@ -337,9 +341,6 @@ size_t kvstoreMemUsage(kvstore *kvs) {
/* Values are dict* shared with kvs->dicts */
mem += listLength(kvs->rehashing) * sizeof(listNode);
if (kvs->dict_size_index)
mem += sizeof(unsigned long long) * (kvs->num_dicts + 1);
return mem;
}
@@ -659,7 +660,7 @@ uint64_t kvstoreIncrementallyRehash(kvstore *kvs, uint64_t threshold_us) {
}
size_t kvstoreOverheadHashtableLut(kvstore *kvs) {
return kvs->overhead_hashtable_lut * sizeof(dictEntry *);
return kvs->bucket_count * sizeof(dictEntry *);
}
size_t kvstoreOverheadHashtableRehashing(kvstore *kvs) {
@@ -1026,6 +1027,31 @@ int kvstoreTest(int argc, char **argv, int flags) {
kvstoreRelease(kvs);
}
TEST("Verify non-empty dict count is correctly updated") {
kvstore *kvs = kvstoreCreate(&KvstoreDictTestType, 2, KVSTORE_ALLOCATE_DICTS_ON_DEMAND);
for (int idx = 0; idx < 4; idx++) {
for (i = 0; i < 16; i++) {
de = kvstoreDictAddRaw(kvs, idx, stringFromInt(i), NULL);
assert(de != NULL);
/* When the first element is inserted, the number of non-empty dictionaries is increased by 1. */
if (i == 0) assert(kvstoreNumNonEmptyDicts(kvs) == idx + 1);
}
}
/* Step by step, clear all dictionaries and ensure non-empty dict count is updated */
for (int idx = 0; idx < 4; idx++) {
kvs_di = kvstoreGetDictSafeIterator(kvs, idx);
while((de = kvstoreDictIteratorNext(kvs_di)) != NULL) {
key = dictGetKey(de);
assert(kvstoreDictDelete(kvs, idx, key) == DICT_OK);
/* When the dictionary is emptied, the number of non-empty dictionaries is reduced by 1. */
if (kvstoreDictSize(kvs, idx) == 0) assert(kvstoreNumNonEmptyDicts(kvs) == 3 - idx);
}
kvstoreReleaseDictIterator(kvs_di);
}
kvstoreRelease(kvs);
}
kvstoreRelease(kvs1);
kvstoreRelease(kvs2);
return 0;
+8 -5
View File
@@ -12364,7 +12364,7 @@ int moduleLoad(const char *path, void **module_argv, int module_argc, int is_loa
}
if (post_load_err) {
moduleUnload(ctx.module->name, NULL);
serverAssert(moduleUnload(ctx.module->name, NULL, 1) == C_OK);
moduleFreeContext(&ctx);
return C_ERR;
}
@@ -12380,14 +12380,17 @@ int moduleLoad(const char *path, void **module_argv, int module_argc, int is_loa
/* Unload the module registered with the specified name. On success
* C_OK is returned, otherwise C_ERR is returned and errmsg is set
* with an appropriate message. */
int moduleUnload(sds name, const char **errmsg) {
* with an appropriate message.
* Only forcefully unload this module, passing forced_unload != 0,
* if it is certain that it has not yet been in use (e.g., immediate
* unload on failed load). */
int moduleUnload(sds name, const char **errmsg, int forced_unload) {
struct RedisModule *module = dictFetchValue(modules,name);
if (module == NULL) {
*errmsg = "no such module with that name";
return C_ERR;
} else if (listLength(module->types)) {
} else if (listLength(module->types) && !forced_unload) {
*errmsg = "the module exports one or more module-side data "
"types, can't unload";
return C_ERR;
@@ -13170,7 +13173,7 @@ NULL
} else if (!strcasecmp(subcmd,"unload") && c->argc == 3) {
const char *errmsg = NULL;
if (moduleUnload(c->argv[2]->ptr, &errmsg) == C_OK)
if (moduleUnload(c->argv[2]->ptr, &errmsg, 0) == C_OK)
addReply(c,shared.ok);
else {
if (errmsg == NULL) errmsg = "operation not possible.";
+8 -2
View File
@@ -3302,11 +3302,12 @@ NULL
if (getLongLongFromObjectOrReply(c,c->argv[2],&id,NULL)
!= C_OK) return;
struct client *target = lookupClientByID(id);
/* Note that we never try to unblock a client blocked on a module command, which
/* Note that we never try to unblock a client blocked on a module command,
* or a client blocked by CLIENT PAUSE or some other blocking type which
* doesn't have a timeout callback (even in the case of UNBLOCK ERROR).
* The reason is that we assume that if a command doesn't expect to be timedout,
* it also doesn't expect to be unblocked by CLIENT UNBLOCK */
if (target && target->flags & CLIENT_BLOCKED && moduleBlockedClientMayTimeout(target)) {
if (target && target->flags & CLIENT_BLOCKED && blockedClientMayTimeout(target)) {
if (unblock_error)
unblockClientOnError(target,
"-UNBLOCKED client unblocked via CLIENT UNBLOCK");
@@ -3926,6 +3927,11 @@ int checkClientOutputBufferLimits(client *c) {
int soft = 0, hard = 0, class;
unsigned long used_mem = getClientOutputBufferMemoryUsage(c);
/* For unauthenticated clients the output buffer is limited to prevent
* them from abusing it by not reading the replies */
if (used_mem > 1024 && authRequired(c))
return 1;
class = getClientType(c);
/* For the purpose of output buffer limiting, masters are handled
* like normal clients. */
+1
View File
@@ -2332,6 +2332,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb, sds key, int dbid, int *error)
rdbReportCorruptRDB("invalid expireAt time: %llu",
(unsigned long long) expireAt);
decrRefCount(o);
if (dupSearchDict != NULL) dictRelease(dupSearchDict);
return NULL;
}
+6
View File
@@ -556,6 +556,12 @@ int redis_check_aof_main(int argc, char **argv) {
goto invalid_args;
}
/* Check if filepath is longer than PATH_MAX */
if (strlen(filepath) > PATH_MAX) {
printf("Error: filepath is too long (exceeds PATH_MAX)\n");
goto invalid_args;
}
/* In the glibc implementation dirname may modify their argument. */
memcpy(temp_filepath, filepath, strlen(filepath) + 1);
dirpath = dirname(temp_filepath);
+2 -2
View File
@@ -64,7 +64,7 @@ lua_State *createLuaState(void) {
size_t sz = sizeof(unsigned int);
int err = je_mallctl("tcache.create", (void *)&tcache, &sz, NULL, 0);
if (err) {
serverLog(LL_WARNING, "Failed creating the lua jemalloc tcache.");
serverLog(LL_WARNING, "Failed creating the lua jemalloc tcache (err=%d).", err);
exit(1);
}
@@ -79,7 +79,7 @@ void luaEnvInit(void) {
size_t sz = sizeof(unsigned int);
int err = je_mallctl("arenas.create", (void *)&arena, &sz, NULL, 0);
if (err) {
serverLog(LL_WARNING, "Failed creating the lua jemalloc arena.");
serverLog(LL_WARNING, "Failed creating the lua jemalloc arena (err=%d).", err);
exit(1);
}
server.lua_arena = arena;
+60 -7
View File
@@ -2,8 +2,13 @@
* Copyright (c) 2009-Present, Redis Ltd.
* All rights reserved.
*
* Copyright (c) 2024-present, Valkey contributors.
* All rights reserved.
*
* Licensed under your choice of the Redis Source Available License 2.0
* (RSALv2) or the Server Side Public License v1 (SSPLv1).
*
* Portions of this file are available under BSD3 terms; see REDISCONTRIBUTIONS for more information.
*/
#include "script_lua.h"
@@ -46,7 +51,6 @@ static char *redis_api_allow_list[] = {
static char *lua_builtins_allow_list[] = {
"xpcall",
"tostring",
"getfenv",
"setmetatable",
"next",
"assert",
@@ -67,15 +71,16 @@ static char *lua_builtins_allow_list[] = {
"loadstring",
"ipairs",
"_VERSION",
"setfenv",
"load",
"error",
NULL,
};
/* Lua builtins which are not documented on the Lua documentation */
static char *lua_builtins_not_documented_allow_list[] = {
/* Lua builtins which are deprecated for sandboxing concerns */
static char *lua_builtins_deprecated[] = {
"newproxy",
"setfenv",
"getfenv",
NULL,
};
@@ -97,7 +102,6 @@ static char **allow_lists[] = {
libraries_allow_list,
redis_api_allow_list,
lua_builtins_allow_list,
lua_builtins_not_documented_allow_list,
lua_builtins_removed_after_initialization_allow_list,
NULL,
};
@@ -1300,7 +1304,22 @@ static int luaNewIndexAllowList(lua_State *lua) {
break;
}
}
if (!*allow_l) {
int allowed = (*allow_l != NULL);
/* If not explicitly allowed, check if it's a deprecated function. If so,
* allow it only if 'lua_enable_deprecated_api' config is enabled. */
int deprecated = 0;
if (!allowed) {
char **c = lua_builtins_deprecated;
for (; *c; ++c) {
if (strcmp(*c, variable_name) == 0) {
deprecated = 1;
allowed = server.lua_enable_deprecated_api ? 1 : 0;
break;
}
}
}
if (!allowed) {
/* Search the value on the back list, if its there we know that it was removed
* on purpose and there is no need to print a warning. */
char **c = deny_list;
@@ -1309,7 +1328,7 @@ static int luaNewIndexAllowList(lua_State *lua) {
break;
}
}
if (!*c) {
if (!*c && !deprecated) {
serverLog(LL_WARNING, "A key '%s' was added to Lua globals which is not on the globals allow list nor listed on the deny list.", variable_name);
}
} else {
@@ -1361,6 +1380,37 @@ void luaSetTableProtectionRecursively(lua_State *lua) {
}
}
/* Set the readonly flag on the metatable of basic types (string, nil etc.) */
void luaSetTableProtectionForBasicTypes(lua_State *lua) {
static const int types[] = {
LUA_TSTRING,
LUA_TNUMBER,
LUA_TBOOLEAN,
LUA_TNIL,
LUA_TFUNCTION,
LUA_TTHREAD,
LUA_TLIGHTUSERDATA
};
for (size_t i = 0; i < sizeof(types) / sizeof(types[0]); i++) {
/* Push a dummy value of the type to get its metatable */
switch (types[i]) {
case LUA_TSTRING: lua_pushstring(lua, ""); break;
case LUA_TNUMBER: lua_pushnumber(lua, 0); break;
case LUA_TBOOLEAN: lua_pushboolean(lua, 0); break;
case LUA_TNIL: lua_pushnil(lua); break;
case LUA_TFUNCTION: lua_pushcfunction(lua, NULL); break;
case LUA_TTHREAD: lua_newthread(lua); break;
case LUA_TLIGHTUSERDATA: lua_pushlightuserdata(lua, (void*)lua); break;
}
if (lua_getmetatable(lua, -1)) {
luaSetTableProtectionRecursively(lua);
lua_pop(lua, 1); /* pop metatable */
}
lua_pop(lua, 1); /* pop dummy value */
}
}
void luaRegisterVersion(lua_State* lua) {
lua_pushstring(lua,"REDIS_VERSION_NUM");
lua_pushnumber(lua,REDIS_VERSION_NUM);
@@ -1579,6 +1629,9 @@ void luaExtractErrorInformation(lua_State *lua, errorInfo *err_info) {
lua_getfield(lua, -1, "err");
if (lua_isstring(lua, -1)) {
err_info->msg = sdsnew(lua_tostring(lua, -1));
} else {
/* Ensure we never return a NULL msg. */
err_info->msg = sdsnew("ERR unknown error");
}
lua_pop(lua, 1);
+1
View File
@@ -50,6 +50,7 @@ void luaRegisterGlobalProtectionFunction(lua_State *lua);
void luaSetErrorMetatable(lua_State *lua);
void luaSetAllowListProtection(lua_State *lua);
void luaSetTableProtectionRecursively(lua_State *lua);
void luaSetTableProtectionForBasicTypes(lua_State *lua);
void luaRegisterLogFunction(lua_State* lua);
void luaRegisterVersion(lua_State* lua);
void luaPushErrorBuff(lua_State *lua, sds err_buff);
+3 -11
View File
@@ -1580,7 +1580,7 @@ void whileBlockedCron(void) {
* activeDefragCycle needs to utilize 25% cpu, it will utilize 2.5ms, so we
* need to call it multiple times. */
long hz_ms = 1000/server.hz;
while (server.blocked_last_cron < server.mstime) {
while (isAOFLoadingContext() && server.blocked_last_cron < server.mstime) {
/* Defrag keys gradually. */
activeDefragCycle();
@@ -1987,6 +1987,7 @@ void createSharedObjects(void) {
shared.special_asterick = createStringObject("*",1);
shared.special_equals = createStringObject("=",1);
shared.redacted = makeObjectShared(createStringObject("(redacted)",10));
shared.fields = createStringObject("FIELDS",6);
for (j = 0; j < OBJ_SHARED_INTEGERS; j++) {
shared.integers[j] =
@@ -3536,7 +3537,7 @@ void call(client *c, int flags) {
* and a client which is reprocessing command again (after being unblocked).
* Blocked clients can be blocked in different places and not always it means the call() function has been
* called. For example this is required for avoiding double logging to monitors.*/
int reprocessing_command = flags & CMD_CALL_REPROCESSING;
int reprocessing_command = (c->flags & CLIENT_REEXECUTING_COMMAND) ? 1 : 0;
/* Initialization: clear the flags that must be set by the command on
* demand, and initialize the array for additional commands propagation. */
@@ -3563,20 +3564,12 @@ void call(client *c, int flags) {
* re-processing and unblock the client.*/
c->flags |= CLIENT_EXECUTING_COMMAND;
/* Setting the CLIENT_REPROCESSING_COMMAND flag so that during the actual
* processing of the command proc, the client is aware that it is being
* re-processed. */
if (reprocessing_command) c->flags |= CLIENT_REPROCESSING_COMMAND;
monotime monotonic_start = 0;
if (monotonicGetType() == MONOTONIC_CLOCK_HW)
monotonic_start = getMonotonicUs();
c->cmd->proc(c);
/* Clear the CLIENT_REPROCESSING_COMMAND flag after the proc is executed. */
if (reprocessing_command) c->flags &= ~CLIENT_REPROCESSING_COMMAND;
exitExecutionUnit();
/* In case client is blocked after trying to execute the command,
@@ -4202,7 +4195,6 @@ int processCommand(client *c) {
addReply(c,shared.queued);
} else {
int flags = CMD_CALL_FULL;
if (client_reprocessing_command) flags |= CMD_CALL_REPROCESSING;
call(c,flags);
if (listLength(server.ready_keys) && !isInsideYieldingLongCommand())
handleClientsBlockedOnKeys();
+6 -5
View File
@@ -387,7 +387,7 @@ extern int configOOMScoreAdjValuesDefaults[CONFIG_OOM_COUNT];
auth had been authenticated from the Module. */
#define CLIENT_MODULE_PREVENT_AOF_PROP (1ULL<<48) /* Module client do not want to propagate to AOF */
#define CLIENT_MODULE_PREVENT_REPL_PROP (1ULL<<49) /* Module client do not want to propagate to replica */
#define CLIENT_REPROCESSING_COMMAND (1ULL<<50) /* The client is re-processing the command. */
#define CLIENT_REEXECUTING_COMMAND (1ULL<<50) /* The client is re-executing the command. */
/* Any flag that does not let optimize FLUSH SYNC to run it in bg as blocking client ASYNC */
#define CLIENT_AVOID_BLOCKING_ASYNC_FLUSH (CLIENT_DENY_BLOCKING|CLIENT_MULTI|CLIENT_LUA_DEBUG|CLIENT_LUA_DEBUG_SYNC|CLIENT_MODULE)
@@ -584,8 +584,7 @@ typedef enum {
#define CMD_CALL_NONE 0
#define CMD_CALL_PROPAGATE_AOF (1<<0)
#define CMD_CALL_PROPAGATE_REPL (1<<1)
#define CMD_CALL_REPROCESSING (1<<2)
#define CMD_CALL_FROM_MODULE (1<<3) /* From RM_Call */
#define CMD_CALL_FROM_MODULE (1<<2) /* From RM_Call */
#define CMD_CALL_PROPAGATE (CMD_CALL_PROPAGATE_AOF|CMD_CALL_PROPAGATE_REPL)
#define CMD_CALL_FULL (CMD_CALL_PROPAGATE)
@@ -1325,7 +1324,7 @@ struct sharedObjectsStruct {
*script, *replconf, *eval, *persist, *set, *pexpireat, *pexpire,
*hdel, *hpexpireat,
*time, *pxat, *absttl, *retrycount, *force, *justid, *entriesread,
*lastid, *ping, *setid, *keepttl, *load, *createconsumer,
*lastid, *ping, *setid, *keepttl, *load, *createconsumer, *fields,
*getack, *special_asterick, *special_equals, *default_username, *redacted,
*ssubscribebulk,*sunsubscribebulk, *smessagebulk,
*select[PROTO_SHARED_SELECT_CMDS],
@@ -2029,6 +2028,7 @@ struct redisServer {
mstime_t busy_reply_threshold; /* Script / module timeout in milliseconds */
int pre_command_oom_state; /* OOM before command (script?) was started */
int script_disable_deny_script; /* Allow running commands marked "noscript" inside a script. */
int lua_enable_deprecated_api; /* Config to enable deprecated api */
/* Lazy free */
int lazyfree_lazy_eviction;
int lazyfree_lazy_expire;
@@ -2510,7 +2510,7 @@ void moduleInitModulesSystem(void);
void moduleInitModulesSystemLast(void);
void modulesCron(void);
int moduleLoad(const char *path, void **argv, int argc, int is_loadex);
int moduleUnload(sds name, const char **errmsg);
int moduleUnload(sds name, const char **errmsg, int forced_unload);
void moduleLoadFromQueue(void);
int moduleGetCommandKeysViaAPI(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
int moduleGetCommandChannelsViaAPI(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
@@ -3502,6 +3502,7 @@ void unblockClient(client *c, int queue_for_reprocessing);
void unblockClientOnTimeout(client *c);
void unblockClientOnError(client *c, const char *err_str);
void queueClientForReprocessing(client *c);
int blockedClientMayTimeout(client *c);
void replyToBlockedClientTimedOut(client *c);
int getTimeoutFromObjectOrReply(client *c, robj *object, mstime_t *timeout, int unit);
void disconnectAllBlockedClients(void);
+2
View File
@@ -298,6 +298,8 @@ static void connSocketAcceptHandler(aeEventLoop *el, int fd, void *privdata, int
while(max--) {
cfd = anetTcpAccept(server.neterr, fd, cip, sizeof(cip), &cport);
if (cfd == ANET_ERR) {
if (anetAcceptFailureNeedsRetry(errno))
continue;
if (errno != EWOULDBLOCK)
serverLog(LL_WARNING,
"Accepting client connection: %s", server.neterr);
+6 -2
View File
@@ -241,8 +241,12 @@ void sortCommandGeneric(client *c, int readonly) {
} else if (!strcasecmp(c->argv[j]->ptr,"get") && leftargs >= 1) {
/* If GET is specified with a real pattern, we can't accept it in cluster mode,
* unless we can make sure the keys formed by the pattern are in the same slot
* as the key to sort. */
if (server.cluster_enabled && patternHashSlot(c->argv[j+1]->ptr, sdslen(c->argv[j+1]->ptr)) != getKeySlot(c->argv[1]->ptr)) {
* as the key to sort. The pattern # represents the key itself, so just skip
* pattern slot check. */
if (server.cluster_enabled &&
strcmp(c->argv[j+1]->ptr, "#") &&
patternHashSlot(c->argv[j+1]->ptr, sdslen(c->argv[j+1]->ptr)) != getKeySlot(c->argv[1]->ptr))
{
addReplyError(c, "GET option of SORT denied in Cluster mode when "
"keys formed by the pattern may be in different slots.");
syntax_error++;
+95 -14
View File
@@ -706,24 +706,29 @@ GetFieldRes hashTypeGetFromHashTable(robj *o, sds field, sds *value, uint64_t *e
* If *vll is populated *vstr is set to NULL, so the caller can
* always check the function return by checking the return value
* for GETF_OK and checking if vll (or vstr) is NULL.
* expiredAt - if the field has an expiration time, it will be set to the expiration
* time of the field. Otherwise, will be set to EB_EXPIRE_TIME_INVALID.
*
*/
GetFieldRes hashTypeGetValue(redisDb *db, robj *o, sds field, unsigned char **vstr,
unsigned int *vlen, long long *vll, int hfeFlags) {
uint64_t expiredAt;
unsigned int *vlen, long long *vll, int hfeFlags, uint64_t *expiredAt)
{
sds key;
GetFieldRes res;
uint64_t dummy;
if (expiredAt == NULL) expiredAt = &dummy;
if (o->encoding == OBJ_ENCODING_LISTPACK ||
o->encoding == OBJ_ENCODING_LISTPACK_EX) {
*vstr = NULL;
res = hashTypeGetFromListpack(o, field, vstr, vlen, vll, &expiredAt);
res = hashTypeGetFromListpack(o, field, vstr, vlen, vll, expiredAt);
if (res == GETF_NOT_FOUND)
return GETF_NOT_FOUND;
} else if (o->encoding == OBJ_ENCODING_HT) {
sds value = NULL;
res = hashTypeGetFromHashTable(o, field, &value, &expiredAt);
res = hashTypeGetFromHashTable(o, field, &value, expiredAt);
if (res == GETF_NOT_FOUND)
return GETF_NOT_FOUND;
@@ -734,7 +739,7 @@ GetFieldRes hashTypeGetValue(redisDb *db, robj *o, sds field, unsigned char **vs
serverPanic("Unknown hash encoding");
}
if (expiredAt >= (uint64_t) commandTimeSnapshot())
if (*expiredAt >= (uint64_t) commandTimeSnapshot())
return GETF_OK;
if (server.masterhost) {
@@ -794,7 +799,7 @@ robj *hashTypeGetValueObject(redisDb *db, robj *o, sds field, int hfeFlags, int
long long vll;
if (isHashDeleted) *isHashDeleted = 0;
GetFieldRes res = hashTypeGetValue(db,o,field,&vstr,&vlen,&vll, hfeFlags);
GetFieldRes res = hashTypeGetValue(db,o,field,&vstr,&vlen,&vll, hfeFlags, NULL);
if (res == GETF_OK) {
if (vstr) return createStringObject((char*)vstr,vlen);
@@ -823,7 +828,7 @@ int hashTypeExists(redisDb *db, robj *o, sds field, int hfeFlags, int *isHashDel
unsigned int vlen = UINT_MAX;
long long vll = LLONG_MAX;
GetFieldRes res = hashTypeGetValue(db, o, field, &vstr, &vlen, &vll, hfeFlags);
GetFieldRes res = hashTypeGetValue(db, o, field, &vstr, &vlen, &vll, hfeFlags, NULL);
if (isHashDeleted)
*isHashDeleted = (res == GETF_EXPIRED_HASH) ? 1 : 0;
return (res == GETF_OK) ? 1 : 0;
@@ -1134,6 +1139,20 @@ int hashTypeSetExInit(robj *key, robj *o, client *c, redisDb *db, const char *cm
dictEntry *de = dbFind(c->db, key->ptr);
serverAssert(de != NULL);
lpt->key = dictGetKey(de);
} else if (o->encoding == OBJ_ENCODING_LISTPACK_EX) {
listpackEx *lpt = o->ptr;
/* If the hash previously had HFEs but later no longer does, the key ref
* (lpt->key) in the hash might become outdated after a MOVE/COPY/RENAME/RESTORE
* operation. These commands maintain the key ref only if HFEs are present.
* That is, we can only be sure that key ref is valid as long as it is not
* "trash". (TODO: dbFind() can be avoided. Instead need to extend the
* lookupKey*() to return dictEntry). */
if (lpt->meta.trash) {
dictEntry *de = dbFind(c->db, key->ptr);
serverAssert(de != NULL);
lpt->key = dictGetKey(de);
}
} else if (o->encoding == OBJ_ENCODING_HT) {
/* Take care dict has HFE metadata */
if (!isDictWithMetaHFE(ht)) {
@@ -1151,6 +1170,18 @@ int hashTypeSetExInit(robj *key, robj *o, client *c, redisDb *db, const char *cm
m->key = dictGetKey(de); /* reference key in keyspace */
m->hfe = ebCreate(); /* Allocate HFE DS */
m->expireMeta.trash = 1; /* mark as trash (as long it wasn't ebAdd()) */
} else {
dictExpireMetadata *m = (dictExpireMetadata *) dictMetadata(ht);
/* If the hash previously had HFEs but later no longer does, the key ref
* (m->key) in the hash might become outdated after a MOVE/COPY/RENAME/RESTORE
* operation. These commands maintain the key ref only if HFEs are present.
* That is, we can only be sure that key ref is valid as long as it is not
* "trash". */
if (m->expireMeta.trash) {
dictEntry *de = dbFind(db, key->ptr);
serverAssert(de != NULL);
m->key = dictGetKey(de); /* reference key in keyspace */
}
}
}
@@ -1985,6 +2016,21 @@ uint64_t hashTypeRemoveFromExpires(ebuckets *hexpires, robj *o) {
return expireTime;
}
int hashTypeIsFieldsWithExpire(robj *o) {
if (o->encoding == OBJ_ENCODING_LISTPACK) {
return 0;
} else if (o->encoding == OBJ_ENCODING_LISTPACK_EX) {
return EB_EXPIRE_TIME_INVALID != listpackExGetMinExpire(o);
} else { /* o->encoding == OBJ_ENCODING_HT */
dict *d = o->ptr;
/* If dict doesn't holds HFE metadata */
if (!isDictWithMetaHFE(d))
return 0;
dictExpireMetadata *meta = (dictExpireMetadata *) dictMetadata(d);
return ebGetTotalItems(meta->hfe, &hashFieldExpireBucketsType) != 0;
}
}
/* Add hash to global HFE DS and update key for notifications.
*
* key - must be the same key instance that is persisted in db->dict
@@ -2154,7 +2200,7 @@ void hincrbyCommand(client *c) {
if ((o = hashTypeLookupWriteOrCreate(c,c->argv[1])) == NULL) return;
GetFieldRes res = hashTypeGetValue(c->db,o,c->argv[2]->ptr,&vstr,&vlen,&value,
HFE_LAZY_EXPIRE);
HFE_LAZY_EXPIRE, NULL);
if (res == GETF_OK) {
if (vstr) {
if (string2ll((char*)vstr,vlen,&value) == 0) {
@@ -2193,6 +2239,8 @@ void hincrbyfloatCommand(client *c) {
sds new;
unsigned char *vstr;
unsigned int vlen;
int has_expiration = 0;
uint64_t expireat = EB_EXPIRE_TIME_INVALID;
if (getLongDoubleFromObjectOrReply(c,c->argv[3],&incr,NULL) != C_OK) return;
if (isnan(incr) || isinf(incr)) {
@@ -2201,7 +2249,7 @@ void hincrbyfloatCommand(client *c) {
}
if ((o = hashTypeLookupWriteOrCreate(c,c->argv[1])) == NULL) return;
GetFieldRes res = hashTypeGetValue(c->db, o,c->argv[2]->ptr,&vstr,&vlen,&ll,
HFE_LAZY_EXPIRE);
HFE_LAZY_EXPIRE, &expireat);
if (res == GETF_OK) {
if (vstr) {
if (string2ld((char*)vstr,vlen,&value) == 0) {
@@ -2211,6 +2259,8 @@ void hincrbyfloatCommand(client *c) {
} else {
value = (long double)ll;
}
/* Field has expiration time. */
if (expireat != EB_EXPIRE_TIME_INVALID) has_expiration = 1;
} else if ((res == GETF_NOT_FOUND) || (res == GETF_EXPIRED)) {
value = 0;
} else {
@@ -2243,6 +2293,25 @@ void hincrbyfloatCommand(client *c) {
rewriteClientCommandArgument(c,0,shared.hset);
rewriteClientCommandArgument(c,3,newobj);
decrRefCount(newobj);
if (has_expiration) {
/* To make sure that the HSET command is propagated before the HPEXPIREAT,
* we need to prevent the HSET command from being propagated, and then
* propagate both commands manually in the correct order. */
preventCommandPropagation(c);
/* Propagate HSET */
alsoPropagate(c->db->id, c->argv, c->argc, PROPAGATE_AOF|PROPAGATE_REPL);
/* Propagate HPEXPIREAT */
robj *argv[6];
argv[0] = shared.hpexpireat;
argv[1] = c->argv[1];
argv[2] = createStringObjectFromLongLong(expireat);
argv[3] = shared.fields;
argv[4] = shared.integers[1];
argv[5] = c->argv[2];
alsoPropagate(c->db->id, argv, 6, PROPAGATE_AOF|PROPAGATE_REPL);
decrRefCount(argv[2]);
}
}
static GetFieldRes addHashFieldToReply(client *c, robj *o, sds field, int hfeFlags) {
@@ -2255,7 +2324,7 @@ static GetFieldRes addHashFieldToReply(client *c, robj *o, sds field, int hfeFla
unsigned int vlen = UINT_MAX;
long long vll = LLONG_MAX;
GetFieldRes res = hashTypeGetValue(c->db, o, field, &vstr, &vlen, &vll, hfeFlags);
GetFieldRes res = hashTypeGetValue(c->db, o, field, &vstr, &vlen, &vll, hfeFlags, NULL);
if (res == GETF_OK) {
if (vstr) {
addReplyBulkCBuffer(c, vstr, vlen);
@@ -2315,6 +2384,14 @@ void hdelCommand(client *c) {
if ((o = lookupKeyWriteOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,o,OBJ_HASH)) return;
/* Hash field expiration is optimized to avoid frequent update global HFE DS for
* each field deletion. Eventually active-expiration will run and update or remove
* the hash from global HFE DS gracefully. Nevertheless, statistic "subexpiry"
* might reflect wrong number of hashes with HFE to the user if it is the last
* field with expiration. The following logic checks if this is indeed the last
* field with expiration and removes it from global HFE DS. */
int isHFE = hashTypeIsFieldsWithExpire(o);
for (j = 2; j < c->argc; j++) {
if (hashTypeDelete(o,c->argv[j]->ptr,1)) {
deleted++;
@@ -2328,9 +2405,13 @@ void hdelCommand(client *c) {
if (deleted) {
signalModifiedKey(c,c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_HASH,"hdel",c->argv[1],c->db->id);
if (keyremoved)
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",c->argv[1],
c->db->id);
if (keyremoved) {
notifyKeyspaceEvent(NOTIFY_GENERIC, "del", c->argv[1], c->db->id);
} else {
if (isHFE && (hashTypeIsFieldsWithExpire(o) == 0)) /* is it last HFE */
ebRemove(&c->db->hexpires, &hashExpireBucketsType, o);
}
server.dirty += deleted;
}
addReplyLongLong(c,deleted);
@@ -2355,7 +2436,7 @@ void hstrlenCommand(client *c) {
checkType(c,o,OBJ_HASH)) return;
GetFieldRes res = hashTypeGetValue(c->db, o, c->argv[2]->ptr, &vstr, &vlen, &vll,
HFE_LAZY_EXPIRE);
HFE_LAZY_EXPIRE, NULL);
if (res == GETF_NOT_FOUND || res == GETF_EXPIRED || res == GETF_EXPIRED_HASH) {
addReply(c, shared.czero);
+28 -16
View File
@@ -1405,11 +1405,6 @@ int streamRangeHasTombstones(stream *s, streamID *start, streamID *end) {
return 0;
}
if (streamCompareID(&s->first_id,&s->max_deleted_entry_id) > 0) {
/* The latest tombstone is before the first entry. */
return 0;
}
if (start) {
start_id = *start;
} else {
@@ -1446,6 +1441,17 @@ void streamReplyWithCGLag(client *c, stream *s, streamCG *cg) {
/* The lag of a newly-initialized stream is 0. */
lag = 0;
valid = 1;
} else if (!s->length) { /* All entries deleted, now empty. */
lag = 0;
valid = 1;
} else if (streamCompareID(&cg->last_id,&s->first_id) < 0 &&
streamCompareID(&s->max_deleted_entry_id,&s->first_id) < 0)
{
/* When both the consumer group's last_id and the maximum tombstone are behind
* the stream's first entry, the consumer group's lag will always be equal to
* the number of remainin entries in the stream. */
lag = s->length;
valid = 1;
} else if (cg->entries_read != SCG_INVALID_ENTRIES_READ && !streamRangeHasTombstones(s,&cg->last_id,NULL)) {
/* No fragmentation ahead means that the group's logical reads counter
* is valid for performing the lag calculation. */
@@ -1502,6 +1508,10 @@ long long streamEstimateDistanceFromFirstEverEntry(stream *s, streamID *id) {
return s->entries_added;
}
/* There are fragmentations between the `id` and the stream's last-generated-id. */
if (!streamIDEqZero(id) && streamCompareID(id,&s->max_deleted_entry_id) < 0)
return SCG_INVALID_ENTRIES_READ;
int cmp_last = streamCompareID(id,&s->last_id);
if (cmp_last == 0) {
/* Return the exact counter of the last entry in the stream. */
@@ -1684,10 +1694,13 @@ size_t streamReplyWithRange(client *c, stream *s, streamID *start, streamID *end
while(streamIteratorGetID(&si,&id,&numfields)) {
/* Update the group last_id if needed. */
if (group && streamCompareID(&id,&group->last_id) > 0) {
if (group->entries_read != SCG_INVALID_ENTRIES_READ && !streamRangeHasTombstones(s,&id,NULL)) {
/* A valid counter and no future tombstones mean we can
* increment the read counter to keep tracking the group's
* progress. */
if (group->entries_read != SCG_INVALID_ENTRIES_READ &&
streamCompareID(&group->last_id, &s->first_id) >= 0 &&
!streamRangeHasTombstones(s,&group->last_id,NULL))
{
/* A valid counter and no tombstones between the group's last-delivered-id
* and the stream's last-generated-id mean we can increment the read counter
* to keep tracking the group's progress. */
group->entries_read++;
} else if (s->entries_added) {
/* The group's counter may be invalid, so we try to obtain it. */
@@ -2289,14 +2302,13 @@ void xreadCommand(client *c) {
"just return an empty result set.");
goto cleanup;
}
if (o) {
if (o && ((stream *)o->ptr)->length) {
stream *s = o->ptr;
ids[id_idx] = s->last_id;
if (streamDecrID(&ids[id_idx]) != C_OK) {
/* shouldn't happen */
addReplyError(c,"the stream last element ID is 0-0");
goto cleanup;
}
/* We need to get the last valid ID.
* It is impossible to use s->last_id because
* entry with s->last_id may have been removed. */
streamLastValidID(s, &ids[id_idx]);
streamDecrID(&ids[id_idx]);
} else {
ids[id_idx].ms = 0;
ids[id_idx].seq = 0;
+7 -1
View File
@@ -3277,7 +3277,7 @@ void genericZrangebyscoreCommand(zrange_result_handler *handler,
handler->beginResultEmission(handler, -1);
/* For invalid offset, return directly. */
if (offset > 0 && offset >= (long)zsetLength(zobj)) {
if (offset < 0 || (offset > 0 && offset >= (long)zsetLength(zobj))) {
handler->finalizeResultEmission(handler, 0);
return;
}
@@ -3551,6 +3551,12 @@ void genericZrangebylexCommand(zrange_result_handler *handler,
handler->beginResultEmission(handler, -1);
/* For invalid offset, return directly. */
if (offset < 0 || (offset > 0 && offset >= (long)zsetLength(zobj))) {
handler->finalizeResultEmission(handler, 0);
return;
}
if (zobj->encoding == OBJ_ENCODING_LISTPACK) {
unsigned char *zl = zobj->ptr;
unsigned char *eptr, *sptr;
+2
View File
@@ -755,6 +755,8 @@ static void tlsAcceptHandler(aeEventLoop *el, int fd, void *privdata, int mask)
while(max--) {
cfd = anetTcpAccept(server.neterr, fd, cip, sizeof(cip), &cport);
if (cfd == ANET_ERR) {
if (anetAcceptFailureNeedsRetry(errno))
continue;
if (errno != EWOULDBLOCK)
serverLog(LL_WARNING,
"Accepting client connection: %s", server.neterr);
+2
View File
@@ -101,6 +101,8 @@ static void connUnixAcceptHandler(aeEventLoop *el, int fd, void *privdata, int m
while(max--) {
cfd = anetUnixAccept(server.neterr, fd);
if (cfd == ANET_ERR) {
if (anetAcceptFailureNeedsRetry(errno))
continue;
if (errno != EWOULDBLOCK)
serverLog(LL_WARNING,
"Accepting client connection: %s", server.neterr);
+6 -3
View File
@@ -55,8 +55,11 @@
/* Glob-style pattern matching. */
static int stringmatchlen_impl(const char *pattern, int patternLen,
const char *string, int stringLen, int nocase, int *skipLongerMatches)
const char *string, int stringLen, int nocase, int *skipLongerMatches, int nesting)
{
/* Protection against abusive patterns. */
if (nesting > 1000) return 0;
while(patternLen && stringLen) {
switch(pattern[0]) {
case '*':
@@ -68,7 +71,7 @@ static int stringmatchlen_impl(const char *pattern, int patternLen,
return 1; /* match */
while(stringLen) {
if (stringmatchlen_impl(pattern+1, patternLen-1,
string, stringLen, nocase, skipLongerMatches))
string, stringLen, nocase, skipLongerMatches, nesting+1))
return 1; /* match */
if (*skipLongerMatches)
return 0; /* no match */
@@ -190,7 +193,7 @@ static int stringmatchlen_impl(const char *pattern, int patternLen,
int stringmatchlen(const char *pattern, int patternLen,
const char *string, int stringLen, int nocase) {
int skipLongerMatches = 0;
return stringmatchlen_impl(pattern,patternLen,string,stringLen,nocase,&skipLongerMatches);
return stringmatchlen_impl(pattern,patternLen,string,stringLen,nocase,&skipLongerMatches,0);
}
int stringmatch(const char *pattern, const char *string, int nocase) {
+2 -2
View File
@@ -1,2 +1,2 @@
#define REDIS_VERSION "255.255.255"
#define REDIS_VERSION_NUM 0x00ffffff
#define REDIS_VERSION "7.4.6"
#define REDIS_VERSION_NUM 0x00070406
@@ -80,7 +80,14 @@ test "Main db not affected when fail to diskless load" {
fail "Fail to full sync"
}
# Replica keys and keys to slots map still both are right
assert_equal {1} [$replica get $slot0_key]
# Replica keys and keys to slots map still both are right.
# CLUSTERDOWN errors are acceptable here because the cluster may be in a transient state
# due to the timing relationship with cluster-node-timeout.
if {[catch {$replica get $slot0_key} result]} {
assert_match "*CLUSTERDOWN*" $result
} else {
assert_equal {1} $result
}
assert_equal $slot0_key [$replica CLUSTER GETKEYSINSLOT 0 1]
}
@@ -124,6 +124,10 @@ test "Verify health as fail for killed node" {
}
}
test "Verify that other nodes can correctly output the new master's slots" {
assert_not_equal {} [dict get [get_node_info_from_shard [R 4 CLUSTER MYID] 8 "shard"] slots]
}
set primary_id 4
set replica_id 0
+2 -1
View File
@@ -220,9 +220,10 @@ proc is_alive pid {
}
proc stop_instance pid {
catch {exec kill $pid}
# Node might have been stopped in the test
# Send SIGCONT before SIGTERM, otherwise shutdown may be slow with ASAN.
catch {exec kill -SIGCONT $pid}
catch {exec kill $pid}
if {$::valgrind} {
set max_wait 120000
} else {
+9
View File
@@ -312,3 +312,12 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_OK;
}
int RedisModule_OnUnload(RedisModuleCtx *ctx) {
REDISMODULE_NOT_USED(ctx);
if (datatype) {
RedisModule_Free(datatype);
datatype = NULL;
}
return REDISMODULE_OK;
}
+2 -1
View File
@@ -141,13 +141,14 @@ size_t FragFreeEffort(RedisModuleString *key, const void *value) {
}
int FragDefrag(RedisModuleDefragCtx *ctx, RedisModuleString *key, void **value) {
REDISMODULE_NOT_USED(key);
unsigned long i = 0;
int steps = 0;
int dbid = RedisModule_GetDbIdFromDefragCtx(ctx);
RedisModule_Assert(dbid != -1);
RedisModule_Log(NULL, "notice", "Defrag key: %s", RedisModule_StringPtrLen(key, NULL));
/* Attempt to get cursor, validate it's what we're exepcting */
if (RedisModule_DefragCursorGet(ctx, &i) == REDISMODULE_OK) {
if (i > 0) datatype_resumes++;
+2 -1
View File
@@ -95,9 +95,10 @@ proc kill_server config {
# kill server and wait for the process to be totally exited
send_data_packet $::test_server_fd server-killing $pid
catch {exec kill $pid}
# Node might have been stopped in the test
# Send SIGCONT before SIGTERM, otherwise shutdown may be slow with ASAN.
catch {exec kill -SIGCONT $pid}
catch {exec kill $pid}
if {$::valgrind} {
set max_wait 120000
} else {
+26
View File
@@ -116,6 +116,32 @@ start_server {tags {"acl external:skip"}} {
assert_match "*NOPERM*key*" $err
}
test {Validate read and write permissions format - empty permission} {
catch {r ACL SETUSER key-permission-RW %~} err
set err
} {ERR Error in ACL SETUSER modifier '%~': Syntax error}
test {Validate read and write permissions format - empty selector} {
catch {r ACL SETUSER key-permission-RW %} err
set err
} {ERR Error in ACL SETUSER modifier '%': Syntax error}
test {Validate read and write permissions format - empty pattern} {
# Empty pattern results with R/W access to no key
r ACL SETUSER key-permission-RW on nopass %RW~ +@all
$r2 auth key-permission-RW password
catch {$r2 SET x 5} err
set err
} {NOPERM No permissions to access a key}
test {Validate read and write permissions format - no pattern} {
# No pattern results with R/W access to no key (currently we accept this syntax error)
r ACL SETUSER key-permission-RW on nopass %RW +@all
$r2 auth key-permission-RW password
catch {$r2 SET x 5} err
set err
} {NOPERM No permissions to access a key}
test {Test separate read and write permissions on different selectors are not additive} {
r ACL SETUSER key-permission-RW-selector on nopass "(%R~read* +@all)" "(%W~write* +@all)"
$r2 auth key-permission-RW-selector password
+18
View File
@@ -45,6 +45,24 @@ start_server {tags {"auth external:skip"} overrides {requirepass foobar}} {
assert_match {*unauthenticated bulk length*} $e
$rr close
}
test {For unauthenticated clients output buffer is limited} {
set rr [redis [srv "host"] [srv "port"] 1 $::tls]
$rr SET x 5
catch {[$rr read]} e
assert_match {*NOAUTH Authentication required*} $e
# Fill the output buffer in a loop without reading it and make
# sure the client disconnected.
# Considering the socket eat some of the replies, we are testing
# that such client can't consume more than few MB's.
catch {
for {set j 0} {$j < 1000000} {incr j} {
$rr SET x 5
}
} e
assert_match {I/O error reading reply} $e
}
}
start_server {tags {"auth_binary_password external:skip"}} {
@@ -47,4 +47,29 @@ start_cluster 2 2 {tags {external:skip cluster}} {
R 0 config set cluster-announce-bus-port 0
assert_match "*@$base_bus_port *" [R 0 CLUSTER NODES]
}
test "CONFIG SET port updates cluster-announced port" {
set count [expr [llength $::servers] + 1]
# Get the original port and change to new_port
if {$::tls} {
set orig_port [lindex [R 0 config get tls-port] 1]
} else {
set orig_port [lindex [R 0 config get port] 1]
}
assert {$orig_port != ""}
set new_port [find_available_port $orig_port $count]
if {$::tls} {
R 0 config set tls-port $new_port
} else {
R 0 config set port $new_port
}
# Verify that the new port appears in the output of cluster slots
wait_for_condition 50 100 {
[string match "*$new_port*" [R 0 cluster slots]]
} else {
fail "Cluster announced port was not updated in cluster slots"
}
}
}
+37 -11
View File
@@ -1,3 +1,16 @@
#
# Copyright (c) 2009-Present, Redis Ltd.
# All rights reserved.
#
# Copyright (c) 2024-present, Valkey contributors.
# All rights reserved.
#
# Licensed under your choice of the Redis Source Available License 2.0
# (RSALv2) or the Server Side Public License v1 (SSPLv1).
#
# Portions of this file are available under BSD3 terms; see REDISCONTRIBUTIONS for more information.
#
proc get_slot_field {slot_output shard_id node_id attrib_id} {
return [lindex [lindex [lindex $slot_output $shard_id] $node_id] $attrib_id]
}
@@ -116,10 +129,11 @@ test "Verify the nodes configured with prefer hostname only show hostname for ne
# Have everyone forget node 6 and isolate it from the cluster.
isolate_node 6
# Set hostnames for the masters, now that the node is isolated
R 0 config set cluster-announce-hostname "shard-1.com"
R 1 config set cluster-announce-hostname "shard-2.com"
R 2 config set cluster-announce-hostname "shard-3.com"
set primaries 3
for {set j 0} {$j < $primaries} {incr j} {
# Set hostnames for the masters, now that the node is isolated
R $j config set cluster-announce-hostname "shard-$j.com"
}
# Prevent Node 0 and Node 6 from properly meeting,
# they'll hang in the handshake phase. This allows us to
@@ -149,9 +163,17 @@ test "Verify the nodes configured with prefer hostname only show hostname for ne
} else {
fail "Node did not learn about the 2 shards it can talk to"
}
set slot_result [R 6 CLUSTER SLOTS]
assert_equal [lindex [get_slot_field $slot_result 0 2 3] 1] "shard-2.com"
assert_equal [lindex [get_slot_field $slot_result 1 2 3] 1] "shard-3.com"
wait_for_condition 50 100 {
[lindex [get_slot_field [R 6 CLUSTER SLOTS] 0 2 3] 1] eq "shard-1.com"
} else {
fail "hostname for shard-1 didn't reach node 6"
}
wait_for_condition 50 100 {
[lindex [get_slot_field [R 6 CLUSTER SLOTS] 1 2 3] 1] eq "shard-2.com"
} else {
fail "hostname for shard-2 didn't reach node 6"
}
# Also make sure we know about the isolated master, we
# just can't reach it.
@@ -170,10 +192,14 @@ test "Verify the nodes configured with prefer hostname only show hostname for ne
} else {
fail "Node did not learn about the 2 shards it can talk to"
}
set slot_result [R 6 CLUSTER SLOTS]
assert_equal [lindex [get_slot_field $slot_result 0 2 3] 1] "shard-1.com"
assert_equal [lindex [get_slot_field $slot_result 1 2 3] 1] "shard-2.com"
assert_equal [lindex [get_slot_field $slot_result 2 2 3] 1] "shard-3.com"
for {set j 0} {$j < $primaries} {incr j} {
wait_for_condition 50 100 {
[lindex [get_slot_field [R 6 CLUSTER SLOTS] $j 2 3] 1] eq "shard-$j.com"
} else {
fail "hostname information for shard-$j didn't reach node 6"
}
}
}
test "Test restart will keep hostname information" {
+55
View File
@@ -137,6 +137,61 @@ start_server {tags {"hll"}} {
set e
} {*WRONGTYPE*}
test {Corrupted sparse HyperLogLogs doesn't cause overflow and out-of-bounds with XZERO opcode} {
r del hll
# Create a sparse-encoded HyperLogLog header
set header "HYLL"
set payload [binary format c12 {1 0 0 0 0 0 0 0 0 0 0 0}]
set pl [binary format a4a12 $header $payload]
# Create an XZERO opcode with the maximum run length of 16384(2^14)
set runlen [expr 16384 - 1]
set chunk [binary format cc [expr {0b01000000 | ($runlen >> 8)}] [expr {$runlen & 0xff}]]
# Fill the HLL with more than 131072(2^17) XZERO opcodes to make the total
# run length exceed 4GB, will cause an integer overflow.
set repeat [expr 131072 + 1000]
for {set i 0} {$i < $repeat} {incr i} {
append pl $chunk
}
# Create a VAL opcode with a value that will cause out-of-bounds.
append pl [binary format c 0b11111111]
r set hll $pl
# This should not overflow and out-of-bounds.
assert_error {*INVALIDOBJ*} {r pfcount hll hll}
assert_error {*INVALIDOBJ*} {r pfdebug getreg hll}
r ping
}
test {Corrupted sparse HyperLogLogs doesn't cause overflow and out-of-bounds with ZERO opcode} {
r del hll
# Create a sparse-encoded HyperLogLog header
set header "HYLL"
set payload [binary format c12 {1 0 0 0 0 0 0 0 0 0 0 0}]
set pl [binary format a4a12 $header $payload]
# # Create an ZERO opcode with the maximum run length of 64(2^6)
set chunk [binary format c [expr {0b00000000 | 0x3f}]]
# Fill the HLL with more than 33554432(2^17) ZERO opcodes to make the total
# run length exceed 4GB, will cause an integer overflow.
set repeat [expr 33554432 + 1000]
for {set i 0} {$i < $repeat} {incr i} {
append pl $chunk
}
# Create a VAL opcode with a value that will cause out-of-bounds.
append pl [binary format c 0b11111111]
r set hll $pl
# This should not overflow and out-of-bounds.
assert_error {*INVALIDOBJ*} {r pfcount hll hll}
assert_error {*INVALIDOBJ*} {r pfdebug getreg hll}
r ping
}
test {Corrupted dense HyperLogLogs are detected: Wrong length} {
r del hll
r pfadd hll a b c
+39 -2
View File
@@ -1,3 +1,16 @@
#
# Copyright (c) 2009-Present, Redis Ltd.
# All rights reserved.
#
# Copyright (c) 2024-present, Valkey contributors.
# All rights reserved.
#
# Licensed under your choice of the Redis Source Available License 2.0
# (RSALv2) or the Server Side Public License v1 (SSPLv1).
#
# Portions of this file are available under BSD3 terms; see REDISCONTRIBUTIONS for more information.
#
proc cmdstat {cmd} {
return [cmdrstat $cmd r]
}
@@ -386,10 +399,10 @@ start_server {tags {"info" "external:skip"}} {
r config set client-output-buffer-limit "normal 10 0 0"
r set key [string repeat a 100000] ;# to trigger output buffer limit check this needs to be big
catch {r get key}
r config set client-output-buffer-limit $org_outbuf_limit
set info [r info stats]
assert_equal [getInfoProperty $info client_output_buffer_limit_disconnections] {1}
r config set client-output-buffer-limit $org_outbuf_limit
} {OK} {logreqres:skip} ;# same as obuf-limits.tcl, skip logreqres
} {} {logreqres:skip} ;# same as obuf-limits.tcl, skip logreqres
test {clients: pubsub clients} {
set info [r info clients]
@@ -522,3 +535,27 @@ start_server {tags {"info" "external:skip"}} {
assert_equal [dict get $mem_stats db.dict.rehashing.count] {1}
}
}
start_cluster 1 0 {tags {external:skip cluster}} {
test "Verify that LUT overhead is properly updated when dicts are emptied or reused (issue #13973)" {
R 0 set k v ;# Make dbs overhead displayed
set info_mem [r memory stats]
set overhead_main [dict get $info_mem db.0 overhead.hashtable.main]
set overhead_expires [dict get $info_mem db.0 overhead.hashtable.expires]
assert_range $overhead_main 1 5000
assert_range $overhead_expires 1 1000
# In cluster mode, we use KVSTORE_FREE_EMPTY_DICTS to ensure that dicts
# are freed when they are emptied. This test verifies that after a dict
# is cleared, the lut overhead is properly updated, preventing it from
# growing indefinitely.
for {set j 1} {$j <= 500} {incr j} {
R 0 set k v
R 0 del k
}
R 0 set k v ;# Make dbs overhead displayed
set info_mem [r memory stats]
assert_equal [dict get $info_mem db.0 overhead.hashtable.main] $overhead_main
assert_equal [dict get $info_mem db.0 overhead.hashtable.expires] $overhead_expires
}
}
+6
View File
@@ -509,6 +509,12 @@ foreach {type large} [array get largevalue] {
r KEYS "a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*b"
} {}
test {Regression for pattern matching very long nested loops} {
r flushdb
r SET [string repeat "a" 50000] 1
r KEYS [string repeat "*?" 50000]
} {}
test {Coverage: basic SWAPDB test and unhappy path} {
r flushall
r select 0
+14
View File
@@ -174,4 +174,18 @@ start_server {tags {"lazyfree"}} {
assert_equal [s lazyfreed_objects] 2
$rd close
}
test "Unblocks client blocked on lazyfree via REPLICAOF command" {
set rd [redis_deferring_client]
populate 50000 ;# Just to make flushdb async slower
$rd flushdb
wait_for_blocked_client
# Test that slaveof command unblocks clients without assertion failure
r slaveof 127.0.0.1 0
assert_equal [$rd read] {OK}
$rd close
r ping
r slaveof no one
} {OK} {external:skip}
}
+53 -25
View File
@@ -309,31 +309,46 @@ run_solo {defrag} {
set expected_frag 1.7
if {$::accurate} {
# scale the hash to 1m fields in order to have a measurable the latency
set count 0
for {set j 10000} {$j < 1000000} {incr j} {
$rd hset bighash $j [concat "asdfasdfasdf" $j]
}
for {set j 10000} {$j < 1000000} {incr j} {
$rd read ; # Discard replies
incr count
if {$count % 10000 == 0} {
for {set k 0} {$k < 10000} {incr k} {
$rd read ; # Discard replies
}
}
}
# creating that big hash, increased used_memory, so the relative frag goes down
set expected_frag 1.3
}
# add a mass of string keys
set count 0
for {set j 0} {$j < 500000} {incr j} {
$rd setrange $j 150 a
}
for {set j 0} {$j < 500000} {incr j} {
$rd read ; # Discard replies
incr count
if {$count % 10000 == 0} {
for {set k 0} {$k < 10000} {incr k} {
$rd read ; # Discard replies
}
}
}
assert_equal [r dbsize] 500015
# create some fragmentation
set count 0
for {set j 0} {$j < 500000} {incr j 2} {
$rd del $j
}
for {set j 0} {$j < 500000} {incr j 2} {
$rd read ; # Discard replies
incr count
if {$count % 10000 == 0} {
for {set k 0} {$k < 10000} {incr k} {
$rd read ; # Discard replies
}
}
}
assert_equal [r dbsize] 250015
@@ -596,22 +611,27 @@ run_solo {defrag} {
r config set active-defrag-cycle-max 75
r config set active-defrag-ignore-bytes 2mb
r config set maxmemory 0
r config set list-max-ziplist-size 5 ;# list of 500k items will have 100k quicklist nodes
r config set list-max-ziplist-size 1 ;# list of 100k items will have 100k quicklist nodes
# create big keys with 10k items
set rd [redis_deferring_client]
set expected_frag 1.7
# add a mass of list nodes to two lists (allocations are interlaced)
set val [string repeat A 100] ;# 5 items of 100 bytes puts us in the 640 bytes bin, which has 32 regs, so high potential for fragmentation
set elements 500000
set val [string repeat A 500] ;# 1 item of 500 bytes puts us in the 640 bytes bin, which has 32 regs, so high potential for fragmentation
set elements 100000
set count 0
for {set j 0} {$j < $elements} {incr j} {
$rd lpush biglist1 $val
$rd lpush biglist2 $val
}
for {set j 0} {$j < $elements} {incr j} {
$rd read ; # Discard replies
$rd read ; # Discard replies
incr count
if {$count % 10000 == 0} {
for {set k 0} {$k < 10000} {incr k} {
$rd read ; # Discard replies
$rd read ; # Discard replies
}
}
}
# create some fragmentation
@@ -676,9 +696,9 @@ run_solo {defrag} {
assert {$max_latency <= 30}
}
# in extreme cases of stagnation, we see over 20m misses before the tests aborts with "defrag didn't stop",
# in normal cases we only see 100k misses out of 500k elements
assert {$misses < $elements}
# in extreme cases of stagnation, we see over 5m misses before the tests aborts with "defrag didn't stop",
# in normal cases we only see 100k misses out of 100k elements
assert {$misses < $elements * 2}
}
# verify the data isn't corrupted or changed
set newdigest [debug_digest]
@@ -717,11 +737,16 @@ run_solo {defrag} {
# add a mass of keys with 600 bytes values, fill the bin of 640 bytes which has 32 regs per slab.
set rd [redis_deferring_client]
set keys 640000
set count 0
for {set j 0} {$j < $keys} {incr j} {
$rd setrange $j 600 x
}
for {set j 0} {$j < $keys} {incr j} {
$rd read ; # Discard replies
incr count
if {$count % 10000 == 0} {
for {set k 0} {$k < 10000} {incr k} {
$rd read ; # Discard replies
}
}
}
# create some fragmentation of 50%
@@ -730,9 +755,12 @@ run_solo {defrag} {
$rd del $j
incr sent
incr j 1
}
for {set j 0} {$j < $sent} {incr j} {
$rd read ; # Discard replies
if {$sent % 10000 == 0} {
for {set k 0} {$k < 10000} {incr k} {
$rd read ; # Discard replies
}
}
}
# create higher fragmentation in the first slab
+5
View File
@@ -1,6 +1,11 @@
set testmodule [file normalize tests/modules/datatype.so]
start_server {tags {"modules"}} {
test {DataType: test loadex with invalid config} {
catch { r module loadex $testmodule CONFIG invalid_config 1 } e
assert_match {*ERR Error loading the extension*} $e
}
r module load $testmodule
test {DataType: Test module is sane, GET/SET work.} {
+3 -1
View File
@@ -37,7 +37,9 @@ start_server {tags {"obuf-limits external:skip logreqres:skip"}} {
set omem 0
while 1 {
r publish foo bar
# The larger content size ensures that client.buf gets filled more quickly,
# allowing us to correctly observe the gradual increase of `omem`
r publish foo [string repeat bar 50]
set clients [split [r client list] "\r\n"]
set c [split [lindex $clients 1] " "]
if {![regexp {omem=([0-9]+)} $c - omem]} break
+64
View File
@@ -1,3 +1,17 @@
#
# Copyright (c) 2009-Present, Redis Ltd.
# All rights reserved.
#
# Copyright (c) 2024-present, Valkey contributors.
# All rights reserved.
#
# Licensed under your choice of (a) the Redis Source Available License 2.0
# (RSALv2); or (b) the Server Side Public License v1 (SSPLv1); or (c) the
# GNU Affero General Public License v3 (AGPLv3).
#
# Portions of this file are available under BSD3 terms; see REDISCONTRIBUTIONS for more information.
#
start_server {tags {"pause network"}} {
test "Test read commands are not blocked by client pause" {
r client PAUSE 100000 WRITE
@@ -359,6 +373,56 @@ start_server {tags {"pause network"}} {
} {bar2}
}
test "Test the randomkey command will not cause the server to get into an infinite loop during the client pause write" {
r flushall
r multi
r set key value px 3
r client pause 10000 write
r exec
after 5
wait_for_condition 50 100 {
[r randomkey] == "key"
} else {
fail "execute randomkey failed, caused by the infinite loop"
}
r client unpause
assert_equal [r randomkey] {}
}
test "CLIENT UNBLOCK is not allow to unblock client blocked by CLIENT PAUSE" {
set rd1 [redis_deferring_client]
set rd2 [redis_deferring_client]
$rd1 client id
$rd2 client id
set client_id1 [$rd1 read]
set client_id2 [$rd2 read]
r del mylist
r client pause 100000 write
$rd1 blpop mylist 0
$rd2 blpop mylist 0
wait_for_blocked_clients_count 2 50 100
# This used to trigger a panic.
assert_equal 0 [r client unblock $client_id1 timeout]
# This used to return a UNBLOCKED error.
assert_equal 0 [r client unblock $client_id2 error]
# After the unpause, it must be able to unblock the client.
r client unpause
assert_equal 1 [r client unblock $client_id1 timeout]
assert_equal 1 [r client unblock $client_id2 error]
assert_equal {} [$rd1 read]
assert_error "UNBLOCKED*" {$rd2 read}
$rd1 close
$rd2 close
}
# Make sure we unpause at the end
r client unpause
}
+220 -6
View File
@@ -1,3 +1,17 @@
#
# Copyright (c) 2009-Present, Redis Ltd.
# All rights reserved.
#
# Copyright (c) 2024-present, Valkey contributors.
# All rights reserved.
#
# Licensed under your choice of (a) the Redis Source Available License 2.0
# (RSALv2); or (b) the Server Side Public License v1 (SSPLv1); or (c) the
# GNU Affero General Public License v3 (AGPLv3).
#
# Portions of this file are available under BSD3 terms; see REDISCONTRIBUTIONS for more information.
#
foreach is_eval {0 1} {
if {$is_eval == 1} {
@@ -341,12 +355,44 @@ start_server {tags {"scripting"}} {
set e
} {*against a key*}
test {EVAL - JSON string encoding a string larger than 2GB} {
run_script {
local s = string.rep("a", 1024 * 1024 * 1024)
return #cjson.encode(s..s..s)
} 0
} {3221225474} {large-memory} ;# length includes two double quotes at both ends
test {EVAL - Test table unpack with invalid indexes} {
catch {run_script { return {unpack({1,2,3}, -2, 2147483647)} } 0} e
assert_match {*too many results to unpack*} $e
catch {run_script { return {unpack({1,2,3}, 0, 2147483647)} } 0} e
assert_match {*too many results to unpack*} $e
catch {run_script { return {unpack({1,2,3}, -2147483648, -2)} } 0} e
assert_match {*too many results to unpack*} $e
set res [run_script { return {unpack({1,2,3}, -1, -2)} } 0]
assert_match {} $res
set res [run_script { return {unpack({1,2,3}, 1, -1)} } 0]
assert_match {} $res
# unpack with range -1 to 5, verify nil indexes
set res [run_script {
local function unpack_to_list(t, i, j)
local n, v = select('#', unpack(t, i, j)), {unpack(t, i, j)}
for i = 1, n do v[i] = v[i] or '_NIL_' end
v.n = n
return v
end
return unpack_to_list({1,2,3}, -1, 5)
} 0]
assert_match {_NIL_ _NIL_ 1 2 3 _NIL_ _NIL_} $res
# unpack with negative range, verify nil indexes
set res [run_script {
local function unpack_to_list(t, i, j)
local n, v = select('#', unpack(t, i, j)), {unpack(t, i, j)}
for i = 1, n do v[i] = v[i] or '_NIL_' end
v.n = n
return v
end
return unpack_to_list({1,2,3}, -2147483648, -2147483646)
} 0]
assert_match {_NIL_ _NIL_ _NIL_} $res
} {}
test {EVAL - JSON numeric decoding} {
# We must return the table as a string because otherwise
@@ -691,6 +737,12 @@ start_server {tags {"scripting"}} {
set e
} {ERR *Attempt to modify a readonly table*}
test {lua bit.tohex bug} {
set res [run_script {return bit.tohex(65535, -2147483648)} 0]
r ping
set res
} {0000FFFF}
test {Test an example script DECR_IF_GT} {
set decr_if_gt {
local current
@@ -1079,6 +1131,27 @@ start_server {tags {"scripting"}} {
set _ $e
} {*Attempt to modify a readonly table*}
test "Try trick readonly table on basic types metatable" {
# Run the following scripts for basic types. Either getmetatable()
# should return nil or the metatable must be readonly.
set scripts {
{getmetatable(nil).__index = function() return 1 end}
{getmetatable('').__index = function() return 1 end}
{getmetatable(123.222).__index = function() return 1 end}
{getmetatable(true).__index = function() return 1 end}
{getmetatable(function() return 1 end).__index = function() return 1 end}
{getmetatable(coroutine.create(function() return 1 end)).__index = function() return 1 end}
}
foreach code $scripts {
catch {run_script $code 0} e
assert {
[string match "*attempt to index a nil value script*" $e] ||
[string match "*Attempt to modify a readonly table*" $e]
}
}
}
test "Test loadfile are not available" {
catch {
run_script {
@@ -1107,6 +1180,101 @@ start_server {tags {"scripting"}} {
} {*Script attempted to access nonexistent global variable 'print'*}
}
# start a new server to test the large-memory tests
start_server {tags {"scripting external:skip large-memory"}} {
test {EVAL - JSON string encoding a string larger than 2GB} {
run_script {
local s = string.rep("a", 1024 * 1024 * 1024)
return #cjson.encode(s..s..s)
} 0
} {3221225474} ;# length includes two double quotes at both ends
test {EVAL - Test long escape sequences for strings} {
run_script {
-- Generate 1gb '==...==' separator
local s = string.rep('=', 1024 * 1024)
local t = {} for i=1,1024 do t[i] = s end
local sep = table.concat(t)
collectgarbage('collect')
local code = table.concat({'return [',sep,'[x]',sep,']'})
collectgarbage('collect')
-- Load the code and run it. Script will return the string length.
-- Escape sequence: [=....=[ to ]=...=] will be ignored
-- Actual string is a single character: 'x'. Script will return 1
local func = loadstring(code)
return #func()
} 0
} {1}
test {EVAL - Lua can parse string with too many new lines} {
# Create a long string consisting only of newline characters. When Lua
# fails to parse a string, it typically includes a snippet like
# "... near ..." in the error message to indicate the last recognizable
# token. In this test, since the input contains only newlines, there
# should be no identifiable token, so the error message should contain
# only the actual error, without a near clause.
run_script {
local s = string.rep('\n', 1024 * 1024)
local t = {} for i=1,2048 do t[#t+1] = s end
local lines = table.concat(t)
local fn, err = loadstring(lines)
return err
} 0
} {*chunk has too many lines}
}
# Start a new server to test lua-enable-deprecated-api config
foreach enabled {no yes} {
start_server [subst {tags {"scripting external:skip"} overrides {lua-enable-deprecated-api $enabled}}] {
test "Test setfenv availability lua-enable-deprecated-api=$enabled" {
catch {
run_script {
local f = function() return 1 end
setfenv(f, {})
return 0
} 0
} e
if {$enabled} {
assert_equal $e 0
} else {
assert_match {*Script attempted to access nonexistent global variable 'setfenv'*} $e
}
}
test "Test getfenv availability lua-enable-deprecated-api=$enabled" {
catch {
run_script {
local f = function() return 1 end
getfenv(f)
return 0
} 0
} e
if {$enabled} {
assert_equal $e 0
} else {
assert_match {*Script attempted to access nonexistent global variable 'getfenv'*} $e
}
}
test "Test newproxy availability lua-enable-deprecated-api=$enabled" {
catch {
run_script {
getmetatable(newproxy(true)).__gc = function() return 1 end
return 0
} 0
} e
if {$enabled} {
assert_equal $e 0
} else {
assert_match {*Script attempted to access nonexistent global variable 'newproxy'*} $e
}
}
}
}
# Start a new server since the last test in this stanza will kill the
# instance at all.
start_server {tags {"scripting"}} {
@@ -1878,6 +2046,35 @@ start_server {tags {"scripting needs:debug"}} {
}
start_server {tags {"scripting"}} {
test "Test script flush will not leak memory - script:$is_eval" {
r flushall
r script flush
r function flush
# This is a best-effort test to check we don't leak some resources on
# script flush and function flush commands. For lua vm, we create a
# jemalloc thread cache. On each script flush command, thread cache is
# destroyed and we create a new one. In this test, running script flush
# many times to verify there is no increase in the memory usage while
# re-creating some of the resources for lua vm.
set used_memory [s used_memory]
set allocator_allocated [s allocator_allocated]
r multi
for {set j 1} {$j <= 500} {incr j} {
if {$is_eval} {
r SCRIPT FLUSH
} else {
r FUNCTION FLUSH
}
}
r exec
# Verify used memory is not (much) higher.
assert_lessthan [s used_memory] [expr $used_memory*1.5]
assert_lessthan [s allocator_allocated] [expr $allocator_allocated*1.5]
}
test "Verify Lua performs GC correctly after script loading" {
set dummy_script "--[string repeat x 10]\nreturn "
set n 50000
@@ -2404,4 +2601,21 @@ start_server {tags {"scripting"}} {
assert { [r memory usage foo] <= $expected_memory};
}
}
test {EVAL - explicit error() call handling} {
# error("simple string error")
assert_error {ERR user_script:1: simple string error script: *} {
r eval "error('simple string error')" 0
}
# error({"err": "ERR table error"})
assert_error {ERR table error script: *} {
r eval "error({err='ERR table error'})" 0
}
# error({})
assert_error {ERR unknown error script: *} {
r eval "error({})" 0
}
}
}
+11 -1
View File
@@ -73,7 +73,7 @@ start_server {
set result [create_random_dataset 16 lpush]
test "SORT GET #" {
assert_equal [lsort -integer $result] [r sort tosort GET #]
} {} {cluster:skip}
}
foreach command {SORT SORT_RO} {
test "$command GET <const>" {
@@ -393,6 +393,11 @@ start_cluster 1 0 {tags {"external:skip cluster sort"}} {
r sort "{a}mylist" by "{a}by*" get "{a}get*"
} {30 200 100}
test "sort get # in cluster mode" {
assert_equal [r sort "{a}mylist" by "{a}by*" get # ] {3 1 2}
r sort "{a}mylist" by "{a}by*" get "{a}get*" get #
} {30 3 200 1 100 2}
test "sort_ro by in cluster mode" {
catch {r sort_ro "{a}mylist" by by*} e
assert_match {ERR BY option of SORT denied in Cluster mode when *} $e
@@ -404,4 +409,9 @@ start_cluster 1 0 {tags {"external:skip cluster sort"}} {
assert_match {ERR GET option of SORT denied in Cluster mode when *} $e
r sort_ro "{a}mylist" by "{a}by*" get "{a}get*"
} {30 200 100}
test "sort_ro get # in cluster mode" {
assert_equal [r sort_ro "{a}mylist" by "{a}by*" get # ] {3 1 2}
r sort_ro "{a}mylist" by "{a}by*" get "{a}get*" get #
} {30 3 200 1 100 2}
}
+154 -25
View File
@@ -199,12 +199,12 @@ start_server {tags {"external:skip needs:debug"}} {
set hash_sizes {1 15 16 17 31 32 33 40}
foreach h $hash_sizes {
for {set i 1} {$i <= $h} {incr i} {
# random expiration time
# Random expiration time (Take care expired not after "mix$h")
r hset hrand$h f$i v$i
r hpexpire hrand$h [expr {50 + int(rand() * 50)}] FIELDS 1 f$i
r hpexpire hrand$h [expr {70 + int(rand() * 30)}] FIELDS 1 f$i
assert_equal 1 [r HEXISTS hrand$h f$i]
# same expiration time
# Same expiration time (Take care expired not after "mix$h")
r hset same$h f$i v$i
r hpexpire same$h 100 FIELDS 1 f$i
assert_equal 1 [r HEXISTS same$h f$i]
@@ -286,10 +286,9 @@ start_server {tags {"external:skip needs:debug"}} {
test "HEXPIRETIME - returns TTL in Unix timestamp ($type)" {
r del myhash
r HSET myhash field1 value1
r HPEXPIRE myhash 1000 NX FIELDS 1 field1
set lo [expr {[clock seconds] + 1}]
set hi [expr {[clock seconds] + 2}]
r HPEXPIRE myhash 1000 NX FIELDS 1 field1
assert_range [r HEXPIRETIME myhash FIELDS 1 field1] $lo $hi
assert_range [r HPEXPIRETIME myhash FIELDS 1 field1] [expr $lo*1000] [expr $hi*1000]
}
@@ -592,6 +591,20 @@ start_server {tags {"external:skip needs:debug"}} {
wait_for_condition 30 10 { [r exists myhash2] == 0 } else { fail "`myhash2` should be expired" }
}
test "Test RENAME hash that had HFEs but not during the rename ($type)" {
r del h1
r hset h1 f1 v1 f2 v2
r hpexpire h1 1 FIELDS 1 f1
after 20
r rename h1 h1_renamed
assert_equal [r exists h1] 0
assert_equal [r exists h1_renamed] 1
assert_equal [r hgetall h1_renamed] {f2 v2}
r hpexpire h1_renamed 1 FIELDS 1 f2
# Only active expire will delete the key
wait_for_condition 30 10 { [r exists h1_renamed] == 0 } else { fail "`h1_renamed` should be expired" }
}
test "MOVE to another DB hash with fields to be expired ($type)" {
r select 9
r flushall
@@ -635,6 +648,20 @@ start_server {tags {"external:skip needs:debug"}} {
} {} {singledb:skip}
test "Test COPY hash that had HFEs but not during the copy ($type)" {
r del h1
r hset h1 f1 v1 f2 v2
r hpexpire h1 1 FIELDS 1 f1
after 20
r COPY h1 h1_copy
assert_equal [r exists h1] 1
assert_equal [r exists h1_copy] 1
assert_equal [r hgetall h1_copy] {f2 v2}
r hpexpire h1_copy 1 FIELDS 1 f2
# Only active expire will delete the key
wait_for_condition 30 10 { [r exists h1_copy] == 0 } else { fail "`h1_copy` should be expired" }
}
test "Test SWAPDB hash-fields to be expired ($type)" {
r select 9
r flushall
@@ -656,6 +683,29 @@ start_server {tags {"external:skip needs:debug"}} {
wait_for_condition 20 10 { [r exists myhash] == 0 } else { fail "'myhash' should be expired" }
} {} {singledb:skip}
test "Test SWAPDB hash that had HFEs but not during the swap ($type)" {
r select 9
r flushall
r hset myhash f1 v1 f2 v2
r hpexpire myhash 1 NX FIELDS 1 f1
after 10
r swapdb 9 10
# Verify the key and its field doesn't exist in the source DB
assert_equal [r exists myhash] 0
assert_equal [r dbsize] 0
# Verify the key and its field exists in the target DB
r select 10
assert_equal [r hgetall myhash] {f2 v2}
assert_equal [r dbsize] 1
r hpexpire myhash 1 NX FIELDS 1 f2
# Eventually the field will be expired and the key will be deleted
wait_for_condition 20 10 { [r exists myhash] == 0 } else { fail "'myhash' should be expired" }
} {} {singledb:skip}
test "HMGET - returns empty entries if fields or hash expired ($type)" {
r debug set-active-expire 0
r del h1 h2
@@ -724,6 +774,20 @@ start_server {tags {"external:skip needs:debug"}} {
assert_equal [r hexpiretime myhash FIELDS 3 a b c] {2524600800 2524600801 -1}
}
test {RESTORE hash that had in the past HFEs but not during the dump} {
r config set sanitize-dump-payload yes
r del myhash
r hmset myhash a 1 b 2 c 3
r hpexpire myhash 1 fields 1 a
after 10
set encoded [r dump myhash]
r del myhash
r restore myhash 0 $encoded
assert_equal [lsort [r hgetall myhash]] "2 3 b c"
r hpexpire myhash 1 fields 2 b c
wait_for_condition 30 10 { [r exists myhash] == 0 } else { fail "`myhash` should be expired" }
}
test {DUMP / RESTORE are able to serialize / unserialize a hash with TTL 0 for all fields} {
r config set sanitize-dump-payload yes
r del myhash
@@ -786,6 +850,36 @@ start_server {tags {"external:skip needs:debug"}} {
}
}
test "Statistics - Hashes with HFEs ($type)" {
r config resetstat
r flushall
# hash1: 5 fields, 3 with TTL. subexpiry incr +1
r hset myhash f1 v1 f2 v2 f3 v3 f4 v4 f5 v5
r hpexpire myhash 150 FIELDS 3 f1 f2 f3
assert_match [get_stat_subexpiry r] 1
# hash2: 5 fields, 3 with TTL. subexpiry incr +1
r hset myhash2 f1 v1 f2 v2 f3 v3 f4 v4 f5 v5
assert_match [get_stat_subexpiry r] 1
r hpexpire myhash2 100 FIELDS 3 f1 f2 f3
assert_match [get_stat_subexpiry r] 2
# hash3: 2 fields, 1 with TTL. HDEL field with TTL. subexpiry decr -1
r hset myhash3 f1 v1 f2 v2
r hpexpire myhash3 100 FIELDS 1 f2
assert_match [get_stat_subexpiry r] 3
r hdel myhash3 f2
assert_match [get_stat_subexpiry r] 2
# Expired fields of hash1 and hash2. subexpiry decr -2
wait_for_condition 50 50 {
[get_stat_subexpiry r] == 0
} else {
fail "Hash field expiry statistics failed"
}
}
r config set hash-max-listpack-entries 512
}
@@ -899,25 +993,6 @@ start_server {tags {"external:skip needs:debug"}} {
r config set hash-max-listpack-value 64
}
test "Statistics - Hashes with HFEs" {
r config resetstat
r del myhash
r hset myhash f1 v1 f2 v2 f3 v3 f4 v4 f5 v5
r hpexpire myhash 100 FIELDS 3 f1 f2 f3
assert_match [get_stat_subexpiry r] 1
r hset myhash2 f1 v1 f2 v2 f3 v3 f4 v4 f5 v5
assert_match [get_stat_subexpiry r] 1
r hpexpire myhash2 100 FIELDS 3 f1 f2 f3
assert_match [get_stat_subexpiry r] 2
wait_for_condition 50 50 {
[get_stat_subexpiry r] == 0
} else {
fail "Hash field expiry statistics failed"
}
}
}
start_server {tags {"external:skip needs:debug"}} {
@@ -932,6 +1007,7 @@ start_server {tags {"external:skip needs:debug"}} {
start_server {overrides {appendonly {yes} appendfsync always} tags {external:skip}} {
set aof [get_last_incr_aof_path r]
r debug set-active-expire 0 ;# Prevent fields from being expired during data preparation
# Time is in the past so it should propagate HDELs to replica
# and delete the fields
@@ -958,6 +1034,7 @@ start_server {tags {"external:skip needs:debug"}} {
r hpexpireat h2 [expr [clock seconds]*1000+100000] LT FIELDS 1 f3
r hexpireat h2 [expr [clock seconds]+10] NX FIELDS 1 f4
r debug set-active-expire 1
wait_for_condition 50 100 {
[r hlen h2] eq 2
} else {
@@ -986,7 +1063,7 @@ start_server {tags {"external:skip needs:debug"}} {
{hdel h2 f2}
}
}
}
} {} {needs:debug}
test "Lazy Expire - fields are lazy deleted and propagated to replicas ($type)" {
start_server {overrides {appendonly {yes} appendfsync always} tags {external:skip}} {
@@ -1189,5 +1266,57 @@ start_server {tags {"external:skip needs:debug"}} {
assert_equal [dumpAllHashes $primary] [dumpAllHashes $replica]
}
}
test "HINCRBYFLOAT command won't remove field expiration on replica ($type)" {
r flushall
set repl [attach_to_replication_stream]
r hset h1 f1 1
r hset h1 f2 1
r hexpire h1 100 FIELDS 1 f1
r hincrbyfloat h1 f1 1.1
r hincrbyfloat h1 f2 1.1
# HINCRBYFLOAT will be replicated as HSET if no expiration time is set.
# Otherwise it will be replicated as HSET+HPEXPIREAT multi command.
assert_replication_stream $repl {
{select *}
{hset h1 f1 1}
{hset h1 f2 1}
{hpexpireat h1 * FIELDS 1 f1}
{multi}
{hset h1 f1 *}
{hpexpireat h1 * FIELDS 1 f1}
{exec}
{hset h1 f2 *}
}
close_replication_stream $repl
start_server {tags {external:skip}} {
r -1 flushall
r slaveof [srv -1 host] [srv -1 port]
wait_for_sync r
r -1 hset h1 f1 1
r -1 hset h1 f2 1
r -1 hexpire h1 100 FIELDS 1 f1
wait_for_ofs_sync [srv -1 client] [srv 0 client]
assert_range [r httl h1 FIELDS 1 f1] 90 100
assert_equal {-1} [r httl h1 FIELDS 1 f2]
r -1 hincrbyfloat h1 f1 1.1
r -1 hincrbyfloat h1 f2 1.1
# Expiration time should not be removed on replica and the value
# should be equal to the master.
wait_for_ofs_sync [srv -1 client] [srv 0 client]
assert_range [r httl h1 FIELDS 1 f1] 90 100
assert_equal [r -1 hget h1 f1] [r hget h1 f1]
# The field f2 should not have any expiration time on replica either.
assert_equal {-1} [r httl h1 FIELDS 1 f2]
assert_equal [r -1 hget h1 f2] [r hget h1 f2]
}
} {} {needs:repl external:skip}
}
}
+36
View File
@@ -1,3 +1,17 @@
#
# Copyright (c) 2009-Present, Redis Ltd.
# All rights reserved.
#
# Copyright (c) 2024-present, Valkey contributors.
# All rights reserved.
#
# Licensed under your choice of (a) the Redis Source Available License 2.0
# (RSALv2); or (b) the Server Side Public License v1 (SSPLv1); or (c) the
# GNU Affero General Public License v3 (AGPLv3).
#
# Portions of this file are available under BSD3 terms; see REDISCONTRIBUTIONS for more information.
#
# check functionality compression of plain and packed nodes
start_server [list overrides [list save ""] ] {
r config set list-compress-depth 2
@@ -2423,4 +2437,26 @@ foreach {pop} {BLPOP BLMPOP_RIGHT} {
close_replication_stream $repl
} {} {needs:repl}
test "Blocking timeout following PAUSE should honor the timeout" {
# cleanup first
r del mylist
# create a test client
set rd [redis_deferring_client]
# first PAUSE all writes for a very long time
r client pause 10000000000000 write
# block a client on the list
$rd BLPOP mylist 1
wait_for_blocked_clients_count 1
# now unpause the writes
r client unpause
# client should time-out
wait_for_blocked_clients_count 0
$rd close
}
} ;# stop servers
+81
View File
@@ -1239,6 +1239,87 @@ start_server {
assert_equal [dict get $group lag] 2
}
test {Consumer Group Lag with XDELs and tombstone after the last_id of consume group} {
r DEL x
r XGROUP CREATE x g1 $ MKSTREAM
r XADD x 1-0 data a
r XREADGROUP GROUP g1 alice STREAMS x > ;# Read one entry
r XADD x 2-0 data c
r XADD x 3-0 data d
r XDEL x 2-0
# Now the latest tombstone(2-0) is before the first entry(3-0), but there is still
# a tombstone(2-0) after the last_id(1-0) of the consume group.
set reply [r XINFO STREAM x FULL]
set group [lindex [dict get $reply groups] 0]
assert_equal [dict get $group entries-read] 1
assert_equal [dict get $group lag] {}
r XDEL x 1-0
# Although there is a tombstone(2-0) after the consumer group's last_id(1-0), all
# entries before the maximal tombstone have been deleted. This means that both the
# last_id and the largest tombstone are behind the first entry. Therefore, tombstones
# no longer affect the lag, which now reflects the remaining entries in the stream.
set reply [r XINFO STREAM x FULL]
set group [lindex [dict get $reply groups] 0]
assert_equal [dict get $group entries-read] 1
assert_equal [dict get $group lag] 1
# Now there is a tombstone(2-0) after the last_id of the consume group, so after consuming
# entry(3-0), the group's counter will be invalid.
r XREADGROUP GROUP g1 alice STREAMS x >
set reply [r XINFO STREAM x FULL]
set group [lindex [dict get $reply groups] 0]
assert_equal [dict get $group entries-read] 3
assert_equal [dict get $group lag] 0
}
test {Consumer group lag with XTRIM} {
r DEL x
r XGROUP CREATE x mygroup $ MKSTREAM
r XADD x 1-0 data a
r XADD x 2-0 data b
r XADD x 3-0 data c
r XADD x 4-0 data d
r XADD x 5-0 data e
r XREADGROUP GROUP mygroup alice COUNT 1 STREAMS x >
set reply [r XINFO STREAM x FULL]
set group [lindex [dict get $reply groups] 0]
assert_equal [dict get $group entries-read] 1
assert_equal [dict get $group lag] 4
# Although XTRIM doesn't update the `max-deleted-entry-id`, it always updates the
# position of the first entry. When trimming causes the first entry to be behind
# the consumer group's last_id, the consumer group's lag will always be equal to
# the number of remainin entries in the stream.
r XTRIM x MAXLEN 1
set reply [r XINFO STREAM x FULL]
set group [lindex [dict get $reply groups] 0]
assert_equal [dict get $reply max-deleted-entry-id] "0-0"
assert_equal [dict get $group entries-read] 1
assert_equal [dict get $group lag] 1
# When all the entries are read, the lag is always 0.
r XREADGROUP GROUP mygroup alice STREAMS x >
set reply [r XINFO STREAM x FULL]
set group [lindex [dict get $reply groups] 0]
assert_equal [dict get $group entries-read] 5
assert_equal [dict get $group lag] 0
r XADD x 6-0 data f
set reply [r XINFO STREAM x FULL]
set group [lindex [dict get $reply groups] 0]
assert_equal [dict get $group entries-read] 5
assert_equal [dict get $group lag] 1
# When all the entries were deleted, the lag is always 0.
r XTRIM x MAXLEN 0
set reply [r XINFO STREAM x FULL]
set group [lindex [dict get $reply groups] 0]
assert_equal [dict get $group lag] 0
}
test {Loading from legacy (Redis <= v6.2.x, rdb_ver < 10) persistence} {
# The payload was DUMPed from a v5 instance after:
# XADD x 1-0 data a
+27
View File
@@ -435,6 +435,17 @@ start_server {
# verify nil is still received when reading last entry
assert_equal [r XREAD STREAMS lestream +] {}
# case when stream created empty
# make sure the stream is not initialized
r DEL lestream
# create empty stream with XGROUP CREATE
r XGROUP CREATE lestream legroup $ MKSTREAM
# verify nil is received when reading last entry
assert_equal [r XREAD STREAMS lestream +] {}
}
test {XREAD last element blocking from empty stream} {
@@ -510,6 +521,22 @@ start_server {
assert_equal $res {{lestream {{3-0 {k3 v3}}}}}
}
test "XREAD: read last element after XDEL (issue #13628)" {
# Should return actual last element after XDEL of current last element
# Add 2 entries to a stream and delete last one
r DEL stream
r XADD stream 1-0 f 1
r XADD stream 2-0 f 2
r XDEL stream 2-0
# Read last entry
set res [r XREAD STREAMS stream +]
# Verify the last entry was read
assert_equal $res {{stream {{1-0 {f 1}}}}}
}
test "XREAD: XADD + DEL should not awake client" {
set rd [redis_deferring_client]
r del s1
+9 -1
View File
@@ -583,20 +583,24 @@ start_server {tags {"zset"}} {
assert_equal {d e f} [r zrangebyscore zset 0 10 LIMIT 2 3]
assert_equal {d e f} [r zrangebyscore zset 0 10 LIMIT 2 10]
assert_equal {} [r zrangebyscore zset 0 10 LIMIT 20 10]
assert_equal {} [r zrangebyscore zset 0 10 LIMIT -1 2]
assert_equal {f e} [r zrevrangebyscore zset 10 0 LIMIT 0 2]
assert_equal {d c b} [r zrevrangebyscore zset 10 0 LIMIT 2 3]
assert_equal {d c b} [r zrevrangebyscore zset 10 0 LIMIT 2 10]
assert_equal {} [r zrevrangebyscore zset 10 0 LIMIT 20 10]
assert_equal {} [r zrevrangebyscore zset 10 0 LIMIT -1 2]
# zrangebyscore uses different logic when offset > ZSKIPLIST_MAX_SEARCH
create_long_zset zset 30
assert_equal {i12 i13 i14} [r zrangebyscore zset 0 20 LIMIT 12 3]
assert_equal {i14 i15} [r zrangebyscore zset 0 20 LIMIT 14 2]
assert_equal {i19 i20 i21} [r zrangebyscore zset 0 30 LIMIT 19 3]
assert_equal {i29} [r zrangebyscore zset 10 30 LIMIT 19 2]
assert_equal {i29} [r zrangebyscore zset 10 30 LIMIT 19 2]
assert_equal {} [r zrangebyscore zset 0 20 LIMIT -1 3]
assert_equal {i17 i16 i15} [r zrevrangebyscore zset 30 10 LIMIT 12 3]
assert_equal {i6 i5} [r zrevrangebyscore zset 20 0 LIMIT 14 2]
assert_equal {i2 i1 i0} [r zrevrangebyscore zset 20 0 LIMIT 18 5]
assert_equal {i0} [r zrevrangebyscore zset 20 0 LIMIT 20 5]
assert_equal {} [r zrevrangebyscore zset 30 10 LIMIT -1 3]
}
test "ZRANGEBYSCORE with LIMIT and WITHSCORES - $encoding" {
@@ -680,9 +684,11 @@ start_server {tags {"zset"}} {
assert_equal {bar} [r zrangebylex zset \[bar \[down LIMIT 0 1]
assert_equal {cool} [r zrangebylex zset \[bar \[down LIMIT 1 1]
assert_equal {bar cool down} [r zrangebylex zset \[bar \[down LIMIT 0 100]
assert_equal {} [r zrangebylex zset - \[cool LIMIT -1 2]
assert_equal {omega hill great foo elephant} [r zrevrangebylex zset + \[d LIMIT 0 5]
assert_equal {omega hill great foo} [r zrevrangebylex zset + \[d LIMIT 0 4]
assert_equal {great foo elephant} [r zrevrangebylex zset + \[d LIMIT 2 3]
assert_equal {} [r zrevrangebylex zset + \[d LIMIT -1 5]
# zrangebylex uses different logic when offset > ZSKIPLIST_MAX_SEARCH
create_long_lex_zset
assert_equal {max null} [r zrangebylex zset - \[tree LIMIT 12 2]
@@ -692,12 +698,14 @@ start_server {tags {"zset"}} {
assert_equal {max} [r zrangebylex zset \[max \[null LIMIT 0 1]
assert_equal {null} [r zrangebylex zset \[max \[null LIMIT 1 1]
assert_equal {max null omega point} [r zrangebylex zset \[max \[point LIMIT 0 100]
assert_equal {} [r zrangebylex zset - \[tree LIMIT -1 2]
assert_equal {tree sea result query point} [r zrevrangebylex zset + \[o LIMIT 0 5]
assert_equal {tree sea result query} [r zrevrangebylex zset + \[o LIMIT 0 4]
assert_equal {omega null max lip} [r zrevrangebylex zset + \[l LIMIT 5 4]
assert_equal {elephant down} [r zrevrangebylex zset + \[a LIMIT 15 2]
assert_equal {bar alpha} [r zrevrangebylex zset + - LIMIT 18 6]
assert_equal {hill great foo} [r zrevrangebylex zset + \[c LIMIT 12 3]
assert_equal {} [r zrevrangebylex zset + \[o LIMIT -1 5]
}
test "ZRANGEBYLEX with invalid lex range specifiers - $encoding" {
+23
View File
@@ -140,6 +140,29 @@ tags {"wait aof network external:skip"} {
assert_error {ERR WAITAOF cannot be used when numlocal is set but appendonly is disabled.} {$master waitaof 1 0 0}
}
test {WAITAOF local client unblock with timeout and error} {
r config set appendonly yes
r config set appendfsync no
set rd [redis_deferring_client]
$rd client id
set client_id [$rd read]
# Test unblock with timeout
$rd incr foo
$rd read
$rd waitaof 1 0 0
wait_for_blocked_client
assert_equal 1 [r client unblock $client_id timeout]
# Test unblock with error
$rd incr foo
$rd read
$rd waitaof 1 0 0
wait_for_blocked_client
assert_equal 1 [r client unblock $client_id error]
$rd close
}
test {WAITAOF local if AOFRW was postponed} {
r config set appendfsync everysec