Compare commits

..
308 Commits
Author SHA1 Message Date
antirez 882ca6962f Redis 5.0.0. 2018-10-17 17:31:39 +02:00
antirez 1eb0994ecc Streams: use bulk replies instead of status replies.
They play better with Lua scripting, otherwise Lua will see status
replies as "ok" = "string" which is very odd, and actually as @oranagra
reasoned in issue #5456 in the rest of the Redis code base there was no
such concern as saving a few bytes when the protocol is emitted.
2018-10-17 17:20:05 +02:00
antirez bcc0916d08 Fix conditional in XGROUP. 2018-10-17 13:00:55 +02:00
antirez 1b2f23f3c4 Update help.h for redis-cli. 2018-10-17 12:57:15 +02:00
antirez de0ae56c84 Tests for XGROUP CREATE MKSTREAM. 2018-10-17 12:12:04 +02:00
antirez 56c3dfa195 Fix XGROUP CREATE MKSTREAM handling of . 2018-10-17 12:11:48 +02:00
antirez 2687f2283c Process MKSTREAM option of XGROUP CREATE at a later time.
This avoids issues with having to replicate a command that produced
errors.
2018-10-17 12:11:48 +02:00
zhaozhao.zzandantirez cfbaf8f1f3 Scripting & Streams: some commands need right flags
xadd with id * generates random stream id

xadd & xtrim with approximate maxlen count may
trim stream randomly

xinfo may get random radix-tree-keys/nodes

xpending may get random idletime

xclaim: master and slave may have different
idletime in stream
2018-10-17 12:11:48 +02:00
antirez 4e4099b95d XGROUP CREATE: MKSTREAM option for automatic stream creation. 2018-10-17 11:27:09 +02:00
zhaozhao.zzandantirez 6dd4d864a9 Streams: Tests modified XSTREAM -> XSETID 2018-10-17 11:02:34 +02:00
zhaozhao.zzandantirez 3aff0e8cb5 Streams: rewrite empty streams with certain lastid 2018-10-17 11:02:34 +02:00
antirez 880b563e2d Tests modified to use XADD MAXLEN 0 + XSETID.
See #5426.
2018-10-17 11:02:34 +02:00
antirez 83c8783598 Streams: rewrite empty streams with XADD MAXLEN 0. Use XSETID.
Related to #5426.
2018-10-17 11:02:34 +02:00
antirez fd22e3acae XSETID: accept IDs based on last entry.
Related to #5426.
2018-10-17 11:02:34 +02:00
antirez dfab3cba2a Streams: XSTREAM SETID -> XSETID.
Keep vanilla stream commands at toplevel, see #5426.
2018-10-17 11:02:34 +02:00
zhaozhao.zzandantirez a3fb28edce Streams: rewrite id in XSTREAM CREATE * 2018-10-17 11:02:34 +02:00
zhaozhao.zzandantirez f4b4db1387 Streams: add tests for aof rewrite 2018-10-17 11:02:34 +02:00
zhaozhao.zzandantirez d22f1ef032 Stream & AOF: rewrite stream in correct way 2018-10-17 11:02:34 +02:00
zhaozhao.zzandantirez 6455274d1c Streams: add tests for XSTREAM command 2018-10-17 11:02:34 +02:00
zhaozhao.zzandantirez 0edbe953ea Streams: add a new command XTREAM
XSTREAM CREATE <key> <id or *> -- Create a new empty stream.
XSTREAM SETID <key> <id or $>  -- Set the current stream ID.
2018-10-17 11:02:34 +02:00
Hamid Alaeiandantirez 9714bba266 fix timer context selected database 2018-10-15 13:05:42 +02:00
antirez eb53f15a3e Make comment about nack->consumer test for minidle more obvious.
Related to #5437.
2018-10-15 13:05:35 +02:00
antirez a77f836e0d Streams: use propagate_last_id itself as streamPropagateGroupID trigger.
Avoid storing the dirty value. See #5437.
2018-10-15 13:05:35 +02:00
antirez 0f0610eb01 Streams: better naming: lastid_updated -> propagate_last_id.
See #5437 but also I updated a previous usage of the same var name.
2018-10-15 13:05:35 +02:00
zhaozhao.zzandantirez a745e4234d Streams: panic if streamID invalid after check, should not be possible. 2018-10-15 13:05:35 +02:00
zhaozhao.zzandantirez 9974be13a4 Streams: propagate lastid in XCLAIM when it has effect 2018-10-15 13:05:35 +02:00
zhaozhao.zzandantirez 69a628d0f0 Streams: XCLAIM ignore minidle if NACK is created by FORCE
Because the NACK->consumer is NULL, if idletime < minidle
the NACK does not belong to any consumer, then redis will crash
in XPENDING.
2018-10-15 13:05:35 +02:00
zhaozhao.zzandantirez a04b43c7ea Streams: bugfix XCLAIM should propagate group name not consumer name 2018-10-15 13:05:35 +02:00
Sergey Chupovandantirez 8977a90c50 fixed typos in readme 2018-10-15 13:05:35 +02:00
antirez 3a745674cf redis.conf typo fixed: ingore -> ignore. 2018-10-15 13:05:35 +02:00
antirez 22770d762c Rax: radix tree updated to latest version from antirez/rax. 2018-10-15 13:05:17 +02:00
antirez fbac534fd3 Test: avoid time related false positive in RESTORE test. 2018-10-15 13:05:17 +02:00
antirez 4987233795 LOLWUT: capitalize Nees. 2018-10-15 13:05:17 +02:00
antirez 80c471f58e Test: cgroup propagation test also for NOACK variant.
Related to #5433.
2018-10-15 13:05:17 +02:00
antirez 8defa5da3c Test: consumer group last ID slave propagation test.
This is a regression for #5433.
2018-10-15 13:05:17 +02:00
zhaozhao.zzandantirez e1e3eacae3 Avoid recreate write handler for protected client. 2018-10-15 13:05:17 +02:00
antirez b501fd5d3e Fix propagation of consumer groups last ID.
Issue #5433.
2018-10-15 13:05:17 +02:00
antirez bedc3dee24 Redis 5.0 RC6. 2018-10-10 11:11:06 +02:00
antirez 9a6fa7d082 changelog.tcl: get optional argument for number of commits. 2018-10-10 11:03:25 +02:00
antirez 101e419ffc Free protected clients asynchronously.
Related to #4840.

Note that when we re-enter the event loop with aeProcessEvents() we
don't process timers, nor before/after sleep callbacks, so we should
never end calling freeClientsInAsyncFreeQueue() when re-entering the
loop.
2018-10-09 18:30:59 +02:00
antirez 726debb835 Actually use the protectClient() API where needed.
Related to #4804.
2018-10-09 18:30:49 +02:00
antirez 0b87f78a5f Introduce protectClient() + some refactoring.
The idea is to have an API for the cases like -BUSY state and DEBUG
RELOAD where we have to manually deinstall the read handler.
See #4804.
2018-10-09 18:30:49 +02:00
zhaozhao.zzandantirez 6aa8ac70a4 debug: avoid free client unexpectedly when reload & loadaof 2018-10-09 18:30:49 +02:00
antirez 48040b0266 aof.c: improve indentation and change warning message.
Related to #5201.

I removed the !!! Warning part since compared to the other errors, a
missing EXEC is in theory a normal happening in the AOF file, at least
in theory: may happen in a differnet number of situations, and it's
probably better to don't give the user the feeling that something really
bad happened.
2018-10-09 18:30:49 +02:00
zhaozhao.zzandantirez 7cc2056965 AOF: discard if we lost EXEC when loading aof 2018-10-09 18:30:49 +02:00
antirez 2007d30c9d Refactoring of XADD / XTRIM MAXLEN rewriting.
See #5141.
2018-10-09 18:30:49 +02:00
zhaozhao.zzandantirez 6a2981101b Streams: add test cases for XADD/XTRIM maxlen 2018-10-09 18:30:34 +02:00
zhaozhao.zzandantirez 041161b7ce Streams: propagate specified MAXLEN instead of approximated
Slaves and rebooting redis may have different radix tree struct,
by different stream* config options. So propagating approximated
MAXLEN to AOF/slaves may lead to date inconsistency.
2018-10-09 18:30:34 +02:00
zhaozhao.zzandantirez f04d799bb9 Streams: reset approx_maxlen in every maxlen loop 2018-10-09 18:30:34 +02:00
zhaozhao.zzandantirez affd93654f Streams: XTRIM will return an error if MAXLEN with a count < 0 2018-10-09 18:30:34 +02:00
zhaozhao.zzandantirez 4c405ad07f Streams: propagate original MAXLEN argument in XADD context
If we rewrite the MAXLEN argument as zero when no trimming
was performed, date between master and slave and aof will
be inconsistent, because `xtrim maxlen 0` means delete all
entries in stream.
2018-10-09 18:30:34 +02:00
antirez 5c6d4b4ad4 Fix typo in replicationCron() comment. 2018-10-09 18:30:22 +02:00
antirez a67a8dbf2f Fix typo in design comment of bio.c. 2018-10-09 18:30:18 +02:00
antirez c4ab5a05a7 xclaimCommand(): fix comment typos. 2018-10-09 18:30:13 +02:00
antirez dc0b628a8e streamAppendItem(): Update the radix tree pointer only if changed. 2018-10-03 11:19:32 +02:00
antirez 4566fbc79d Listpack: optionally force reallocation on inserts.
This is useful in order to spot bugs where we fail
at updating the pointer returned by the insertion
function. Normally often the same pointer is returned,
making it harder than needed to spot bugs.

Related to #5210.
2018-10-03 11:19:32 +02:00
antirez 5eca170c5b Fix printf type mismatch in genRedisInfoString(). 2018-10-03 11:19:32 +02:00
antirez 260b53a284 streamIteratorRemoveEntry(): set back lp only if pointer changed.
Most of the times the pointer will remain the same since integers of
similar size don't take more space in listpacks.

Related to #5210.
2018-10-03 11:19:32 +02:00
zhaozhao.zzandantirez 5d12f9d98b Streams: update listpack with new pointer in XDEL 2018-10-03 11:19:32 +02:00
zhaozhao.zzandantirez 6b7ad8381a bugfix: replace lastcmd with cmd when rewrite BRPOPLPUSH as RPOPLPUSH
There are two problems if we use lastcmd:

1. BRPOPLPUSH cannot be rewrited as RPOPLPUSH in multi/exec
    In mulit/exec context, the lastcmd is exec.
2. Redis will crash when execute RPOPLPUSH loading from AOF
    In fakeClient, the lastcmd is NULL.
2018-10-03 11:19:32 +02:00
Oran Agraandantirez 3454a04399 script cache memory in INFO and MEMORY includes both script code and overheads 2018-10-03 11:19:32 +02:00
Oran Agraandantirez d6aeca862c fix #5024 - commandstats for multi-exec were logged as EXEC.
this was broken a while back by ba9154d7e7
the purpose of which was to fix commandstats for GEOADD
2018-10-03 11:19:32 +02:00
antirez a996b2a285 Fix XINFO comment for consistency. 2018-10-03 11:19:32 +02:00
Bruce Merryandantirez 1a8447b6c1 Fix invalid use of sdsZmallocSize on an embedded string
sdsZmallocSize assumes a dynamically allocated SDS. When given a string
object created by createEmbeddedStringObject, it calls zmalloc_size on a
pointer that isn't the one returned by zmalloc
2018-10-03 11:19:12 +02:00
Bruce Merryandantirez 8dde46ad16 Fix incorrect memory usage accounting in zrealloc
When HAVE_MALLOC_SIZE is false, each call to zrealloc causes used_memory
to increase by PREFIX_SIZE more than it should, due to mis-matched
accounting between the original zmalloc (which includes PREFIX size in
its increment) and zrealloc (which misses it from its decrement).

I've also supplied a command-line test to easily demonstrate the
problem. It's not wired into the test framework, because I don't know
TCL so I'm not sure how to automate it.
2018-10-03 11:19:12 +02:00
Hamid Alaeiandantirez b362a1b758 fix dict get on not found 2018-10-03 11:19:12 +02:00
antirez 55e9df8a41 Try to avoid issues with GCC pragmas and older compilers.
See issue #5394.
2018-10-03 11:19:12 +02:00
antirez b0d2270290 Modules: hellodict example WIP #3: KEYRANGE. 2018-10-03 11:19:12 +02:00
antirez af2f668264 Modules: Modules: dictionary API WIP #13: Compare API exported. 2018-10-03 11:18:55 +02:00
antirez f9a3e6ef79 Modules: Modules: dictionary API WIP #12: DictCompare API. 2018-10-03 11:18:55 +02:00
antirez 01e0341a5f Modules: Modules: dictionary API WIP #11: DictCompareC API. 2018-10-03 11:18:55 +02:00
antirez f9b3ce9a56 Modules: hellodict example WIP #1: GET command. 2018-10-03 11:18:55 +02:00
antirez 36e66d861f Modules: hellodict example WIP #1: SET command. 2018-10-03 11:18:55 +02:00
antirez e33fdbe8e3 Modules: remove useless defines in hellotimer.c 2018-10-03 11:18:55 +02:00
antirez 1c8b22486d Modules: fix top comment of hellotimer.c 2018-10-03 11:18:55 +02:00
Guy Korlandandantirez 7ded552d6d add missing argument to function doc 2018-10-03 11:18:55 +02:00
Pavel Skuratovichandantirez f92b3273fe Fix typo in comment 2018-10-03 11:18:55 +02:00
antirez 57b6c34308 Modules: dictionary API WIP #10: export API to modules. 2018-10-03 11:18:37 +02:00
antirez 3f82e59c0d Modules: dictionary API WIP #9: iterator returning string object. 2018-10-03 11:18:37 +02:00
antirez 6a73aca3fe Modules: dictionary API WIP #8: Iterator next/prev. 2018-10-03 11:18:37 +02:00
antirez ef8413db74 Modules: dictionary API WIP #7: don't store the context.
Storing the context is useless, because we can't really reuse that
later. For instance in the API RM_DictNext() that returns a
RedisModuleString for the next key iterated, the user should pass the
new context, because we may run the keys of the dictionary in a
different context of the one where the dictionary was created. Also the
dictionary may be created without a context, but we may still demand
automatic memory management for the returned strings while iterating.
2018-10-03 11:18:37 +02:00
antirez 05579e38d7 Modules: dictionary API WIP #6: implement automatic memory management. 2018-10-03 11:18:37 +02:00
antirez 11c53f8ca1 Modules: dictionary API work in progress #5: rename API for consistency.
By using the "C" suffix for functions getting pointer/len, we can do the
same in the future for other modules APIs that need a variant with
pointer/len and that are now accepting a RedisModuleString.
2018-10-03 11:18:37 +02:00
antirez 0bd7091b88 Modules: change RedisModuleString API to allow NULL context.
The burden of having to always create RedisModuleString objects within a
module context was too much, especially now that we have threaded
operations and modules are doing more interesting things. The context in
the string API is currently only used for automatic memory managemnet,
so now the API was modified so that the user can opt-out this feature by
passing a NULL context.
2018-10-03 11:18:36 +02:00
antirez 5fc16f173b Modules: dictionary API work in progress #4: reseek API. 2018-10-03 11:18:36 +02:00
antirez 45b7f77970 Modules: dictionary API work in progress #3: Iterator creation. 2018-10-03 11:18:36 +02:00
antirez 8576b0aef7 Modules: dictionary API work in progress #2: Del API. 2018-10-03 11:18:36 +02:00
antirez 4b0fa7a71f Modules: dictionary API work in progress #1. 2018-10-03 11:18:36 +02:00
antirez 282107609f Module cluster flags: use RM_SetClusterFlags() in the example. 2018-10-03 11:18:17 +02:00
antirez 18c5ab930d Module cluster flags: add RM_SetClusterFlags() API. 2018-10-03 11:18:17 +02:00
antirez 4ce6bff28f Module cluster flags: add hooks for NO_FAILOVER flag. 2018-10-03 11:18:17 +02:00
antirez 2ba52889b5 Module cluster flags: add hooks for NO_REDIRECTION flag. 2018-10-03 11:18:17 +02:00
antirez 6a39ece652 Module cluster flags: initial vars / defines added. 2018-10-03 11:18:17 +02:00
antirez 0ff35370d2 Modules: rename the reused static client to something more general. 2018-10-03 11:18:17 +02:00
antirez 2d11ee956f Modules: associate a fake client to timer context callback. 2018-10-03 11:18:17 +02:00
antirez 851b2ed30f Modules: associate a fake client to cluster message context callback.
Fixes #5354.
2018-10-03 11:18:17 +02:00
artixandantirez 148e491148 Cluster Manager: clusterManagerFixOpenSlot now counts node's keys in slot
if node is neither migrating nor importing.
2018-10-03 11:17:59 +02:00
Guy Korlandandantirez 8afca145b9 No need to return "OK"
No need to return "+OK" in this case since the result is an Array of all the nodes
2018-10-03 11:17:59 +02:00
Guy Korlandandantirez 9a278db2c0 typo fix 2018-10-03 11:17:59 +02:00
antirez 2647903616 Revert "fix repeat argument issue and reduce unnessary loop times for redis-cli."
Reverts commit 9505dd2011
since the commit introduced the very serious bug issue #5286.
2018-10-03 11:17:59 +02:00
Guy Korlandandantirez 27b7fb5a93 Fix few typos 2018-10-03 11:17:59 +02:00
Guy Korlandandantirez 233aa2d331 RedisModule_HashSet call must end with NULL
Extended the RedisModule_HashSet doc to mark that each call must end with NULL
2018-10-03 11:17:59 +02:00
antirez a84940721f Sentinel: document how to undo a renamed command. 2018-10-03 11:17:44 +02:00
antirez 6c8a8f2eb7 LOLWUT: split the command from version-specific implementations. 2018-09-14 12:38:07 +02:00
antirez 5c75840641 Slave removal: add a few forgotten aliases for CONFIG SET. 2018-09-14 12:38:07 +02:00
antirez 2da823c43d LOLWUT: add Redis version in the output.
This creates an incentive to run the command and as a side effect
experience the art piece inside.
2018-09-14 12:38:07 +02:00
antirez bfcba42066 LOLWUT: Ness -> Nees. 2018-09-14 12:38:07 +02:00
antirez efed898a4c LOLWUT: Limit maximum CPU effort. 2018-09-14 12:38:07 +02:00
antirez eb0fbd71b4 LOLWUT: change padding conditional to a more direct one. 2018-09-14 12:38:07 +02:00
Slobodan Miškovićandantirez ed08feb70b Fix spelling descrive -> describe 2018-09-14 12:38:07 +02:00
antirez 2ffb4413dd LOLWUT: fix crash when col < 2.
Close #5345.
2018-09-14 12:38:07 +02:00
antirez 55dae693bf LOLWUT: fix structure typo in comment. 2018-09-14 12:38:07 +02:00
antirez 9b3098b93a LOLWUT: Fix license copyright year. 2018-09-14 12:38:07 +02:00
antirez 263dbadcbe LOLWUT: increase the translation factor. 2018-09-14 12:38:07 +02:00
antirez a622f6c018 LOLWUT: change default size to fit a normal terminal better. 2018-09-14 12:38:07 +02:00
antirez 38b0d25af2 LOLWUT: wrap it into a proper command. 2018-09-14 12:38:07 +02:00
antirez 34ebd8985a LOLWUT: draw Schotter by Georg Nees. 2018-09-14 12:38:07 +02:00
antirez 46286e6411 LOLWUT: draw rotated squares using trivial trigonometry. 2018-09-14 12:38:07 +02:00
antirez 2d4143fd42 LOLWUT: draw lines using Bresenham algorithm. 2018-09-14 12:38:07 +02:00
antirez 3546d9ce5f LOLWUT: Rendering of the virtual canvas to a string. 2018-09-14 12:38:07 +02:00
antirez b404a6cef7 LOLWUT: show the output verbatim in redis-cli. 2018-09-14 12:38:07 +02:00
antirez e30ba94fe8 LOLWUT: canvas structure and BSD license on top. 2018-09-14 12:38:07 +02:00
antirez 9c771145bc LOLWUT: Emit Braille unicode according to pixel pattern. 2018-09-14 12:38:06 +02:00
Jakub Vranaandantirez 4a1d6c7df5 Slave removal: capitalize Replica 2018-09-14 12:37:28 +02:00
antirez 72e0368a36 Slave removal: remove slave from integration tests descriptions. 2018-09-14 12:37:28 +02:00
antirez c7841c2b71 Slave removal: remove slave from top-level tests descriptions. 2018-09-14 12:37:28 +02:00
antirez 1b9b19ba8f Slave removal: remove slave from object.c. 2018-09-14 12:37:28 +02:00
antirez 7da266e6c4 Slave removal: remove slave from the README. 2018-09-14 12:37:28 +02:00
antirez 93d803c9a6 Slave removal: server.c logs fixed. 2018-09-14 12:37:28 +02:00
antirez 8943403208 Slave removal: remove slave from sentinel.conf when possible.
All the occurrences translated, but the ones referring to SLAVEOF
must be intact because that describe the actual Sentinel -> Redis
protocol. In theory we could send REPLICAOF to Redis instances, but
actually this would prevent Sentinel to be compatible with older Redis
instances.
2018-09-14 12:37:28 +02:00
antirez 7673d88df7 Slave removal: replace very few things in Sentinel.
SENTINEL REPLICAS was added as an alias, in the configuration rewriting
now it uses known-replica, however all the rest is basically at API
level of logged events and messages having to do with the protocol, so
there is very little to do here compared to the Redis core itself, to
preserve compatibility.
2018-09-14 12:37:28 +02:00
antirez f1de29b34e Slave removal: scripting.c logs and other stuff fixed. 2018-09-14 12:37:27 +02:00
antirez 53fe558eaf Slave removal: replication.c logs fixed. 2018-09-14 12:37:27 +02:00
antirez c92b02ddc2 Slave removal: networking.c logs fixed. 2018-09-14 12:36:59 +02:00
antirez be76ed0c60 Slave removal: blocked.c logs fixed. 2018-09-14 12:36:59 +02:00
antirez 3fd7315149 Slave removal: Make obvious in redis.conf what a replica is. 2018-09-14 12:36:59 +02:00
antirez a22168e49e Slave removal: slave mode -> replica mode text in redis-cli. 2018-09-14 12:36:59 +02:00
antirez 0e222fbec2 Slave removal: fix typo of replicaof. 2018-09-14 12:36:59 +02:00
antirez 34a5615e1a Slave removal: slave -> replica in redis.conf and output buffer option. 2018-09-14 12:36:59 +02:00
antirez 1d2fcf6f45 Slave removal: Convert cluster.c log messages and command names. 2018-09-14 12:36:59 +02:00
antirez 2546158d2d Slave removal: config.c converted + config rewriting hacks.
Aliases added for all the commands mentioning slave. Moreover CONFIG
REWRITE will use the new names, and will be able to reuse the old lines
mentioning the old options.
2018-09-14 12:36:44 +02:00
antirez c0952c0d15 Slave removal: redis-cli --slave -> --replica.
--slave alias remains but is undocumented, just for backward
compatibiltiy.
2018-09-14 12:36:44 +02:00
antirez 1f37f1dd53 Slave removal: SLAVEOF -> REPLICAOF. SLAVEOF is now an alias. 2018-09-14 12:36:44 +02:00
Amin Mesbahandantirez 7928f578e2 Use geohash limit defines in constraint check
Slight robustness improvement, especially if the limit values are
changed, as was suggested in antires/redis#4291 [1].

[1] https://github.com/antirez/redis/pull/4291
2018-09-14 12:36:34 +02:00
Jeffrey Lovitzandantirez bb2bed7866 CLI Help text loop verifies arg count 2018-09-14 12:36:34 +02:00
youjiali1995andantirez 246980d091 sentinel: fix randomized sentinelTimer. 2018-09-14 12:36:34 +02:00
youjiali1995andantirez fa7de8c499 bio: fix bioWaitStepOfType. 2018-09-14 12:36:34 +02:00
Weiliang Liandantirez 7642f9d517 fix usage typo in redis-cli 2018-09-14 12:36:33 +02:00
antirez a72af0eac6 Redis 5.0 RC5. 2018-09-06 13:04:23 +02:00
antirez 1d1bf7f032 Document that effects replication is Redis 5 default. 2018-09-05 19:58:25 +02:00
antirez cfd969c777 Fix scripting tests now that we default to commands repl. 2018-09-05 19:55:55 +02:00
antirez 3e1fb5ff4c Use commands (effects) replication by default in scripts.
See issue #5250 and issue #5292 for more info.
2018-09-05 19:55:52 +02:00
antirez c6c71abe55 Safer script stop condition on OOM.
Here the idea is that we do not want freeMemoryIfNeeded() to propagate a
DEL command before the script and change what happens in the script
execution once it reaches the slave. For example see this potential
issue (in the words of @soloestoy):

On master, we run the following script:

    if redis.call('get','key')
    then
        redis.call('set','xxx','yyy')
    end
    redis.call('set','c','d')

Then when redis attempts to execute redis.call('set','xxx','yyy'), we call freeMemoryIfNeeded(), and the key may get deleted, and because redis.call('set','xxx','yyy') has already been executed on master, this script will be replicated to slave.

But the slave received "DEL key" before the script, and will ignore maxmemory, so after that master has xxx and c, slave has only one key c.

Note that this patch (and other related work) was authored collaboratively in
issue #5250 with the help of @soloestoy and @oranagra.

Related to issue #5250.
2018-09-05 19:55:49 +02:00
antirez dfbce91a14 Propagate read-only scripts as SCRIPT LOAD.
See issue #5250 and the new comments added to the code in this commit
for details.
2018-09-05 19:55:45 +02:00
antirez 1705e42eb2 Don't perform eviction when re-entering the event loop.
Related to #5250.
2018-09-05 19:55:42 +02:00
antirez a0dd6f822b Clarify why remaining may be zero in readQueryFromClient().
See #5304.
2018-09-05 19:55:38 +02:00
zhaozhao.zzandantirez 2eed31a59e networking: fix unexpected negative or zero readlen
To avoid copying buffers to create a large Redis Object which
exceeding PROTO_IOBUF_LEN 32KB, we just read the remaining data
we need, which may less than PROTO_IOBUF_LEN. But the remaining
len may be zero, if the bulklen+2 equals sdslen(c->querybuf),
in client pause context.

For example:

Time1:

python
>>> import os, socket
>>> server="127.0.0.1"
>>> port=6379
>>> data1="*3\r\n$3\r\nset\r\n$1\r\na\r\n$33000\r\n"
>>> data2="".join("x" for _ in range(33000)) + "\r\n"
>>> data3="\n\n"
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> s.settimeout(10)
>>> s.connect((server, port))
>>> s.send(data1)
28

Time2:

redis-cli client pause 10000

Time3:

>>> s.send(data2)
33002
>>> s.send(data3)
2
>>> s.send(data3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
socket.error: [Errno 104] Connection reset by peer

To fix that, we should check if remaining is greater than zero.
2018-09-05 19:55:34 +02:00
antirez 37fb606cdc Merge branch '5.0' of github.com:/antirez/redis into 5.0 2018-09-04 13:13:47 +02:00
zhaozhao.zzandantirez 1898e6ce7f networking: optimize parsing large bulk greater than 32k
If we are going to read a large object from network
try to make it likely that it will start at c->querybuf
boundary so that we can optimize object creation
avoiding a large copy of data.

But only when the data we have not parsed is less than
or equal to ll+2. If the data length is greater than
ll+2, trimming querybuf is just a waste of time, because
at this time the querybuf contains not only our bulk.

It's easy to reproduce the that:

Time1: call `client pause 10000` on slave.

Time2: redis-benchmark -t set -r 10000 -d 33000 -n 10000.

Then slave hung after 10 seconds.
2018-09-04 12:54:16 +02:00
antirez 82fc63d151 Unblocked clients API refactoring. See #4418. 2018-09-04 12:54:14 +02:00
zhaozhao.zzandantirez 839bb52cc2 if master is already unblocked, do not unblock it twice 2018-09-04 12:54:09 +02:00
zhaozhao.zzandantirez 2e1cd82d96 fix multiple unblock for clientsArePaused() 2018-09-04 12:54:02 +02:00
antirez 17233080c3 Make pending buffer processing safe for CLIENT_MASTER client.
Related to #5305.
2018-09-04 12:53:56 +02:00
antirez 42bce87a83 Test: processing of master stream in slave -BUSY state.
See #5297.
2018-09-04 12:53:53 +02:00
antirez 8bf42f6031 After slave Lua script leaves busy state, re-process the master buffer.
Technically speaking we don't really need to put the master client in
the clients that need to be processed, since in practice the PING
commands from the master will take care, however it is conceptually more
sane to do so.
2018-09-04 12:53:48 +02:00
antirez c2b104c73c While the slave is busy, just accumulate master input.
Processing command from the master while the slave is in busy state is
not correct, however we cannot, also, just reply -BUSY to the
replication stream commands from the master. The correct solution is to
stop processing data from the master, but just accumulate the stream
into the buffers and resume the processing later.

Related to #5297.
2018-09-04 12:53:41 +02:00
antirez 7b75f4ae7e Allow scripts to timeout even if from the master instance.
However the master scripts will be impossible to kill.

Related to #5297.
2018-09-04 12:53:38 +02:00
antirez adc4e031bf Allow scripts to timeout on slaves as well.
See reasoning in #5297.
2018-09-04 12:53:35 +02:00
dejun.xdjandantirez 20ec1f0ced Revise the comments of latency command. 2018-09-04 12:53:32 +02:00
Chris Lambandantirez 8e5423eb85 Correct "did not received" -> "did not receive" typos/grammar. 2018-09-04 12:53:28 +02:00
zhaozhao.zzandantirez 395063d74e remove duplicate bind in sentinel.conf 2018-09-04 12:53:21 +02:00
Salvatore SanfilippoandGitHub b221ca41da Merge pull request #5300 from SaschaRoland/xread-block-5299
#5299 Fix blocking XREAD for streams that ran dry
2018-08-31 18:39:46 +02:00
Sascha Roland eea0d3c50a #5299 Fix blocking XREAD for streams that ran dry
The conclusion, that a xread request can be answered syncronously in
case that the stream's last_id is larger than the passed last-received-id
parameter, assumes, that there must be entries present, which could be
returned immediately.
This assumption fails for empty streams that actually contained some
entries which got removed by xdel, ... .

As result, the client is answered synchronously with an empty result,
instead of blocking for new entries to arrive.
An additional check for a non-empty stream is required.
2018-08-29 19:12:29 +02:00
antirez 4cb9ee111e Add maxmemory slave behavior change in the change log. 2018-08-29 12:31:52 +02:00
zhaozhao.zzandantirez 5ad888ba17 Supplement to PR #4835, just take info/memory/command as random commands 2018-08-29 12:29:28 +02:00
zhaozhao.zzandantirez d928487f2b some commands' flags should be set correctly, issue #4834 2018-08-29 12:29:26 +02:00
Oran Agraandantirez af675f0a58 Fix unstable tests on slow machines.
Few tests had borderline thresholds that were adjusted.

The slave buffers test had two issues, preventing the slave buffer from growing:
1) the slave didn't necessarily go to sleep on time, or woke up too early,
   now using SIGSTOP to make sure it goes to sleep exactly when we want.
2) the master disconnected the slave on timeout
2018-08-29 12:29:23 +02:00
antirez f2cd16bea5 Document slave-ignore-maxmemory in redis.conf. 2018-08-29 12:29:18 +02:00
antirez 02d729b492 Make slave-ignore-maxmemory configurable. 2018-08-29 12:29:16 +02:00
antirez 447da44d96 Introduce repl_slave_ignore_maxmemory flag internally.
Note: this breaks backward compatibility with Redis 4, since now slaves
by default are exact copies of masters and do not try to evict keys
independently.
2018-08-29 12:29:14 +02:00
antirez 868b29252b Better variable meaning in processCommand(). 2018-08-29 12:29:00 +02:00
antirez 319f2ee659 Re-apply rebased #2358. 2018-08-29 12:28:55 +02:00
zhaozhao.zzandantirez 22c166da5a block: format code 2018-08-29 12:28:43 +02:00
zhaozhao.zzandantirez c03c591330 block: rewrite BRPOPLPUSH as RPOPLPUSH to propagate 2018-08-29 12:28:39 +02:00
zhaozhao.zzandantirez fcd5ef1624 networking: make setProtocolError simple and clear
Function setProtocolError just records proctocol error
details in server log, set client as CLIENT_CLOSE_AFTER_REPLY.
It doesn't care about querybuf sdsrange, because we
will do it after procotol parsing.
2018-08-29 12:28:35 +02:00
zhaozhao.zzandantirez 656e4b2f9d networking: just move qb_pos instead of sdsrange in processInlineBuffer 2018-08-29 12:28:31 +02:00
zhaozhao.zzandantirez 2c7972cec8 networking: just return C_OK if multibulk processing saw a <= 0 length. 2018-08-29 12:28:25 +02:00
zhaozhao.zzandantirez 1203a04f5e adjust qbuf to 26 in test case for client list 2018-08-29 12:28:22 +02:00
zhaozhao.zzandantirez aff86fa1f5 pipeline: do not sdsrange querybuf unless all commands processed
This is an optimization for processing pipeline, we discussed a
problem in issue #5229: clients may be paused if we apply `CLIENT
PAUSE` command, and then querybuf may grow too large, the cost of
memmove in sdsrange after parsing a completed command will be
horrible. The optimization is that parsing all commands in queyrbuf
, after that we can just call sdsrange only once.
2018-08-29 12:28:19 +02:00
Chris Lambandantirez 45a6c5be2a Use SOURCE_DATE_EPOCH over unreproducible uname + date calls.
See <https://reproducible-builds.org/specs/source-date-epoch/> for more
details.

Signed-off-by: Chris Lamb <chris@chris-lamb.co.uk>
2018-08-29 12:28:16 +02:00
Chris Lambandantirez 186df14811 Make some defaults explicit in the sentinel.conf for package maintainers
This may look a little pointless (and it is a complete no-op change here)
but as package maintainers need to modify these lines to actually
daemonize (etc. etc) but it's far preferable if the diff is restricted to
actually changing just that bit, not adding docs, etc. The less diff the
better, in general.

Signed-off-by: Chris Lamb <chris@chris-lamb.co.uk>
2018-08-29 12:28:12 +02:00
dejun.xdjandantirez b59f04a099 Streams: ID of xclaim command starts from the sixth argument. 2018-08-29 12:28:09 +02:00
shenlongxingandantirez a3f2437bdd Fix stream command paras 2018-08-29 12:28:06 +02:00
antirez df9112354b Fix AOF comment to report the current behavior.
Realted to #5201.
2018-08-29 12:28:04 +02:00
antirez 5b06bdf457 Redis 5.0 RC4. 2018-08-03 16:46:06 +02:00
antirez a4d1201eba Test suite: add --loop option.
Very useful with --stop in order to catch heisenbugs.
2018-08-03 12:28:04 +02:00
antirez 273d8191e1 Test suite: new --stop option.
It pauses the test execution once the first failure is found.
2018-08-03 12:28:04 +02:00
antirez fbbcc6a657 Streams IDs parsing refactoring.
Related to #5184.
2018-08-02 18:34:42 +02:00
antirez 70c4bcb7bc Test: new sorted set skiplist order consistency.
This should be able to find new bugs and regressions about the new
sorted set update function when ZADD is used to update an element
already existing.

The test is able to find the bug fixed at 2f282aee immediately.
2018-08-02 18:34:34 +02:00
antirez 63addc5c1e Fix zslUpdateScore() edge case.
When the element new score is the same of prev/next node, the
lexicographical order kicks in, so we can safely update the node in
place only when the new score is strictly between the adjacent nodes
but never equal to one of them.

Technically speaking we could do extra checks to make sure that even if the
score is the same as one of the adjacent nodes, we can still update on
place, but this rarely happens, so probably not a good deal to make it
more complex.

Related to #5179.
2018-08-02 18:34:34 +02:00
antirez 724740cc19 More commenting of zslUpdateScore(). 2018-08-02 18:34:34 +02:00
antirez ddc87eef4f Explain what's the point of zslUpdateScore() in top comment. 2018-08-02 18:34:34 +02:00
antirez 741f29ea52 Remove old commented zslUpdateScore() from source. 2018-08-02 18:34:34 +02:00
antirez 201168368a Optimize zslUpdateScore() as asked in #5179. 2018-08-02 18:34:34 +02:00
antirez 8c297e8b43 zsetAdd() refactored adding zslUpdateScore(). 2018-08-02 18:34:34 +02:00
dejun.xdjandantirez bd2f3f6bb1 Streams: rearrange the usage of '-' and '+' IDs in stream commands. 2018-08-02 18:34:34 +02:00
dejun.xdjandantirez c0c06b8456 Streams: add mmid_supp argument in streamParseIDOrReply().
If 'mmid_supp' is set to 0, "-" and "+" will be
treated as an invalid ID.
2018-08-02 18:34:34 +02:00
antirez ab237a8e26 Minor improvements to PR #5187. 2018-08-02 18:34:34 +02:00
Oran Agraandantirez 1ce3cf7a8f test suite conveniency improvements
* allowing --single to be repeated
* adding --only so that only a specific test inside a unit can be run
* adding --skiptill useful to resume a test that crashed passed the problematic unit.
  useful together with --clients 1
* adding --skipfile to use a file containing list of tests names to skip
* printing the names of the tests that are skiped by skipfile or denytags
* adding --config to add config file options from command line
2018-08-02 18:34:34 +02:00
Oran Agraandantirez 3662289995 add DEBUG LOG, to to assist test suite debugging 2018-08-02 18:34:34 +02:00
antirez 83d4311acd Cluster cron announce IP minor refactoring. 2018-08-02 18:34:04 +02:00
shenlongxingandantirez a633f8e130 Fix cluster-announce-ip memory leak 2018-08-02 18:34:04 +02:00
antirez 24c455381a Tranfer -> transfer typo fixed. 2018-08-02 18:34:04 +02:00
zhaozhao.zzandantirez c609f240a5 refactor dbOverwrite to make lazyfree work 2018-08-02 18:34:04 +02:00
antirez 9e97173988 Refactoring: replace low-level checks with writeCommandsDeniedByDiskError(). 2018-08-02 18:34:04 +02:00
antirez 0e77cef096 Fix writeCommandsDeniedByDiskError() inverted return value. 2018-08-02 18:34:04 +02:00
antirez acfe9d138a Better top comment for writeCommandsDeniedByDiskError(). 2018-08-02 18:34:04 +02:00
antirez 4e933e0059 Introduce writeCommandsDeniedByDiskError(). 2018-08-02 18:34:04 +02:00
WuYunlongandantirez 41607dfd25 Consider aof write error as well as rdb in lua script. 2018-08-02 18:34:04 +02:00
Salvatore SanfilippoandGitHub 1d073a64f7 Merge pull request #5168 from rpv-tomsk/issue-5033
INFO CPU: higher precision of reported values
2018-07-30 18:03:15 +02:00
Guy Korlandandantirez 2db31fd4bb Few typo fixes 2018-07-30 18:00:32 +02:00
antirez 64242757d7 Add year in log.
User: "is there a reason why redis server logs  are missing the year in
the "date time"?"

Me: "I guess I did not imagine it would be stable enough to run for
several years".
2018-07-30 18:00:30 +02:00
antirez 50be4a1f5c Document dynamic-hz in the example redis.conf. 2018-07-30 18:00:27 +02:00
antirez 9a76472d71 Make dynamic hz actually configurable. 2018-07-30 18:00:24 +02:00
antirez a330d06c82 Control dynamic HZ via server configuration. 2018-07-30 18:00:21 +02:00
antirez d42602ffc8 Dynamic HZ: adapt cron frequency to number of clients. 2018-07-30 18:00:19 +02:00
antirez 7b5f0223f8 Dynamic HZ: separate hz from the configured hz.
This way we can remember what the user configured HZ is, but change the
actual HZ dynamically if needed in the dynamic HZ feature
implementation.
2018-07-30 18:00:16 +02:00
antirez 037b00dece Remove useless conditional from emptyDb().
Related to #4852.
2018-07-30 17:59:52 +02:00
antirez 0e97ae79b0 Make emptyDb() change introduced in #4852 simpler to read. 2018-07-30 17:59:49 +02:00
zhaozhao.zzandantirez f7740fafbd optimize flushdb, avoid useless loops 2018-07-30 17:59:46 +02:00
zhaozhao.zzandantirez 0c00837669 Streams: fix xdel memory leak 2018-07-30 17:59:43 +02:00
antirez dc600a25cd Example the magic +1 in migrateCommand().
Related to #5154.
2018-07-30 17:59:40 +02:00
antirez d6827ab638 Make changes of PR #5154 hopefully simpler. 2018-07-30 17:59:37 +02:00
WuYunlongandantirez 89ec14531b Do not migrate already expired keys. 2018-07-30 17:59:35 +02:00
Pavel Rochnyack cd25ed17b9 INFO CPU: higher precision of reported values
Closes: #5033
2018-07-25 21:50:24 +07:00
antirez 6bfb4745fa Streams: refactoring of next entry seek in the iterator.
After #5161 the code could be made a bit more obvious for newcomers.
2018-07-24 11:16:25 +02:00
zhaozhao.zzandantirez 4724548e07 Streams: skip master fileds only when we are going forward in streamIteratorGetID 2018-07-24 11:16:21 +02:00
Oran Agraandantirez 4b79fdf1d6 fix slave buffer test suite false positives
it looks like on slow machines we're getting:
[err]: slave buffer are counted correctly in tests/unit/maxmemory.tcl
Expected condition '$slave_buf > 2*1024*1024' to be true (16914 > 2*1024*1024)

this is a result of the slave waking up too early and eating the
slave buffer before the traffic and the test ends.
2018-07-24 11:16:15 +02:00
antirez a1e081f719 string2ll(): better commenting. 2018-07-24 11:16:11 +02:00
dsomeshwarandantirez 8b4fe752ec removing redundant check 2018-07-24 11:16:07 +02:00
antirez 9e5bf0471a Restore string2ll() to original version.
See PR #5157.
2018-07-24 11:16:05 +02:00
Oran Agraandantirez c2ecdcde98 fix recursion typo in zmalloc_usable 2018-07-24 11:16:00 +02:00
antirez 4f742bd6bf string2ll(): remove duplicated check for special case.
Related to #5157. The PR author correctly indentified that the check was
duplicated, but removing the second one introduces a bug that was fixed
in the past (hence the duplication). Instead we can remove the first
instance of the check without issues.
2018-07-24 11:15:57 +02:00
antirez a4efac0067 string2ll(): test for NULL pointer in all the cases. 2018-07-24 11:15:54 +02:00
antirez 2c07c107cb Change 42 to 1000 as warning level for cached scripts.
Related to #4883.
2018-07-23 18:46:12 +02:00
Itamar Haberandantirez 270903d6b2 Adds Lua overheads to MEMORY STATS, smartens the MEMORY DOCTOR 2018-07-23 18:46:12 +02:00
Itamar Haberandantirez faf3dbfcf9 Adds memory information about the script's cache to INFO
Implementation notes: as INFO is "already broken", I didn't want to break it further. Instead of computing the server.lua_script dict size on every call, I'm keeping a running sum of the body's length and dict overheads.

This implementation is naive as it **does not** take into consideration dict rehashing, but that inaccuracy pays off in speed ;)

Demo time:

```bash
$ redis-cli info memory | grep "script"
used_memory_scripts:96
used_memory_scripts_human:96B
number_of_cached_scripts:0
$ redis-cli eval "" 0 ; redis-cli info memory | grep "script"
(nil)
used_memory_scripts:120
used_memory_scripts_human:120B
number_of_cached_scripts:1
$ redis-cli script flush ; redis-cli info memory | grep "script"
OK
used_memory_scripts:96
used_memory_scripts_human:96B
number_of_cached_scripts:0
$ redis-cli eval "return('Hello, Script Cache :)')" 0 ; redis-cli info memory | grep "script"
"Hello, Script Cache :)"
used_memory_scripts:152
used_memory_scripts_human:152B
number_of_cached_scripts:1
$ redis-cli eval "return redis.sha1hex(\"return('Hello, Script Cache :)')\")" 0 ; redis-cli info memory | grep "script"
"1be72729d43da5114929c1260a749073732dc822"
used_memory_scripts:232
used_memory_scripts_human:232B
number_of_cached_scripts:2
✔ 19:03:54 redis [lua_scripts-in-info-memory L ✚…⚑] $ redis-cli evalsha 1be72729d43da5114929c1260a749073732dc822 0
"Hello, Script Cache :)"
```
2018-07-23 18:46:12 +02:00
antirez 49841a54db Fix merge errors.
For some reason I made a merge fiasco between 5.0 and unstable.
This fixes the error introduced by merging unstable.
2018-07-23 18:36:54 +02:00
antirez 77a7ec7200 Merge branch 'unstable' into 5.0 branch 2018-07-23 18:29:33 +02:00
antirez 48dfd42d72 Redis 5.0 RC3. 2018-06-14 09:52:48 +02:00
antirez b4b0251ca6 Rax library updated. 2018-06-14 09:50:46 +02:00
antirez f7209749a6 Redis 5.0 RC2. 2018-06-13 13:26:28 +02:00
antirez 5171a3ff8e RDB: Apply fix to rdbLoadMillisecondTime() only for new RDB versions.
This way we let big endian systems to still load old RDB versions.
However newver versions will be saved and loaded in a way that make RDB
expires cross-endian again. Thanks to @oranagra for the reporting and
the discussion about this problem, leading to this fix.
2018-06-13 13:26:13 +02:00
antirez af7069627e Streams: generate a few additional events.
Currently it does not look it's sensible to generate events for streams
consumer groups modification, being them metadata, however at least for
key-level events, like the creation or removal of a consumer group, I
added a few events here and there. Later we can evaluate if it makes
sense to add more. From the POV instead of WAIT (in Redis transaciton)
and signaling the key as modified, it looks like that the transaction
should not fail when a stream is modified, so no calls are made in
consumer groups related functions to signalModifiedKey().
2018-06-13 13:26:12 +02:00
antirez 3ad68a07ff Fix rdbSaveKeyValuePair() integer overflow.
Again thanks to @oranagra. The object idle time does not fit into an int
sometimes: use the native type that the serialization function will get
as argument, which is uint64_t.
2018-06-13 13:26:12 +02:00
antirez 68382091a0 In scanDatabaseForReadyLists() now we need to handle ZSETs as well.
Since the introduction of ZPOP makes this needed. Thanks to @oranagra
for reporting.
2018-06-13 13:26:12 +02:00
antirez 6b0cdbd903 RDB: store times consistently in little endian.
I'm not sure how this escaped the attention of Redis users for years,
but finally @oranagra reported this issue... Thanks to Oran.
2018-06-13 13:26:12 +02:00
antirez 574017b7d2 Streams: improve type correctness in t_stream.c. 2018-06-13 13:26:12 +02:00
antirez cd2b5a79ff Fix XGROUP help missing space. 2018-06-13 13:26:12 +02:00
Baoyi Chenandantirez 8246a38e83 fix typo
fix [#5005](https://github.com/antirez/redis/issues/5005)
2018-06-13 13:26:12 +02:00
antirez b80e4b6925 Security: fix redis-cli buffer overflow.
Thanks to Fakhri Zulkifli for reporting it.

The fix switched to dynamic allocation, copying the final prompt in the
static buffer only at the end.
2018-06-13 12:40:43 +02:00
antirez 5e0d9841d4 Security: fix Lua struct package offset handling.
After the first fix to the struct package I found another similar
problem, which is fixed by this patch. It could be reproduced easily by
running the following script:

    return struct.unpack('f', "xxxxxxxxxxxxx",-3)

The above will access bytes before the 'data' pointer.
2018-06-13 12:40:43 +02:00
antirez e27a040146 Security: more cmsgpack fixes by @soloestoy.
@soloestoy sent me this additional fixes, after searching for similar
problems to the one reported in mp_pack(). I'm committing the changes
because it was not possible during to make a public PR to protect Redis
users and give Redis providers some time to patch their systems.
2018-06-13 12:40:43 +02:00
antirez c5dfff465e Security: update Lua struct package for security.
During an auditing Apple found that the "struct" Lua package
we ship with Redis (http://www.inf.puc-rio.br/~roberto/struct/) contains
a security problem. A bound-checking statement fails because of integer
overflow. The bug exists since we initially integrated this package with
Lua, when scripting was introduced, so every version of Redis with
EVAL/EVALSHA capabilities exposed is affected.

Instead of just fixing the bug, the library was updated to the latest
version shipped by the author.
2018-06-13 12:40:43 +02:00
antirez 86aade7475 Security: fix Lua cmsgpack library stack overflow.
During an auditing effort, the Apple Vulnerability Research team discovered
a critical Redis security issue affecting the Lua scripting part of Redis.

-- Description of the problem

Several years ago I merged a pull request including many small changes at
the Lua MsgPack library (that originally I authored myself). The Pull
Request entered Redis in commit 90b6337c1, in 2014.
Unfortunately one of the changes included a variadic Lua function that
lacked the check for the available Lua C stack. As a result, calling the
"pack" MsgPack library function with a large number of arguments, results
into pushing into the Lua C stack a number of new values proportional to
the number of arguments the function was called with. The pushed values,
moreover, are controlled by untrusted user input.

This in turn causes stack smashing which we believe to be exploitable,
while not very deterministic, but it is likely that an exploit could be
created targeting specific versions of Redis executables. However at its
minimum the issue results in a DoS, crashing the Redis server.

-- Versions affected

Versions greater or equal to Redis 2.8.18 are affected.

-- Reproducing

Reproduce with this (based on the original reproduction script by
Apple security team):

https://gist.github.com/antirez/82445fcbea6d9b19f97014cc6cc79f8a

-- Verification of the fix

The fix was tested in the following way:

1) I checked that the problem is no longer observable running the trigger.
2) The Lua code was analyzed to understand the stack semantics, and that
actually enough stack is allocated in all the cases of mp_pack() calls.
3) The mp_pack() function was modified in order to show exactly what items
in the stack were being set, to make sure that there is no silent overflow
even after the fix.

-- Credits

Thank you to the Apple team and to the other persons that helped me
checking the patch and coordinating this communication.
2018-06-13 12:40:43 +02:00
antirez 071e235a52 Regression test for issue #5006. 2018-06-12 13:15:53 +02:00
Shen Longxingandantirez 6acd8a31a4 fix active-defrag-threshold value error
The active-defrag-threshold-lower/active-defrag-threshold-upper min/max  value in redis.conf should be consistent with 'config set' command.
2018-06-12 13:15:53 +02:00
antirez 20bc3786c0 Streams: fix backward iteration when entry is not flagged SAMEFIELD.
See issue #5006. The comment in the code was also wrong and
was rectified as well.
2018-06-12 13:15:53 +02:00
antirez 2cadef46f7 Streams: increment dirty counter for XGROUP SETID/DESTROY.
See issue #5005 comments.
2018-06-12 13:15:53 +02:00
antirez 093ec57d06 Use a less aggressive query buffer resize policy.
A user with many connections (10 thousand) on a single Redis server
reports in issue #4983 that sometimes Redis is idle becuase at the same
time many clients need to resize their query buffer according to the old
policy.

It looks like this was created by the fact that we allow the query
buffer to grow without problems to a size up to PROTO_MBULK_BIG_ARG
normally, but when the client is idle we immediately are more strict,
and a query buffer greater than 1024 bytes is already enough to trigger
the resize. So for instance if most of the clients stop at the same time
this issue should be easily triggered.

This behavior actually looks odd, and there should be only a clear limit
after we say, let's look at this query buffer to check if it's time to
resize it. This commit puts the limit at PROTO_MBULK_BIG_ARG, and the
check is performed both if compared to the peak usage the current usage
is too big, or if the client is idle.

Then when the check is performed, to waste just a few kbytes is
considered enough to proceed with the resize. This should fix the issue.
2018-06-12 13:15:52 +02:00
antirez d769ce0a20 Fix client unblocking for XREADGROUP, issue #4978.
We unblocked the client too early, when the group name object was no
longer valid in client->bpop, so propagating XCLAIM later in
streamPropagateXCLAIM() deferenced a field already set to NULL.
2018-06-12 13:15:52 +02:00
antirez 280b2dc1e4 Improved regression test for #4906.
Removing the fix about 50% of the times the test will not be able to
pass cleanly. It's very hard to write a test that will always fail, or
actually, it is possible but then it's likely that it will consistently
pass if we change some random bit, so better to use randomization here.
2018-06-12 13:15:52 +02:00
antirez 260b32edff Regression test for the dictScan() issue #4906. 2018-06-12 13:15:52 +02:00
zhaozhao.zzandantirez 28e0ca5a8c Streams: checkType for xread & xinfo 2018-06-12 13:15:52 +02:00
zhaozhao.zzandantirez 9d2b1f294d Streams: lookupKey[Read->Write]OrReply in xdel and xtrim 2018-06-12 13:15:52 +02:00
michael-grunderandantirez 3d4ad250cd Abort in XGROUP if the key is not a stream 2018-06-12 13:15:52 +02:00
shenlongxingandantirez 5a3d490c50 fix integer case error 2018-06-12 13:15:52 +02:00
antirez 24fb4e3894 Implement DEBUG htstats-key. 2018-06-12 13:15:52 +02:00
antirez a67ecb91f9 redis-cli inline help updated. 2018-06-12 13:15:52 +02:00
antirez 5cdd475066 Add the stream group to the script generating the help. 2018-06-12 13:15:52 +02:00
shenlongxingandantirez 2cd6b001da fix stream config typo 2018-06-12 13:15:52 +02:00
antirez b2a5a70e5a Streams: better document the max node limits. 2018-06-12 13:15:13 +02:00
antirez eb26a8174a Typo: entires -> entries in several places. 2018-06-12 13:15:13 +02:00
antirez f2d65c380d Streams: make macro node limits configurable. 2018-06-12 13:15:13 +02:00
antirez b16e5432b6 Streams: max node limits only checked if non zero. 2018-06-12 13:15:13 +02:00
antirez d01af7abe4 Streams: use non static macro node limits.
Also add the concept of size/items limit, instead of just having as
limit the number of bytes.
2018-06-12 13:15:13 +02:00
shenlongxingandantirez 77acc63eef Fix write() errno error 2018-06-12 13:15:13 +02:00
michael-grunderandantirez 3ddc537605 Return early in XPENDING if sent a nonexistent consumer group. 2018-06-12 13:15:13 +02:00
Krzysztof Filipekandantirez b85086c573 Typo in preprocessor condition 2018-06-12 13:15:13 +02:00
zhaozhao.zzandantirez 4a9670f517 RDB: expand dict if needed when rdb load object 2018-06-12 13:15:13 +02:00
zhaozhao.zzandantirez d5712d2151 adjust position of _dictNextPower in dictExpand 2018-06-12 13:15:13 +02:00
zhaozhao.zzandantirez 3ca02fb939 zset: change the span of zskiplistNode to unsigned long 2018-06-12 13:15:13 +02:00
zhaozhao.zzandantirez 59e51f92d2 zset: fix the int problem 2018-06-12 13:15:13 +02:00
antirez 3502d1f16b Fix streamIteratorRemoveEntry() to update elements count.
Close #4989.
2018-06-06 11:41:06 +02:00
antirez 1317bab0d1 ZPOP: invert score-ele to match ZRANGE WITHSCORES order. 2018-06-05 17:06:46 +02:00
antirez 63ed0f7593 Remove XINFO <key> special form.
As observed by Michael Grunder this usage while practical is
inconsistent because for instance it does not work against a key called
HELP. Removed.
2018-06-05 17:06:46 +02:00
antirez 8b9b02ade5 XGROUP SETID implemented + consumer groups core fixes.
Now that we have SETID, the inetrnals of consumer groups should be able
to handle the case of the same message delivered multiple times just
as a side effect of calling XREADGROUP. Normally this should never
happen but if the admin manually "XGROUP SETID mykey mygroup 0",
messages will get re-delivered to clients waiting for the ">" special
ID. The consumer groups internals were not able to handle the case of a
message re-delivered in this circumstances that was already assigned to
another owner.
2018-06-04 17:30:34 +02:00
Yossi Gottliebandantirez d4ed462b39 Clean gcc 7.x warnings, redis-cli cluster fix. 2018-06-04 17:30:34 +02:00
antirez 525a7b399b XGROUP DESTROY implemented. 2018-06-04 12:58:41 +02:00
赵磊andantirez ae9687de6f Fix dictScan(): It can't scan all buckets when dict is shrinking. 2018-06-01 16:54:28 +02:00
artixandantirez 9bb7c4b2e5 Cluster Manager: fixed master_id check in clusterManagerNodeLoadInfo 2018-06-01 16:20:08 +02:00
zhaozhao.zzandantirez 07142f034a ZPOP: unblock multiple clients in right way 2018-06-01 16:20:08 +02:00
zhaozhao.zzandantirez 733348bbb4 MEMORY: fix the missing of monitor clients buffers 2018-06-01 16:20:08 +02:00
Motaandantirez 4223c41587 Fix debug crash-and-recover help info. 2018-06-01 16:20:08 +02:00
antirez ad4e5ca925 Capitalize OBJECT HELP subcommands. 2018-05-31 17:21:20 +02:00
Remi Colletandantirez d55c5e8caf include stdint.h for unit64_t definition 2018-05-31 17:21:20 +02:00
artixandantirez d2aa10d029 Cluster Manager: fixed infinite loop in rebalance (Issue #4941). 2018-05-31 15:58:55 +02:00
antirez 2ee4a1c980 Redis 5.0 RC1. 2018-05-29 14:19:01 +02:00
38 changed files with 2541 additions and 1366 deletions
+2145 -11
View File
File diff suppressed because it is too large Load Diff
+1 -21
View File
@@ -21,11 +21,6 @@ NODEPS:=clean distclean
# Default settings
STD=-std=c99 -pedantic -DREDIS_STATIC=''
ifneq (,$(findstring clang,$(CC)))
ifneq (,$(findstring FreeBSD,$(uname_S)))
STD+=-Wno-c11-extensions
endif
endif
WARN=-Wall -W -Wno-missing-field-initializers
OPT=$(OPTIMIZATION)
@@ -46,10 +41,6 @@ endif
# To get ARM stack traces if Redis crashes we need a special C flag.
ifneq (,$(filter aarch64 armv,$(uname_M)))
CFLAGS+=-funwind-tables
else
ifneq (,$(findstring armv,$(uname_M)))
CFLAGS+=-funwind-tables
endif
endif
# Backwards compatibility for selecting an allocator
@@ -102,20 +93,10 @@ else
ifeq ($(uname_S),OpenBSD)
# OpenBSD
FINAL_LIBS+= -lpthread
ifeq ($(USE_BACKTRACE),yes)
FINAL_CFLAGS+= -DUSE_BACKTRACE -I/usr/local/include
FINAL_LDFLAGS+= -L/usr/local/lib
FINAL_LIBS+= -lexecinfo
endif
else
ifeq ($(uname_S),FreeBSD)
# FreeBSD
FINAL_LIBS+= -lpthread -lexecinfo
else
ifeq ($(uname_S),DragonFly)
# FreeBSD
FINAL_LIBS+= -lpthread -lexecinfo
FINAL_LIBS+= -lpthread
else
# All the other OSes (notably Linux)
FINAL_LDFLAGS+= -rdynamic
@@ -125,7 +106,6 @@ endif
endif
endif
endif
endif
# Include paths to dependencies
FINAL_CFLAGS+= -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src
+1 -1
View File
@@ -1,7 +1,7 @@
/* This file implements atomic counters using __atomic or __sync macros if
* available, otherwise synchronizing different threads using a mutex.
*
* The exported interface is composed of three macros:
* The exported interaface is composed of three macros:
*
* atomicIncr(var,count) -- Increment the atomic counter
* atomicGetIncr(var,oldvalue_var,count) -- Get and increment the atomic counter
+6 -14
View File
@@ -5164,10 +5164,10 @@ try_again:
serverAssertWithInfo(c,NULL,rioWriteBulkLongLong(&cmd,dbid));
}
int non_expired = 0; /* Number of keys that we'll find non expired.
Note that serializing large keys may take some time
so certain keys that were found non expired by the
lookupKey() function, may be expired later. */
int expired = 0; /* Number of keys that we'll find already expired.
Note that serializing large keys may take some time
so certain keys that were found non expired by the
lookupKey() function, may be expired later. */
/* Create RESTORE payload and generate the protocol to call the command. */
for (j = 0; j < num_keys; j++) {
@@ -5177,16 +5177,11 @@ try_again:
if (expireat != -1) {
ttl = expireat-mstime();
if (ttl < 0) {
expired++;
continue;
}
if (ttl < 1) ttl = 1;
}
/* Relocate valid (non expired) keys into the array in successive
* positions to remove holes created by the keys that were present
* in the first lookup but are now expired after the second lookup. */
kv[non_expired++] = kv[j];
serverAssertWithInfo(c,NULL,
rioWriteBulkCount(&cmd,'*',replace ? 5 : 4));
@@ -5214,9 +5209,6 @@ try_again:
serverAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,"REPLACE",7));
}
/* Fix the actual number of keys we are migrating. */
num_keys = non_expired;
/* Transfer the query to the other node in 64K chunks. */
errno = 0;
{
@@ -5258,7 +5250,7 @@ try_again:
* command name itself. */
if (!copy) newargv = zmalloc(sizeof(robj*)*(num_keys+1));
for (j = 0; j < num_keys; j++) {
for (j = 0; j < num_keys-expired; j++) {
if (syncReadLine(cs->fd, buf2, sizeof(buf2), timeout) <= 0) {
socket_error = 1;
break;
+3 -3
View File
@@ -120,7 +120,7 @@ const char *configEnumGetName(configEnum *ce, int val) {
return NULL;
}
/* Wrapper for configEnumGetName() returning "unknown" instead of NULL if
/* Wrapper for configEnumGetName() returning "unknown" insetad of NULL if
* there is no match. */
const char *configEnumGetNameOrUnknown(configEnum *ce, int val) {
const char *name = configEnumGetName(ce,val);
@@ -684,7 +684,7 @@ void loadServerConfigFromString(char *config) {
goto loaderr;
}
} else if ((!strcasecmp(argv[0],"cluster-slave-no-failover") ||
!strcasecmp(argv[0],"cluster-replica-no-failover")) &&
!strcasecmp(argv[0],"cluster-replica-no-failiver")) &&
argc == 2)
{
server.cluster_slave_no_failover = yesnotoi(argv[1]);
@@ -1248,7 +1248,7 @@ void configSetCommand(client *c) {
if (server.maxmemory < zmalloc_used_memory()) {
serverLog(LL_WARNING,"WARNING: the new maxmemory value set via CONFIG SET is smaller than the current memory usage. This will result in key eviction and/or the inability to accept new write commands depending on the maxmemory-policy.");
}
freeMemoryIfNeededAndSafe();
freeMemoryIfNeeded();
}
} config_set_memory_field(
"proto-max-bulk-len",server.proto_max_bulk_len) {
+1 -3
View File
@@ -62,9 +62,7 @@
#endif
/* Test for backtrace() */
#if defined(__APPLE__) || (defined(__linux__) && defined(__GLIBC__)) || \
defined(__FreeBSD__) || (defined(__OpenBSD__) && defined(USE_BACKTRACE))\
|| defined(__DragonFly__)
#if defined(__APPLE__) || (defined(__linux__) && defined(__GLIBC__))
#define HAVE_BACKTRACE 1
#endif
+22 -32
View File
@@ -38,8 +38,6 @@
* C-level DB API
*----------------------------------------------------------------------------*/
int keyIsExpired(redisDb *db, robj *key);
/* Update LFU when an object is accessed.
* Firstly, decrement the counter if the decrement time is reached.
* Then logarithmically increment the counter, and update the access time. */
@@ -104,10 +102,7 @@ robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
/* Key expired. If we are in the context of a master, expireIfNeeded()
* returns 0 only when the key does not exist at all, so it's safe
* to return NULL ASAP. */
if (server.masterhost == NULL) {
server.stat_keyspace_misses++;
return NULL;
}
if (server.masterhost == NULL) return NULL;
/* However if we are in the context of a slave, expireIfNeeded() will
* not really try to expire the key, it only returns information
@@ -126,7 +121,6 @@ robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
server.current_client->cmd &&
server.current_client->cmd->flags & CMD_READONLY)
{
server.stat_keyspace_misses++;
return NULL;
}
}
@@ -212,7 +206,7 @@ void dbOverwrite(redisDb *db, robj *key, robj *val) {
* 2) clients WATCHing for the destination key notified.
* 3) The expire time of the key is reset (the key is made persistent).
*
* All the new keys in the database should be created via this interface. */
* All the new keys in the database should be craeted via this interface. */
void setKey(redisDb *db, robj *key, robj *val) {
if (lookupKeyWrite(db,key) == NULL) {
dbAdd(db,key,val);
@@ -549,7 +543,7 @@ void keysCommand(client *c) {
if (allkeys || stringmatchlen(pattern,plen,key,sdslen(key),0)) {
keyobj = createStringObject(key,sdslen(key));
if (!keyIsExpired(c->db,keyobj)) {
if (expireIfNeeded(c->db,keyobj) == 0) {
addReplyBulk(c,keyobj);
numkeys++;
}
@@ -1126,25 +1120,6 @@ void propagateExpire(redisDb *db, robj *key, int lazy) {
decrRefCount(argv[1]);
}
/* Check if the key is expired. */
int keyIsExpired(redisDb *db, robj *key) {
mstime_t when = getExpire(db,key);
if (when < 0) return 0; /* No expire for this key */
/* Don't expire anything while loading. It will be done later. */
if (server.loading) return 0;
/* If we are in the context of a Lua script, we pretend that time is
* blocked to when the Lua script started. This way a key can expire
* only the first time it is accessed and not in the middle of the
* script execution, making propagation to slaves / AOF consistent.
* See issue #1525 on Github for more information. */
mstime_t now = server.lua_caller ? server.lua_time_start : mstime();
return now > when;
}
/* This function is called when we are going to perform some operation
* in a given key, but such key may be already logically expired even if
* it still exists in the database. The main way this function is called
@@ -1165,17 +1140,32 @@ int keyIsExpired(redisDb *db, robj *key) {
* The return value of the function is 0 if the key is still valid,
* otherwise the function returns 1 if the key is expired. */
int expireIfNeeded(redisDb *db, robj *key) {
if (!keyIsExpired(db,key)) return 0;
mstime_t when = getExpire(db,key);
mstime_t now;
/* If we are running in the context of a slave, instead of
* evicting the expired key from the database, we return ASAP:
if (when < 0) return 0; /* No expire for this key */
/* Don't expire anything while loading. It will be done later. */
if (server.loading) return 0;
/* If we are in the context of a Lua script, we pretend that time is
* blocked to when the Lua script started. This way a key can expire
* only the first time it is accessed and not in the middle of the
* script execution, making propagation to slaves / AOF consistent.
* See issue #1525 on Github for more information. */
now = server.lua_caller ? server.lua_time_start : mstime();
/* If we are running in the context of a slave, return ASAP:
* the slave key expiration is controlled by the master that will
* send us synthesized DEL operations for expired keys.
*
* Still we try to return the right information to the caller,
* that is, 0 if we think the key should be still valid, 1 if
* we think the key is expired at this time. */
if (server.masterhost != NULL) return 1;
if (server.masterhost != NULL) return now > when;
/* Return when this key has not expired */
if (now <= when) return 0;
/* Delete the key */
server.stat_expiredkeys++;
+135 -325
View File
@@ -37,11 +37,7 @@
#ifdef HAVE_BACKTRACE
#include <execinfo.h>
#ifndef __OpenBSD__
#include <ucontext.h>
#else
typedef ucontext_t sigcontext_t;
#endif
#include <fcntl.h>
#include "bio.h"
#include <unistd.h>
@@ -74,7 +70,7 @@ void xorDigest(unsigned char *digest, void *ptr, size_t len) {
digest[j] ^= hash[j];
}
void xorStringObjectDigest(unsigned char *digest, robj *o) {
void xorObjectDigest(unsigned char *digest, robj *o) {
o = getDecodedObject(o);
xorDigest(digest,o->ptr,sdslen(o->ptr));
decrRefCount(o);
@@ -104,151 +100,12 @@ void mixDigest(unsigned char *digest, void *ptr, size_t len) {
SHA1Final(digest,&ctx);
}
void mixStringObjectDigest(unsigned char *digest, robj *o) {
void mixObjectDigest(unsigned char *digest, robj *o) {
o = getDecodedObject(o);
mixDigest(digest,o->ptr,sdslen(o->ptr));
decrRefCount(o);
}
/* This function computes the digest of a data structure stored in the
* object 'o'. It is the core of the DEBUG DIGEST command: when taking the
* digest of a whole dataset, we take the digest of the key and the value
* pair, and xor all those together.
*
* Note that this function does not reset the initial 'digest' passed, it
* will continue mixing this object digest to anything that was already
* present. */
void xorObjectDigest(redisDb *db, robj *keyobj, unsigned char *digest, robj *o) {
uint32_t aux = htonl(o->type);
mixDigest(digest,&aux,sizeof(aux));
long long expiretime = getExpire(db,keyobj);
char buf[128];
/* Save the key and associated value */
if (o->type == OBJ_STRING) {
mixStringObjectDigest(digest,o);
} else if (o->type == OBJ_LIST) {
listTypeIterator *li = listTypeInitIterator(o,0,LIST_TAIL);
listTypeEntry entry;
while(listTypeNext(li,&entry)) {
robj *eleobj = listTypeGet(&entry);
mixStringObjectDigest(digest,eleobj);
decrRefCount(eleobj);
}
listTypeReleaseIterator(li);
} else if (o->type == OBJ_SET) {
setTypeIterator *si = setTypeInitIterator(o);
sds sdsele;
while((sdsele = setTypeNextObject(si)) != NULL) {
xorDigest(digest,sdsele,sdslen(sdsele));
sdsfree(sdsele);
}
setTypeReleaseIterator(si);
} else if (o->type == OBJ_ZSET) {
unsigned char eledigest[20];
if (o->encoding == OBJ_ENCODING_ZIPLIST) {
unsigned char *zl = o->ptr;
unsigned char *eptr, *sptr;
unsigned char *vstr;
unsigned int vlen;
long long vll;
double score;
eptr = ziplistIndex(zl,0);
serverAssert(eptr != NULL);
sptr = ziplistNext(zl,eptr);
serverAssert(sptr != NULL);
while (eptr != NULL) {
serverAssert(ziplistGet(eptr,&vstr,&vlen,&vll));
score = zzlGetScore(sptr);
memset(eledigest,0,20);
if (vstr != NULL) {
mixDigest(eledigest,vstr,vlen);
} else {
ll2string(buf,sizeof(buf),vll);
mixDigest(eledigest,buf,strlen(buf));
}
snprintf(buf,sizeof(buf),"%.17g",score);
mixDigest(eledigest,buf,strlen(buf));
xorDigest(digest,eledigest,20);
zzlNext(zl,&eptr,&sptr);
}
} else if (o->encoding == OBJ_ENCODING_SKIPLIST) {
zset *zs = o->ptr;
dictIterator *di = dictGetIterator(zs->dict);
dictEntry *de;
while((de = dictNext(di)) != NULL) {
sds sdsele = dictGetKey(de);
double *score = dictGetVal(de);
snprintf(buf,sizeof(buf),"%.17g",*score);
memset(eledigest,0,20);
mixDigest(eledigest,sdsele,sdslen(sdsele));
mixDigest(eledigest,buf,strlen(buf));
xorDigest(digest,eledigest,20);
}
dictReleaseIterator(di);
} else {
serverPanic("Unknown sorted set encoding");
}
} else if (o->type == OBJ_HASH) {
hashTypeIterator *hi = hashTypeInitIterator(o);
while (hashTypeNext(hi) != C_ERR) {
unsigned char eledigest[20];
sds sdsele;
memset(eledigest,0,20);
sdsele = hashTypeCurrentObjectNewSds(hi,OBJ_HASH_KEY);
mixDigest(eledigest,sdsele,sdslen(sdsele));
sdsfree(sdsele);
sdsele = hashTypeCurrentObjectNewSds(hi,OBJ_HASH_VALUE);
mixDigest(eledigest,sdsele,sdslen(sdsele));
sdsfree(sdsele);
xorDigest(digest,eledigest,20);
}
hashTypeReleaseIterator(hi);
} else if (o->type == OBJ_STREAM) {
streamIterator si;
streamIteratorStart(&si,o->ptr,NULL,NULL,0);
streamID id;
int64_t numfields;
while(streamIteratorGetID(&si,&id,&numfields)) {
sds itemid = sdscatfmt(sdsempty(),"%U.%U",id.ms,id.seq);
mixDigest(digest,itemid,sdslen(itemid));
sdsfree(itemid);
while(numfields--) {
unsigned char *field, *value;
int64_t field_len, value_len;
streamIteratorGetField(&si,&field,&value,
&field_len,&value_len);
mixDigest(digest,field,field_len);
mixDigest(digest,value,value_len);
}
}
streamIteratorStop(&si);
} else if (o->type == OBJ_MODULE) {
RedisModuleDigest md;
moduleValue *mv = o->ptr;
moduleType *mt = mv->type;
moduleInitDigestContext(md);
if (mt->digest) {
mt->digest(&md,mv->value);
xorDigest(digest,md.x,sizeof(md.x));
}
} else {
serverPanic("Unknown object type");
}
/* If the key has an expire, add it to the mix */
if (expiretime != -1) xorDigest(digest,"!!expire!!",10);
}
/* Compute the dataset digest. Since keys, sets elements, hashes elements
* are not ordered, we use a trick: every aggregate digest is the xor
* of the digests of their elements. This way the order will not change
@@ -257,6 +114,7 @@ void xorObjectDigest(redisDb *db, robj *keyobj, unsigned char *digest, robj *o)
* a different digest. */
void computeDatasetDigest(unsigned char *final) {
unsigned char digest[20];
char buf[128];
dictIterator *di = NULL;
dictEntry *de;
int j;
@@ -279,6 +137,7 @@ void computeDatasetDigest(unsigned char *final) {
while((de = dictNext(di)) != NULL) {
sds key;
robj *keyobj, *o;
long long expiretime;
memset(digest,0,20); /* This key-val digest */
key = dictGetKey(de);
@@ -287,8 +146,134 @@ void computeDatasetDigest(unsigned char *final) {
mixDigest(digest,key,sdslen(key));
o = dictGetVal(de);
xorObjectDigest(db,keyobj,digest,o);
aux = htonl(o->type);
mixDigest(digest,&aux,sizeof(aux));
expiretime = getExpire(db,keyobj);
/* Save the key and associated value */
if (o->type == OBJ_STRING) {
mixObjectDigest(digest,o);
} else if (o->type == OBJ_LIST) {
listTypeIterator *li = listTypeInitIterator(o,0,LIST_TAIL);
listTypeEntry entry;
while(listTypeNext(li,&entry)) {
robj *eleobj = listTypeGet(&entry);
mixObjectDigest(digest,eleobj);
decrRefCount(eleobj);
}
listTypeReleaseIterator(li);
} else if (o->type == OBJ_SET) {
setTypeIterator *si = setTypeInitIterator(o);
sds sdsele;
while((sdsele = setTypeNextObject(si)) != NULL) {
xorDigest(digest,sdsele,sdslen(sdsele));
sdsfree(sdsele);
}
setTypeReleaseIterator(si);
} else if (o->type == OBJ_ZSET) {
unsigned char eledigest[20];
if (o->encoding == OBJ_ENCODING_ZIPLIST) {
unsigned char *zl = o->ptr;
unsigned char *eptr, *sptr;
unsigned char *vstr;
unsigned int vlen;
long long vll;
double score;
eptr = ziplistIndex(zl,0);
serverAssert(eptr != NULL);
sptr = ziplistNext(zl,eptr);
serverAssert(sptr != NULL);
while (eptr != NULL) {
serverAssert(ziplistGet(eptr,&vstr,&vlen,&vll));
score = zzlGetScore(sptr);
memset(eledigest,0,20);
if (vstr != NULL) {
mixDigest(eledigest,vstr,vlen);
} else {
ll2string(buf,sizeof(buf),vll);
mixDigest(eledigest,buf,strlen(buf));
}
snprintf(buf,sizeof(buf),"%.17g",score);
mixDigest(eledigest,buf,strlen(buf));
xorDigest(digest,eledigest,20);
zzlNext(zl,&eptr,&sptr);
}
} else if (o->encoding == OBJ_ENCODING_SKIPLIST) {
zset *zs = o->ptr;
dictIterator *di = dictGetIterator(zs->dict);
dictEntry *de;
while((de = dictNext(di)) != NULL) {
sds sdsele = dictGetKey(de);
double *score = dictGetVal(de);
snprintf(buf,sizeof(buf),"%.17g",*score);
memset(eledigest,0,20);
mixDigest(eledigest,sdsele,sdslen(sdsele));
mixDigest(eledigest,buf,strlen(buf));
xorDigest(digest,eledigest,20);
}
dictReleaseIterator(di);
} else {
serverPanic("Unknown sorted set encoding");
}
} else if (o->type == OBJ_HASH) {
hashTypeIterator *hi = hashTypeInitIterator(o);
while (hashTypeNext(hi) != C_ERR) {
unsigned char eledigest[20];
sds sdsele;
memset(eledigest,0,20);
sdsele = hashTypeCurrentObjectNewSds(hi,OBJ_HASH_KEY);
mixDigest(eledigest,sdsele,sdslen(sdsele));
sdsfree(sdsele);
sdsele = hashTypeCurrentObjectNewSds(hi,OBJ_HASH_VALUE);
mixDigest(eledigest,sdsele,sdslen(sdsele));
sdsfree(sdsele);
xorDigest(digest,eledigest,20);
}
hashTypeReleaseIterator(hi);
} else if (o->type == OBJ_STREAM) {
streamIterator si;
streamIteratorStart(&si,o->ptr,NULL,NULL,0);
streamID id;
int64_t numfields;
while(streamIteratorGetID(&si,&id,&numfields)) {
sds itemid = sdscatfmt(sdsempty(),"%U.%U",id.ms,id.seq);
mixDigest(digest,itemid,sdslen(itemid));
sdsfree(itemid);
while(numfields--) {
unsigned char *field, *value;
int64_t field_len, value_len;
streamIteratorGetField(&si,&field,&value,
&field_len,&value_len);
mixDigest(digest,field,field_len);
mixDigest(digest,value,value_len);
}
}
streamIteratorStop(&si);
} else if (o->type == OBJ_MODULE) {
RedisModuleDigest md;
moduleValue *mv = o->ptr;
moduleType *mt = mv->type;
moduleInitDigestContext(md);
if (mt->digest) {
mt->digest(&md,mv->value);
xorDigest(digest,md.x,sizeof(md.x));
}
} else {
serverPanic("Unknown object type");
}
/* If the key has an expire, add it to the mix */
if (expiretime != -1) xorDigest(digest,"!!expire!!",10);
/* We can finally xor the key-val digest to the final digest */
xorDigest(final,digest,20);
decrRefCount(keyobj);
@@ -304,7 +289,6 @@ void debugCommand(client *c) {
"CHANGE-REPL-ID -- Change the replication IDs of the instance. Dangerous, should be used only for testing the replication subsystem.",
"CRASH-AND-RECOVER <milliseconds> -- Hard crash and restart after <milliseconds> delay.",
"DIGEST -- Output a hex signature representing the current DB content.",
"DIGEST-VALUE <key-1> ... <key-N>-- Output a hex signature of the values of all the specified keys.",
"ERROR <string> -- Return a Redis protocol error with <string> as message. Useful for clients unit tests to simulate Redis errors.",
"LOG <message> -- write message to the server log.",
"HTSTATS <dbid> -- Return hash table statistics of the specified Redis database.",
@@ -322,7 +306,6 @@ void debugCommand(client *c) {
"SLEEP <seconds> -- Stop the server for <seconds>. Decimals allowed.",
"STRUCTSIZE -- Return the size of different Redis core C structures.",
"ZIPLIST <key> -- Show low level info about the ziplist encoding.",
"STRINGMATCH-TEST -- Run a fuzz tester against the stringmatchlen() function.",
NULL
};
addReplyHelp(c, help);
@@ -349,6 +332,7 @@ NULL
zfree(ptr);
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"assert")) {
if (c->argc >= 3) c->argv[2] = tryObjectEncoding(c->argv[2]);
serverAssertWithInfo(c,c->argv[0],1 == 2);
} else if (!strcasecmp(c->argv[1]->ptr,"log") && c->argc == 3) {
serverLog(LL_WARNING, "DEBUG LOG: %s", (char*)c->argv[2]->ptr);
@@ -507,28 +491,15 @@ NULL
}
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"digest") && c->argc == 2) {
/* DEBUG DIGEST (form without keys specified) */
unsigned char digest[20];
sds d = sdsempty();
int j;
computeDatasetDigest(digest);
for (int i = 0; i < 20; i++) d = sdscatprintf(d, "%02x",digest[i]);
for (j = 0; j < 20; j++)
d = sdscatprintf(d, "%02x",digest[j]);
addReplyStatus(c,d);
sdsfree(d);
} else if (!strcasecmp(c->argv[1]->ptr,"digest-value") && c->argc >= 2) {
/* DEBUG DIGEST-VALUE key key key ... key. */
addReplyMultiBulkLen(c,c->argc-2);
for (int j = 2; j < c->argc; j++) {
unsigned char digest[20];
memset(digest,0,20); /* Start with a clean result */
robj *o = lookupKeyReadWithFlags(c->db,c->argv[j],LOOKUP_NOTOUCH);
if (o) xorObjectDigest(c->db,c->argv[j],digest,o);
sds d = sdsempty();
for (int i = 0; i < 20; i++) d = sdscatprintf(d, "%02x",digest[i]);
addReplyStatus(c,d);
sdsfree(d);
}
} else if (!strcasecmp(c->argv[1]->ptr,"sleep") && c->argc == 3) {
double dtime = strtod(c->argv[2]->ptr,NULL);
long long utime = dtime*1000000;
@@ -620,10 +591,6 @@ NULL
changeReplicationId();
clearReplicationId2();
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"stringmatch-test") && c->argc == 2)
{
stringmatchlen_fuzz_test();
addReplyStatus(c,"Apparently Redis did not crash: test passed");
} else {
addReplySubcommandSyntaxError(c);
return;
@@ -762,22 +729,6 @@ static void *getMcontextEip(ucontext_t *uc) {
#elif defined(__aarch64__) /* Linux AArch64 */
return (void*) uc->uc_mcontext.pc;
#endif
#elif defined(__FreeBSD__)
/* FreeBSD */
#if defined(__i386__)
return (void*) uc->uc_mcontext.mc_eip;
#elif defined(__x86_64__)
return (void*) uc->uc_mcontext.mc_rip;
#endif
#elif defined(__OpenBSD__)
/* OpenBSD */
#if defined(__i386__)
return (void*) uc->sc_eip;
#elif defined(__x86_64__)
return (void*) uc->sc_rip;
#endif
#elif defined(__DragonFly__)
return (void*) uc->uc_mcontext.mc_rip;
#else
return NULL;
#endif
@@ -919,145 +870,6 @@ void logRegisters(ucontext_t *uc) {
);
logStackContent((void**)uc->uc_mcontext.gregs[15]);
#endif
#elif defined(__FreeBSD__)
#if defined(__x86_64__)
serverLog(LL_WARNING,
"\n"
"RAX:%016lx RBX:%016lx\nRCX:%016lx RDX:%016lx\n"
"RDI:%016lx RSI:%016lx\nRBP:%016lx RSP:%016lx\n"
"R8 :%016lx R9 :%016lx\nR10:%016lx R11:%016lx\n"
"R12:%016lx R13:%016lx\nR14:%016lx R15:%016lx\n"
"RIP:%016lx EFL:%016lx\nCSGSFS:%016lx",
(unsigned long) uc->uc_mcontext.mc_rax,
(unsigned long) uc->uc_mcontext.mc_rbx,
(unsigned long) uc->uc_mcontext.mc_rcx,
(unsigned long) uc->uc_mcontext.mc_rdx,
(unsigned long) uc->uc_mcontext.mc_rdi,
(unsigned long) uc->uc_mcontext.mc_rsi,
(unsigned long) uc->uc_mcontext.mc_rbp,
(unsigned long) uc->uc_mcontext.mc_rsp,
(unsigned long) uc->uc_mcontext.mc_r8,
(unsigned long) uc->uc_mcontext.mc_r9,
(unsigned long) uc->uc_mcontext.mc_r10,
(unsigned long) uc->uc_mcontext.mc_r11,
(unsigned long) uc->uc_mcontext.mc_r12,
(unsigned long) uc->uc_mcontext.mc_r13,
(unsigned long) uc->uc_mcontext.mc_r14,
(unsigned long) uc->uc_mcontext.mc_r15,
(unsigned long) uc->uc_mcontext.mc_rip,
(unsigned long) uc->uc_mcontext.mc_rflags,
(unsigned long) uc->uc_mcontext.mc_cs
);
logStackContent((void**)uc->uc_mcontext.mc_rsp);
#elif defined(__i386__)
serverLog(LL_WARNING,
"\n"
"EAX:%08lx EBX:%08lx ECX:%08lx EDX:%08lx\n"
"EDI:%08lx ESI:%08lx EBP:%08lx ESP:%08lx\n"
"SS :%08lx EFL:%08lx EIP:%08lx CS:%08lx\n"
"DS :%08lx ES :%08lx FS :%08lx GS:%08lx",
(unsigned long) uc->uc_mcontext.mc_eax,
(unsigned long) uc->uc_mcontext.mc_ebx,
(unsigned long) uc->uc_mcontext.mc_ebx,
(unsigned long) uc->uc_mcontext.mc_edx,
(unsigned long) uc->uc_mcontext.mc_edi,
(unsigned long) uc->uc_mcontext.mc_esi,
(unsigned long) uc->uc_mcontext.mc_ebp,
(unsigned long) uc->uc_mcontext.mc_esp,
(unsigned long) uc->uc_mcontext.mc_ss,
(unsigned long) uc->uc_mcontext.mc_eflags,
(unsigned long) uc->uc_mcontext.mc_eip,
(unsigned long) uc->uc_mcontext.mc_cs,
(unsigned long) uc->uc_mcontext.mc_es,
(unsigned long) uc->uc_mcontext.mc_fs,
(unsigned long) uc->uc_mcontext.mc_gs
);
logStackContent((void**)uc->uc_mcontext.mc_esp);
#endif
#elif defined(__OpenBSD__)
#if defined(__x86_64__)
serverLog(LL_WARNING,
"\n"
"RAX:%016lx RBX:%016lx\nRCX:%016lx RDX:%016lx\n"
"RDI:%016lx RSI:%016lx\nRBP:%016lx RSP:%016lx\n"
"R8 :%016lx R9 :%016lx\nR10:%016lx R11:%016lx\n"
"R12:%016lx R13:%016lx\nR14:%016lx R15:%016lx\n"
"RIP:%016lx EFL:%016lx\nCSGSFS:%016lx",
(unsigned long) uc->sc_rax,
(unsigned long) uc->sc_rbx,
(unsigned long) uc->sc_rcx,
(unsigned long) uc->sc_rdx,
(unsigned long) uc->sc_rdi,
(unsigned long) uc->sc_rsi,
(unsigned long) uc->sc_rbp,
(unsigned long) uc->sc_rsp,
(unsigned long) uc->sc_r8,
(unsigned long) uc->sc_r9,
(unsigned long) uc->sc_r10,
(unsigned long) uc->sc_r11,
(unsigned long) uc->sc_r12,
(unsigned long) uc->sc_r13,
(unsigned long) uc->sc_r14,
(unsigned long) uc->sc_r15,
(unsigned long) uc->sc_rip,
(unsigned long) uc->sc_rflags,
(unsigned long) uc->sc_cs
);
logStackContent((void**)uc->sc_rsp);
#elif defined(__i386__)
serverLog(LL_WARNING,
"\n"
"EAX:%08lx EBX:%08lx ECX:%08lx EDX:%08lx\n"
"EDI:%08lx ESI:%08lx EBP:%08lx ESP:%08lx\n"
"SS :%08lx EFL:%08lx EIP:%08lx CS:%08lx\n"
"DS :%08lx ES :%08lx FS :%08lx GS:%08lx",
(unsigned long) uc->sc_eax,
(unsigned long) uc->sc_ebx,
(unsigned long) uc->sc_ebx,
(unsigned long) uc->sc_edx,
(unsigned long) uc->sc_edi,
(unsigned long) uc->sc_esi,
(unsigned long) uc->sc_ebp,
(unsigned long) uc->sc_esp,
(unsigned long) uc->sc_ss,
(unsigned long) uc->sc_eflags,
(unsigned long) uc->sc_eip,
(unsigned long) uc->sc_cs,
(unsigned long) uc->sc_es,
(unsigned long) uc->sc_fs,
(unsigned long) uc->sc_gs
);
logStackContent((void**)uc->sc_esp);
#endif
#elif defined(__DragonFly__)
serverLog(LL_WARNING,
"\n"
"RAX:%016lx RBX:%016lx\nRCX:%016lx RDX:%016lx\n"
"RDI:%016lx RSI:%016lx\nRBP:%016lx RSP:%016lx\n"
"R8 :%016lx R9 :%016lx\nR10:%016lx R11:%016lx\n"
"R12:%016lx R13:%016lx\nR14:%016lx R15:%016lx\n"
"RIP:%016lx EFL:%016lx\nCSGSFS:%016lx",
(unsigned long) uc->uc_mcontext.mc_rax,
(unsigned long) uc->uc_mcontext.mc_rbx,
(unsigned long) uc->uc_mcontext.mc_rcx,
(unsigned long) uc->uc_mcontext.mc_rdx,
(unsigned long) uc->uc_mcontext.mc_rdi,
(unsigned long) uc->uc_mcontext.mc_rsi,
(unsigned long) uc->uc_mcontext.mc_rbp,
(unsigned long) uc->uc_mcontext.mc_rsp,
(unsigned long) uc->uc_mcontext.mc_r8,
(unsigned long) uc->uc_mcontext.mc_r9,
(unsigned long) uc->uc_mcontext.mc_r10,
(unsigned long) uc->uc_mcontext.mc_r11,
(unsigned long) uc->uc_mcontext.mc_r12,
(unsigned long) uc->uc_mcontext.mc_r13,
(unsigned long) uc->uc_mcontext.mc_r14,
(unsigned long) uc->uc_mcontext.mc_r15,
(unsigned long) uc->uc_mcontext.mc_rip,
(unsigned long) uc->uc_mcontext.mc_rflags,
(unsigned long) uc->uc_mcontext.mc_cs
);
logStackContent((void**)uc->uc_mcontext.mc_rsp);
#else
serverLog(LL_WARNING,
" Dumping of registers not supported for this OS/arch");
@@ -1377,8 +1189,6 @@ void serverLogHexDump(int level, char *descr, void *value, size_t len) {
void watchdogSignalHandler(int sig, siginfo_t *info, void *secret) {
#ifdef HAVE_BACKTRACE
ucontext_t *uc = (ucontext_t*) secret;
#else
(void)secret;
#endif
UNUSED(info);
UNUSED(sig);
+3 -14
View File
@@ -364,7 +364,7 @@ size_t freeMemoryGetNotCountedMemory(void) {
}
}
if (server.aof_state != AOF_OFF) {
overhead += sdsalloc(server.aof_buf)+aofRewriteBufferSize();
overhead += sdslen(server.aof_buf)+aofRewriteBufferSize();
}
return overhead;
}
@@ -444,8 +444,8 @@ int getMaxmemoryState(size_t *total, size_t *logical, size_t *tofree, float *lev
* Otehrwise if we are over the memory limit, but not enough memory
* was freed to return back under the limit, the function returns C_ERR. */
int freeMemoryIfNeeded(void) {
/* By default replicas should ignore maxmemory
* and just be masters exact copies. */
/* By default slaves should ignore maxmemory and just be masters excat
* copies. */
if (server.masterhost && server.repl_slave_ignore_maxmemory) return C_OK;
size_t mem_reported, mem_tofree, mem_freed;
@@ -622,14 +622,3 @@ cant_free:
return C_ERR;
}
/* This is a wrapper for freeMemoryIfNeeded() that only really calls the
* function if right now there are the conditions to do so safely:
*
* - There must be no script in timeout condition.
* - Nor we are loading data right now.
*
*/
int freeMemoryIfNeededAndSafe(void) {
if (server.lua_timedout || server.loading) return C_OK;
return freeMemoryIfNeeded();
}
+1 -1
View File
@@ -123,7 +123,7 @@ static uint8_t intsetSearch(intset *is, int64_t value, uint32_t *pos) {
} else {
/* Check for the case where we know we cannot find the value,
* but do know the insert position. */
if (value > _intsetGet(is,max)) {
if (value > _intsetGet(is,intrev32ifbe(is->length)-1)) {
if (pos) *pos = intrev32ifbe(is->length);
return 0;
} else if (value < _intsetGet(is,0)) {
+2 -14
View File
@@ -562,21 +562,11 @@ sds latencyCommandGenSparkeline(char *event, struct latencyTimeSeries *ts) {
*
* LATENCY HISTORY: return time-latency samples for the specified event.
* LATENCY LATEST: return the latest latency for all the events classes.
* LATENCY DOCTOR: returns a human readable analysis of instance latency.
* LATENCY DOCTOR: returns an human readable analysis of instance latency.
* LATENCY GRAPH: provide an ASCII graph of the latency of the specified event.
* LATENCY RESET: reset data of a specified event or all the data if no event provided.
*/
void latencyCommand(client *c) {
const char *help[] = {
"DOCTOR -- Returns a human readable latency analysis report.",
"GRAPH <event> -- Returns an ASCII latency graph for the event class.",
"HISTORY <event> -- Returns time-latency samples for the event class.",
"LATEST -- Returns the latest latency samples for all events.",
"RESET [event ...] -- Resets latency data of one or more event classes.",
" (default: reset all data for all event classes)",
"HELP -- Prints this help.",
NULL
};
struct latencyTimeSeries *ts;
if (!strcasecmp(c->argv[1]->ptr,"history") && c->argc == 3) {
@@ -621,10 +611,8 @@ NULL
resets += latencyResetEvent(c->argv[j]->ptr);
addReplyLongLong(c,resets);
}
} else if (!strcasecmp(c->argv[1]->ptr,"help") && c->argc >= 2) {
addReplyHelp(c, help);
} else {
addReplySubcommandSyntaxError(c);
addReply(c,shared.syntaxerr);
}
return;
+4 -146
View File
@@ -47,21 +47,9 @@ struct RedisModule {
int ver; /* Module version. We use just progressive integers. */
int apiver; /* Module API version as requested during initialization.*/
list *types; /* Module data types. */
list *usedby; /* List of modules using APIs from this one. */
list *using; /* List of modules we use some APIs of. */
};
typedef struct RedisModule RedisModule;
/* This represents a shared API. Shared APIs will be used to populate
* the server.sharedapi dictionary, mapping names of APIs exported by
* modules for other modules to use, to their structure specifying the
* function pointer that can be called. */
struct RedisModuleSharedAPI {
void *func;
RedisModule *module;
};
typedef struct RedisModuleSharedAPI RedisModuleSharedAPI;
static dict *modules; /* Hash table of modules. SDS -> RedisModule ptr.*/
/* Entries in the context->amqueue array, representing objects to free
@@ -712,8 +700,6 @@ void RM_SetModuleAttribs(RedisModuleCtx *ctx, const char *name, int ver, int api
module->ver = ver;
module->apiver = apiver;
module->types = listCreate();
module->usedby = listCreate();
module->using = listCreate();
ctx->module = module;
}
@@ -4629,121 +4615,6 @@ void RM_GetRandomHexChars(char *dst, size_t len) {
getRandomHexChars(dst,len);
}
/* --------------------------------------------------------------------------
* Modules API exporting / importing
* -------------------------------------------------------------------------- */
/* This function is called by a module in order to export some API with a
* given name. Other modules will be able to use this API by calling the
* symmetrical function RM_GetSharedAPI() and casting the return value to
* the right function pointer.
*
* The function will return REDISMODULE_OK if the name is not already taken,
* otherwise REDISMODULE_ERR will be returned and no operation will be
* performed.
*
* IMPORTANT: the apiname argument should be a string literal with static
* lifetime. The API relies on the fact that it will always be valid in
* the future. */
int RM_ExportSharedAPI(RedisModuleCtx *ctx, const char *apiname, void *func) {
RedisModuleSharedAPI *sapi = zmalloc(sizeof(*sapi));
sapi->module = ctx->module;
sapi->func = func;
if (dictAdd(server.sharedapi, (char*)apiname, sapi) != DICT_OK) {
zfree(sapi);
return REDISMODULE_ERR;
}
return REDISMODULE_OK;
}
/* Request an exported API pointer. The return value is just a void pointer
* that the caller of this function will be required to cast to the right
* function pointer, so this is a private contract between modules.
*
* If the requested API is not available then NULL is returned. Because
* modules can be loaded at different times with different order, this
* function calls should be put inside some module generic API registering
* step, that is called every time a module attempts to execute a
* command that requires external APIs: if some API cannot be resolved, the
* command should return an error.
*
* Here is an exmaple:
*
* int ... myCommandImplementation() {
* if (getExternalAPIs() == 0) {
* reply with an error here if we cannot have the APIs
* }
* // Use the API:
* myFunctionPointer(foo);
* }
*
* And the function registerAPI() is:
*
* int getExternalAPIs(void) {
* static int api_loaded = 0;
* if (api_loaded != 0) return 1; // APIs already resolved.
*
* myFunctionPointer = RedisModule_GetOtherModuleAPI("...");
* if (myFunctionPointer == NULL) return 0;
*
* return 1;
* }
*/
void *RM_GetSharedAPI(RedisModuleCtx *ctx, const char *apiname) {
dictEntry *de = dictFind(server.sharedapi, apiname);
if (de == NULL) return NULL;
RedisModuleSharedAPI *sapi = dictGetVal(de);
if (listSearchKey(sapi->module->usedby,ctx->module) == NULL) {
listAddNodeTail(sapi->module->usedby,ctx->module);
listAddNodeTail(ctx->module->using,sapi->module);
}
return sapi->func;
}
/* Remove all the APIs registered by the specified module. Usually you
* want this when the module is going to be unloaded. This function
* assumes that's caller responsibility to make sure the APIs are not
* used by other modules.
*
* The number of unregistered APIs is returned. */
int moduleUnregisterSharedAPI(RedisModule *module) {
int count = 0;
dictIterator *di = dictGetSafeIterator(server.sharedapi);
dictEntry *de;
while ((de = dictNext(di)) != NULL) {
const char *apiname = dictGetKey(de);
RedisModuleSharedAPI *sapi = dictGetVal(de);
if (sapi->module == module) {
dictDelete(server.sharedapi,apiname);
zfree(sapi);
count++;
}
}
dictReleaseIterator(di);
return count;
}
/* Remove the specified module as an user of APIs of ever other module.
* This is usually called when a module is unloaded.
*
* Returns the number of modules this module was using APIs from. */
int moduleUnregisterUsedAPI(RedisModule *module) {
listIter li;
listNode *ln;
int count = 0;
listRewind(module->using,&li);
while((ln = listNext(&li))) {
RedisModule *used = ln->value;
listNode *ln = listSearchKey(used->usedby,module);
if (ln) {
listDelNode(module->using,ln);
count++;
}
}
return count;
}
/* --------------------------------------------------------------------------
* Modules API internals
* -------------------------------------------------------------------------- */
@@ -4887,8 +4758,6 @@ int moduleLoad(const char *path, void **module_argv, int module_argc) {
if (onload((void*)&ctx,module_argv,module_argc) == REDISMODULE_ERR) {
if (ctx.module) {
moduleUnregisterCommands(ctx.module);
moduleUnregisterSharedAPI(ctx.module);
moduleUnregisterUsedAPI(ctx.module);
moduleFreeModuleStructure(ctx.module);
}
dlclose(handle);
@@ -4918,17 +4787,14 @@ int moduleUnload(sds name) {
if (module == NULL) {
errno = ENOENT;
return REDISMODULE_ERR;
} else if (listLength(module->types)) {
}
if (listLength(module->types)) {
errno = EBUSY;
return REDISMODULE_ERR;
} else if (listLength(module->usedby)) {
errno = EPERM;
return REDISMODULE_ERR;
}
moduleUnregisterCommands(module);
moduleUnregisterSharedAPI(module);
moduleUnregisterUsedAPI(module);
/* Remove any notification subscribers this module might have */
moduleUnsubscribeNotifications(module);
@@ -4990,12 +4856,7 @@ NULL
errmsg = "no such module with that name";
break;
case EBUSY:
errmsg = "the module exports one or more module-side data "
"types, can't unload";
break;
case EPERM:
errmsg = "the module exports APIs used by other modules. "
"Please unload them first and try again";
errmsg = "the module exports one or more module-side data types, can't unload";
break;
default:
errmsg = "operation not possible.";
@@ -5033,7 +4894,6 @@ size_t moduleCount(void) {
* file so that's easy to seek it to add new entries. */
void moduleRegisterCoreAPI(void) {
server.moduleapi = dictCreate(&moduleAPIDictType,NULL);
server.sharedapi = dictCreate(&moduleAPIDictType,NULL);
REGISTER_API(Alloc);
REGISTER_API(Calloc);
REGISTER_API(Realloc);
@@ -5184,6 +5044,4 @@ void moduleRegisterCoreAPI(void) {
REGISTER_API(DictPrev);
REGISTER_API(DictCompareC);
REGISTER_API(DictCompare);
REGISTER_API(ExportSharedAPI);
REGISTER_API(GetSharedAPI);
}
-17
View File
@@ -35,7 +35,6 @@
void initClientMultiState(client *c) {
c->mstate.commands = NULL;
c->mstate.count = 0;
c->mstate.cmd_flags = 0;
}
/* Release all the resources associated with MULTI/EXEC state */
@@ -68,7 +67,6 @@ void queueMultiCommand(client *c) {
for (j = 0; j < c->argc; j++)
incrRefCount(mc->argv[j]);
c->mstate.count++;
c->mstate.cmd_flags |= c->cmd->flags;
}
void discardTransaction(client *c) {
@@ -139,21 +137,6 @@ void execCommand(client *c) {
goto handle_monitor;
}
/* If there are write commands inside the transaction, and this is a read
* only slave, we want to send an error. This happens when the transaction
* was initiated when the instance was a master or a writable replica and
* then the configuration changed (for example instance was turned into
* a replica). */
if (!server.loading && server.masterhost && server.repl_slave_ro &&
!(c->flags & CLIENT_MASTER) && c->mstate.cmd_flags & CMD_WRITE)
{
addReplyError(c,
"Transaction contains write commands but instance "
"is now a read-only replica. EXEC aborted.");
discardTransaction(c);
goto handle_monitor;
}
/* Exec all the queued commands */
unwatchAllKeys(c); /* Unwatch ASAP otherwise we'll waste CPU cycles */
orig_argv = c->argv;
+8 -3
View File
@@ -365,13 +365,19 @@ void addReplyErrorLength(client *c, const char *s, size_t len) {
* Where the master must propagate the first change even if the second
* will produce an error. However it is useful to log such events since
* they are rare and may hint at errors in a script or a bug in Redis. */
if (c->flags & (CLIENT_MASTER|CLIENT_SLAVE) && !(c->flags & CLIENT_MONITOR)) {
if (c->flags & (CLIENT_MASTER|CLIENT_SLAVE)) {
char* to = c->flags & CLIENT_MASTER? "master": "replica";
char* from = c->flags & CLIENT_MASTER? "replica": "master";
char *cmdname = c->lastcmd ? c->lastcmd->name : "<unknown>";
serverLog(LL_WARNING,"== CRITICAL == This %s is sending an error "
"to its %s: '%s' after processing the command "
"'%s'", from, to, s, cmdname);
/* Here we want to panic because when a master is sending an
* error to some slave in the context of replication, this can
* only create some kind of offset or data desynchronization. Better
* to catch it ASAP and crash instead of continuing. */
if (c->flags & CLIENT_SLAVE)
serverPanic("Continuing is unsafe: replication protocol violation.");
}
}
@@ -1464,7 +1470,7 @@ void processInputBuffer(client *c) {
}
/* Trim to pos */
if (server.current_client != NULL && c->qb_pos) {
if (c->qb_pos) {
sdsrange(c->querybuf,c->qb_pos,-1);
c->qb_pos = 0;
}
@@ -2103,7 +2109,6 @@ int checkClientOutputBufferLimits(client *c) {
* called from contexts where the client can't be freed safely, i.e. from the
* lower level functions pushing data inside the client output buffers. */
void asyncCloseClientOnOutputBufferLimitReached(client *c) {
if (c->fd == -1) return; /* It is unsafe to free fake clients. */
serverAssert(c->reply_bytes < SIZE_MAX-(1024*64));
if (c->reply_bytes == 0 || c->flags & CLIENT_CLOSE_ASAP) return;
if (checkClientOutputBufferLimits(c)) {
+22 -21
View File
@@ -185,7 +185,7 @@ robj *createStringObjectFromLongDouble(long double value, int humanfriendly) {
/* Duplicate a string object, with the guarantee that the returned object
* has the same encoding as the original one.
*
* This function also guarantees that duplicating a small integer object
* This function also guarantees that duplicating a small integere object
* (or a string object that contains a representation of a small integer)
* will always result in a fresh object that is unshared (refcount == 1).
*
@@ -1011,7 +1011,7 @@ struct redisMemOverhead *getMemoryOverheadData(void) {
mem = 0;
if (server.aof_state != AOF_OFF) {
mem += sdsalloc(server.aof_buf);
mem += sdslen(server.aof_buf);
mem += aofRewriteBufferSize();
}
mh->aof_buffer = mem;
@@ -1285,18 +1285,9 @@ NULL
*
* Usage: MEMORY usage <key> */
void memoryCommand(client *c) {
if (!strcasecmp(c->argv[1]->ptr,"help") && c->argc == 2) {
const char *help[] = {
"DOCTOR - Return memory problems reports.",
"MALLOC-STATS -- Return internal statistics report from the memory allocator.",
"PURGE -- Attempt to purge dirty pages for reclamation by the allocator.",
"STATS -- Return information about the memory usage of the server.",
"USAGE <key> [SAMPLES <count>] -- Return memory in bytes used by <key> and its value. Nested values are sampled up to <count> times (default: 5).",
NULL
};
addReplyHelp(c, help);
} else if (!strcasecmp(c->argv[1]->ptr,"usage") && c->argc >= 3) {
dictEntry *de;
robj *o;
if (!strcasecmp(c->argv[1]->ptr,"usage") && c->argc >= 3) {
long long samples = OBJ_COMPUTE_SIZE_DEF_SAMPLES;
for (int j = 3; j < c->argc; j++) {
if (!strcasecmp(c->argv[j]->ptr,"samples") &&
@@ -1315,12 +1306,10 @@ NULL
return;
}
}
if ((de = dictFind(c->db->dict,c->argv[2]->ptr)) == NULL) {
addReply(c, shared.nullbulk);
return;
}
size_t usage = objectComputeSize(dictGetVal(de),samples);
usage += sdsAllocSize(dictGetKey(de));
if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.nullbulk))
== NULL) return;
size_t usage = objectComputeSize(o,samples);
usage += sdsAllocSize(c->argv[2]->ptr);
usage += sizeof(dictEntry);
addReplyLongLong(c,usage);
} else if (!strcasecmp(c->argv[1]->ptr,"stats") && c->argc == 2) {
@@ -1445,7 +1434,19 @@ NULL
addReply(c, shared.ok);
/* Nothing to do for other allocators. */
#endif
} else if (!strcasecmp(c->argv[1]->ptr,"help") && c->argc == 2) {
addReplyMultiBulkLen(c,5);
addReplyBulkCString(c,
"MEMORY DOCTOR - Outputs memory problems report");
addReplyBulkCString(c,
"MEMORY USAGE <key> [SAMPLES <count>] - Estimate memory usage of key");
addReplyBulkCString(c,
"MEMORY STATS - Show memory usage details");
addReplyBulkCString(c,
"MEMORY PURGE - Ask the allocator to release memory");
addReplyBulkCString(c,
"MEMORY MALLOC-STATS - Show allocator internal stats");
} else {
addReplyErrorFormat(c, "Unknown subcommand or wrong number of arguments for '%s'. Try MEMORY HELP", (char*)c->argv[1]->ptr);
addReplyError(c,"Syntax error. Try MEMORY HELP");
}
}
+1 -1
View File
@@ -40,7 +40,7 @@
* container: 2 bits, NONE=1, ZIPLIST=2.
* recompress: 1 bit, bool, true if node is temporarry decompressed for usage.
* attempted_compress: 1 bit, boolean, used for verifying during testing.
* extra: 10 bits, free for future use; pads out the remainder of 32 bits */
* extra: 12 bits, free for future use; pads out the remainder of 32 bits */
typedef struct quicklistNode {
struct quicklistNode *prev;
struct quicklistNode *next;
+1 -1
View File
@@ -1406,7 +1406,7 @@ int raxIteratorNextStep(raxIterator *it, int noup) {
}
}
/* Seek the greatest key in the subtree at the current node. Return 0 on
/* Seek the grestest key in the subtree at the current node. Return 0 on
* out of memory, otherwise 1. This is an helper function for different
* iteration functions below. */
int raxSeekGreatest(raxIterator *it) {
-3
View File
@@ -1645,9 +1645,6 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
* node: the entries inside the listpack itself are delta-encoded
* relatively to this ID. */
sds nodekey = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL);
if (nodekey == NULL) {
rdbExitReportCorruptRDB("Stream master ID loading failed: invalid encoding or I/O error.");
}
if (sdslen(nodekey) != sizeof(streamID)) {
rdbExitReportCorruptRDB("Stream node key entry is not the "
"size of a stream ID");
+106 -452
View File
@@ -67,7 +67,6 @@
#define REDIS_CLI_HISTFILE_DEFAULT ".rediscli_history"
#define REDIS_CLI_RCFILE_ENV "REDISCLI_RCFILE"
#define REDIS_CLI_RCFILE_DEFAULT ".redisclirc"
#define REDIS_CLI_AUTH_ENV "REDISCLI_AUTH"
#define CLUSTER_MANAGER_SLOTS 16384
#define CLUSTER_MANAGER_MIGRATE_TIMEOUT 60000
@@ -117,7 +116,6 @@
#define CLUSTER_MANAGER_CMD_FLAG_REPLACE 1 << 6
#define CLUSTER_MANAGER_CMD_FLAG_COPY 1 << 7
#define CLUSTER_MANAGER_CMD_FLAG_COLOR 1 << 8
#define CLUSTER_MANAGER_CMD_FLAG_CHECK_OWNERS 1 << 9
#define CLUSTER_MANAGER_OPT_GETFRIENDS 1 << 0
#define CLUSTER_MANAGER_OPT_COLD 1 << 1
@@ -1379,9 +1377,6 @@ static int parseOptions(int argc, char **argv) {
} else if (!strcmp(argv[i],"--cluster-use-empty-masters")) {
config.cluster_manager_command.flags |=
CLUSTER_MANAGER_CMD_FLAG_EMPTYMASTER;
} else if (!strcmp(argv[i],"--cluster-search-multiple-owners")) {
config.cluster_manager_command.flags |=
CLUSTER_MANAGER_CMD_FLAG_CHECK_OWNERS;
} else if (!strcmp(argv[i],"-v") || !strcmp(argv[i], "--version")) {
sds version = cliVersion();
printf("redis-cli %s\n", version);
@@ -1424,14 +1419,6 @@ static int parseOptions(int argc, char **argv) {
return i;
}
static void parseEnv() {
/* Set auth from env, but do not overwrite CLI arguments if passed */
char *auth = getenv(REDIS_CLI_AUTH_ENV);
if (auth != NULL && config.auth == NULL) {
config.auth = auth;
}
}
static sds readArgFromStdin(void) {
char buf[1024];
sds arg = sdsempty();
@@ -1459,9 +1446,6 @@ static void usage(void) {
" -p <port> Server port (default: 6379).\n"
" -s <socket> Server socket (overrides hostname and port).\n"
" -a <password> Password to use when connecting to the server.\n"
" You can also use the " REDIS_CLI_AUTH_ENV " environment\n"
" variable to pass this password more safely\n"
" (if both are used, this argument takes predecence).\n"
" -u <uri> Server URI.\n"
" -r <repeat> Execute specified command N times.\n"
" -i <interval> When -r is used, waits <interval> seconds per command.\n"
@@ -1850,7 +1834,7 @@ static int evalMode(int argc, char **argv) {
if (eval_ldb) {
if (!config.eval_ldb) {
/* If the debugging session ended immediately, there was an
* error compiling the script. Show it and they don't enter
* error compiling the script. Show it and don't enter
* the REPL at all. */
printf("Eval debugging session can't start:\n");
cliReadReply(0);
@@ -1933,7 +1917,6 @@ static dictType clusterManagerDictType = {
};
typedef int clusterManagerCommandProc(int argc, char **argv);
typedef int (*clusterManagerOnReplyError)(redisReply *reply, int bulk_idx);
/* Cluster Manager helper functions */
@@ -1995,17 +1978,14 @@ typedef struct clusterManagerCommandDef {
clusterManagerCommandDef clusterManagerCommands[] = {
{"create", clusterManagerCommandCreate, -2, "host1:port1 ... hostN:portN",
"replicas <arg>"},
{"check", clusterManagerCommandCheck, -1, "host:port",
"search-multiple-owners"},
{"check", clusterManagerCommandCheck, -1, "host:port", NULL},
{"info", clusterManagerCommandInfo, -1, "host:port", NULL},
{"fix", clusterManagerCommandFix, -1, "host:port",
"search-multiple-owners"},
{"fix", clusterManagerCommandFix, -1, "host:port", NULL},
{"reshard", clusterManagerCommandReshard, -1, "host:port",
"from <arg>,to <arg>,slots <arg>,yes,timeout <arg>,pipeline <arg>,"
"replace"},
"from <arg>,to <arg>,slots <arg>,yes,timeout <arg>,pipeline <arg>"},
{"rebalance", clusterManagerCommandRebalance, -1, "host:port",
"weight <node1=w1...nodeN=wN>,use-empty-masters,"
"timeout <arg>,simulate,pipeline <arg>,threshold <arg>,replace"},
"timeout <arg>,simulate,pipeline <arg>,threshold <arg>"},
{"add-node", clusterManagerCommandAddNode, 2,
"new_host:new_port existing_host:existing_port", "slave,master-id <arg>"},
{"del-node", clusterManagerCommandDeleteNode, 2, "host:port node_id",NULL},
@@ -2196,44 +2176,6 @@ static int clusterManagerCheckRedisReply(clusterManagerNode *n,
return 1;
}
/* Execute MULTI command on a cluster node. */
static int clusterManagerStartTransaction(clusterManagerNode *node) {
redisReply *reply = CLUSTER_MANAGER_COMMAND(node, "MULTI");
int success = clusterManagerCheckRedisReply(node, reply, NULL);
if (reply) freeReplyObject(reply);
return success;
}
/* Execute EXEC command on a cluster node. */
static int clusterManagerExecTransaction(clusterManagerNode *node,
clusterManagerOnReplyError onerror)
{
redisReply *reply = CLUSTER_MANAGER_COMMAND(node, "EXEC");
int success = clusterManagerCheckRedisReply(node, reply, NULL);
if (success) {
if (reply->type != REDIS_REPLY_ARRAY) {
success = 0;
goto cleanup;
}
size_t i;
for (i = 0; i < reply->elements; i++) {
redisReply *r = reply->element[i];
char *err = NULL;
success = clusterManagerCheckRedisReply(node, r, &err);
if (!success && onerror) success = onerror(r, i);
if (err) {
if (!success)
CLUSTER_MANAGER_PRINT_REPLY_ERROR(node, err);
zfree(err);
}
if (!success) break;
}
}
cleanup:
if (reply) freeReplyObject(reply);
return success;
}
static int clusterManagerNodeConnect(clusterManagerNode *node) {
if (node->context) redisFree(node->context);
node->context = redisConnect(node->ip, node->port);
@@ -2784,7 +2726,8 @@ static int clusterManagerSetSlot(clusterManagerNode *node1,
if (err != NULL) {
*err = zmalloc((reply->len + 1) * sizeof(char));
strcpy(*err, reply->str);
} else CLUSTER_MANAGER_PRINT_REPLY_ERROR(node1, reply->str);
CLUSTER_MANAGER_PRINT_REPLY_ERROR(node1, err);
}
goto cleanup;
}
cleanup:
@@ -2792,84 +2735,6 @@ cleanup:
return success;
}
static int clusterManagerClearSlotStatus(clusterManagerNode *node, int slot) {
redisReply *reply = CLUSTER_MANAGER_COMMAND(node,
"CLUSTER SETSLOT %d %s", slot, "STABLE");
int success = clusterManagerCheckRedisReply(node, reply, NULL);
if (reply) freeReplyObject(reply);
return success;
}
static int clusterManagerDelSlot(clusterManagerNode *node, int slot,
int ignore_unassigned_err)
{
redisReply *reply = CLUSTER_MANAGER_COMMAND(node,
"CLUSTER DELSLOTS %d", slot);
char *err = NULL;
int success = clusterManagerCheckRedisReply(node, reply, &err);
if (!success && reply && reply->type == REDIS_REPLY_ERROR &&
ignore_unassigned_err &&
strstr(reply->str, "already unassigned") != NULL) success = 1;
if (!success && err != NULL) {
CLUSTER_MANAGER_PRINT_REPLY_ERROR(node, err);
zfree(err);
}
if (reply) freeReplyObject(reply);
return success;
}
static int clusterManagerAddSlot(clusterManagerNode *node, int slot) {
redisReply *reply = CLUSTER_MANAGER_COMMAND(node,
"CLUSTER ADDSLOTS %d", slot);
int success = clusterManagerCheckRedisReply(node, reply, NULL);
if (reply) freeReplyObject(reply);
return success;
}
static signed int clusterManagerCountKeysInSlot(clusterManagerNode *node,
int slot)
{
redisReply *reply = CLUSTER_MANAGER_COMMAND(node,
"CLUSTER COUNTKEYSINSLOT %d", slot);
int count = -1;
int success = clusterManagerCheckRedisReply(node, reply, NULL);
if (success && reply->type == REDIS_REPLY_INTEGER) count = reply->integer;
if (reply) freeReplyObject(reply);
return count;
}
static int clusterManagerBumpEpoch(clusterManagerNode *node) {
redisReply *reply = CLUSTER_MANAGER_COMMAND(node, "CLUSTER BUMPEPOCH");
int success = clusterManagerCheckRedisReply(node, reply, NULL);
if (reply) freeReplyObject(reply);
return success;
}
static int clusterManagerIgnoreUnassignedErr(redisReply *reply, int bulk_idx) {
if (bulk_idx == 0 && reply) {
if (reply->type == REDIS_REPLY_ERROR)
return strstr(reply->str, "already unassigned") != NULL;
}
return 0;
}
static int clusterManagerSetSlotOwner(clusterManagerNode *owner,
int slot,
int do_clear)
{
int success = clusterManagerStartTransaction(owner);
if (!success) return 0;
/* Ensure the slot is not already assigned. */
clusterManagerDelSlot(owner, slot, 1);
/* Add the slot and bump epoch. */
clusterManagerAddSlot(owner, slot);
if (do_clear) clusterManagerClearSlotStatus(owner, slot);
clusterManagerBumpEpoch(owner);
success = clusterManagerExecTransaction(owner,
clusterManagerIgnoreUnassignedErr);
return success;
}
/* Migrate keys taken from reply->elements. It returns the reply from the
* MIGRATE command, or NULL if something goes wrong. If the argument 'dots'
* is not NULL, a dot will be printed for every migrated key. */
@@ -2950,8 +2815,8 @@ static int clusterManagerMigrateKeysInSlot(clusterManagerNode *source,
char **err)
{
int success = 1;
int replace_existing_keys = (config.cluster_manager_command.flags &
(CLUSTER_MANAGER_CMD_FLAG_FIX | CLUSTER_MANAGER_CMD_FLAG_REPLACE));
int do_fix = (config.cluster_manager_command.flags &
CLUSTER_MANAGER_CMD_FLAG_FIX);
while (1) {
char *dots = NULL;
redisReply *reply = NULL, *migrate_reply = NULL;
@@ -2965,7 +2830,7 @@ static int clusterManagerMigrateKeysInSlot(clusterManagerNode *source,
if (err != NULL) {
*err = zmalloc((reply->len + 1) * sizeof(char));
strcpy(*err, reply->str);
CLUSTER_MANAGER_PRINT_REPLY_ERROR(source, *err);
CLUSTER_MANAGER_PRINT_REPLY_ERROR(source, err);
}
goto next;
}
@@ -2982,23 +2847,15 @@ static int clusterManagerMigrateKeysInSlot(clusterManagerNode *source,
dots);
if (migrate_reply == NULL) goto next;
if (migrate_reply->type == REDIS_REPLY_ERROR) {
int is_busy = strstr(migrate_reply->str, "BUSYKEY") != NULL;
int not_served = strstr(migrate_reply->str, "slot not served") != NULL;
if (replace_existing_keys && (is_busy || not_served)) {
/* If the key already exists, try to migrate keys
* adding REPLACE option.
* If the key's slot is not served, try to assign slot
* to the target node. */
if (not_served)
clusterManagerSetSlot(source, target, slot, "node", NULL);
if (do_fix && strstr(migrate_reply->str, "BUSYKEY")) {
clusterManagerLogWarn("*** Target key exists. "
"Replacing it for FIX.\n");
freeReplyObject(migrate_reply);
/* Try to migrate keys adding REPLACE option. */
migrate_reply = clusterManagerMigrateKeysInReply(source,
target,
reply,
is_busy,
timeout,
1, timeout,
NULL);
success = (migrate_reply != NULL &&
migrate_reply->type != REDIS_REPLY_ERROR);
@@ -3084,7 +2941,7 @@ static int clusterManagerMoveSlot(clusterManagerNode *source,
if (err != NULL) {
*err = zmalloc((r->len + 1) * sizeof(char));
strcpy(*err, r->str);
CLUSTER_MANAGER_PRINT_REPLY_ERROR(n, *err);
CLUSTER_MANAGER_PRINT_REPLY_ERROR(n, err);
}
}
freeReplyObject(r);
@@ -3444,8 +3301,8 @@ static sds clusterManagerGetConfigSignature(clusterManagerNode *node) {
nodename = token;
tot_size = (p - token);
name_len = tot_size++; // Make room for ':' in tot_size
}
if (++i == 8) break;
} else if (i == 8) break;
i++;
}
if (i != 8) continue;
if (nodename == NULL) continue;
@@ -3484,7 +3341,7 @@ static sds clusterManagerGetConfigSignature(clusterManagerNode *node) {
char *sp = cfg + name_len;
*(sp++) = ':';
for (i = 0; i < c; i++) {
if (i > 0) *(sp++) = ',';
if (i > 0) *(sp++) = '|';
int slen = strlen(slots[i]);
memcpy(sp, slots[i], slen);
sp += slen;
@@ -3642,34 +3499,6 @@ static clusterManagerNode *clusterManagerNodeWithLeastReplicas() {
return node;
}
/* This function returns a random master node, return NULL if none */
static clusterManagerNode *clusterManagerNodeMasterRandom() {
int master_count = 0;
int idx;
listIter li;
listNode *ln;
listRewind(cluster_manager.nodes, &li);
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n->flags & CLUSTER_MANAGER_FLAG_SLAVE) continue;
master_count++;
}
srand(time(NULL));
idx = rand() % master_count;
listRewind(cluster_manager.nodes, &li);
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n->flags & CLUSTER_MANAGER_FLAG_SLAVE) continue;
if (!idx--) {
return n;
}
}
/* Can not be reached */
return NULL;
}
static int clusterManagerFixSlotsCoverage(char *all_slots) {
int i, fixed = 0;
list *none = NULL, *single = NULL, *multi = NULL;
@@ -3742,22 +3571,26 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
"across the cluster:\n");
clusterManagerPrintSlotsList(none);
if (confirmWithYes("Fix these slots by covering with a random node?")){
srand(time(NULL));
listIter li;
listNode *ln;
listRewind(none, &li);
while ((ln = listNext(&li)) != NULL) {
sds slot = ln->value;
int s = atoi(slot);
clusterManagerNode *n = clusterManagerNodeMasterRandom();
long idx = (long) (rand() % listLength(cluster_manager.nodes));
listNode *node_n = listIndex(cluster_manager.nodes, idx);
assert(node_n != NULL);
clusterManagerNode *n = node_n->value;
clusterManagerLogInfo(">>> Covering slot %s with %s:%d\n",
slot, n->ip, n->port);
if (!clusterManagerSetSlotOwner(n, s, 0)) {
fixed = -1;
goto cleanup;
}
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER ADDSLOTS %s", slot);
if (!clusterManagerCheckRedisReply(n, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
/* Since CLUSTER ADDSLOTS succeeded, we also update the slot
* info into the node struct, in order to keep it synced */
n->slots[s] = 1;
n->slots[atoi(slot)] = 1;
fixed++;
}
}
@@ -3765,7 +3598,7 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
/* Handle case "2": keys only in one node. */
if (listLength(single) > 0) {
printf("The following uncovered slots have keys in just one node:\n");
printf("The following uncovered slots have keys in just one node:\n");
clusterManagerPrintSlotsList(single);
if (confirmWithYes("Fix these slots by covering with those nodes?")){
listIter li;
@@ -3773,7 +3606,6 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
listRewind(single, &li);
while ((ln = listNext(&li)) != NULL) {
sds slot = ln->value;
int s = atoi(slot);
dictEntry *entry = dictFind(clusterManagerUncoveredSlots, slot);
assert(entry != NULL);
list *nodes = (list *) dictGetVal(entry);
@@ -3782,10 +3614,11 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
clusterManagerNode *n = fn->value;
clusterManagerLogInfo(">>> Covering slot %s with %s:%d\n",
slot, n->ip, n->port);
if (!clusterManagerSetSlotOwner(n, s, 0)) {
fixed = -1;
goto cleanup;
}
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER ADDSLOTS %s", slot);
if (!clusterManagerCheckRedisReply(n, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
/* Since CLUSTER ADDSLOTS succeeded, we also update the slot
* info into the node struct, in order to keep it synced */
n->slots[atoi(slot)] = 1;
@@ -3818,10 +3651,16 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
clusterManagerLogInfo(">>> Covering slot %s moving keys "
"to %s:%d\n", slot,
target->ip, target->port);
if (!clusterManagerSetSlotOwner(target, s, 1)) {
fixed = -1;
goto cleanup;
}
redisReply *r = CLUSTER_MANAGER_COMMAND(target,
"CLUSTER ADDSLOTS %s", slot);
if (!clusterManagerCheckRedisReply(target, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
r = CLUSTER_MANAGER_COMMAND(target,
"CLUSTER SETSLOT %s %s", slot, "STABLE");
if (!clusterManagerCheckRedisReply(target, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
/* Since CLUSTER ADDSLOTS succeeded, we also update the slot
* info into the node struct, in order to keep it synced */
target->slots[atoi(slot)] = 1;
@@ -3831,16 +3670,16 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
while ((nln = listNext(&nli)) != NULL) {
clusterManagerNode *src = nln->value;
if (src == target) continue;
/* Assign the slot to target node in the source node. */
if (!clusterManagerSetSlot(src, target, s, "NODE", NULL))
fixed = -1;
if (fixed < 0) goto cleanup;
/* Set the source node in 'importing' state
* (even if we will actually migrate keys away)
* in order to avoid receiving redirections
* for MIGRATE. */
if (!clusterManagerSetSlot(src, target, s,
"IMPORTING", NULL)) fixed = -1;
redisReply *r = CLUSTER_MANAGER_COMMAND(src,
"CLUSTER SETSLOT %s %s %s", slot,
"IMPORTING", target->name);
if (!clusterManagerCheckRedisReply(target, r, NULL))
fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
int opts = CLUSTER_MANAGER_OPT_VERBOSE |
CLUSTER_MANAGER_OPT_COLD;
@@ -3848,9 +3687,6 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
fixed = -1;
goto cleanup;
}
if (!clusterManagerClearSlotStatus(src, s))
fixed = -1;
if (fixed < 0) goto cleanup;
}
fixed++;
}
@@ -3884,22 +3720,11 @@ static int clusterManagerFixOpenSlot(int slot) {
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n->flags & CLUSTER_MANAGER_FLAG_SLAVE) continue;
if (n->slots[slot]) listAddNodeTail(owners, n);
else {
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER COUNTKEYSINSLOT %d", slot);
success = clusterManagerCheckRedisReply(n, r, NULL);
if (success && r->integer > 0) {
clusterManagerLogWarn("*** Found keys about slot %d "
"in non-owner node %s:%d!\n", slot,
n->ip, n->port);
listAddNodeTail(owners, n);
}
if (r) freeReplyObject(r);
if (!success) goto cleanup;
if (n->slots[slot]) {
if (owner == NULL) owner = n;
listAddNodeTail(owners, n);
}
}
if (listLength(owners) == 1) owner = listFirst(owners)->value;
listRewind(cluster_manager.nodes, &li);
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
@@ -3910,7 +3735,7 @@ static int clusterManagerFixOpenSlot(int slot) {
sds migrating_slot = n->migrating[i];
if (atoi(migrating_slot) == slot) {
char *sep = (listLength(migrating) == 0 ? "" : ",");
migrating_str = sdscatfmt(migrating_str, "%s%s:%u",
migrating_str = sdscatfmt(migrating_str, "%s%S:%u",
sep, n->ip, n->port);
listAddNodeTail(migrating, n);
is_migrating = 1;
@@ -3923,7 +3748,7 @@ static int clusterManagerFixOpenSlot(int slot) {
sds importing_slot = n->importing[i];
if (atoi(importing_slot) == slot) {
char *sep = (listLength(importing) == 0 ? "" : ",");
importing_str = sdscatfmt(importing_str, "%s%s:%u",
importing_str = sdscatfmt(importing_str, "%s%S:%u",
sep, n->ip, n->port);
listAddNodeTail(importing, n);
is_importing = 1;
@@ -3942,20 +3767,15 @@ static int clusterManagerFixOpenSlot(int slot) {
clusterManagerLogWarn("*** Found keys about slot %d "
"in node %s:%d!\n", slot, n->ip,
n->port);
char *sep = (listLength(importing) == 0 ? "" : ",");
importing_str = sdscatfmt(importing_str, "%s%S:%u",
sep, n->ip, n->port);
listAddNodeTail(importing, n);
}
if (r) freeReplyObject(r);
if (!success) goto cleanup;
}
}
if (sdslen(migrating_str) > 0)
printf("Set as migrating in: %s\n", migrating_str);
if (sdslen(importing_str) > 0)
printf("Set as importing in: %s\n", importing_str);
/* If there is no slot owner, set as owner the node with the biggest
printf("Set as migrating in: %s\n", migrating_str);
printf("Set as importing in: %s\n", importing_str);
/* If there is no slot owner, set as owner the slot with the biggest
* number of keys, among the set of migrating / importing nodes. */
if (owner == NULL) {
clusterManagerLogInfo(">>> Nobody claims ownership, "
@@ -3973,9 +3793,15 @@ static int clusterManagerFixOpenSlot(int slot) {
// Use ADDSLOTS to assign the slot.
clusterManagerLogWarn("*** Configuring %s:%d as the slot owner\n",
owner->ip, owner->port);
success = clusterManagerClearSlotStatus(owner, slot);
redisReply *reply = CLUSTER_MANAGER_COMMAND(owner, "CLUSTER "
"SETSLOT %d %s",
slot, "STABLE");
success = clusterManagerCheckRedisReply(owner, reply, NULL);
if (reply) freeReplyObject(reply);
if (!success) goto cleanup;
success = clusterManagerSetSlotOwner(owner, slot, 0);
reply = CLUSTER_MANAGER_COMMAND(owner, "CLUSTER ADDSLOTS %d", slot);
success = clusterManagerCheckRedisReply(owner, reply, NULL);
if (reply) freeReplyObject(reply);
if (!success) goto cleanup;
/* Since CLUSTER ADDSLOTS succeeded, we also update the slot
* info into the node struct, in order to keep it synced */
@@ -3983,7 +3809,9 @@ static int clusterManagerFixOpenSlot(int slot) {
/* Make sure this information will propagate. Not strictly needed
* since there is no past owner, so all the other nodes will accept
* whatever epoch this node will claim the slot with. */
success = clusterManagerBumpEpoch(owner);
reply = CLUSTER_MANAGER_COMMAND(owner, "CLUSTER BUMPEPOCH");
success = clusterManagerCheckRedisReply(owner, reply, NULL);
if (reply) freeReplyObject(reply);
if (!success) goto cleanup;
/* Remove the owner from the list of migrating/importing
* nodes. */
@@ -3999,38 +3827,32 @@ static int clusterManagerFixOpenSlot(int slot) {
* in migrating state, since migrating is a valid state only for
* slot owners. */
if (listLength(owners) > 1) {
/* Owner cannot be NULL at this point, since if there are more owners,
* the owner has been set in the previous condition (owner == NULL). */
assert(owner != NULL);
owner = clusterManagerGetNodeWithMostKeysInSlot(owners, slot, NULL);
listRewind(owners, &li);
redisReply *reply = NULL;
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n == owner) continue;
success = clusterManagerDelSlot(n, slot, 1);
if (!success) goto cleanup;
n->slots[slot] = 0;
/* Assign the slot to the owner in the node 'n' configuration.' */
success = clusterManagerSetSlot(n, owner, slot, "node", NULL);
reply = CLUSTER_MANAGER_COMMAND(n, "CLUSTER DELSLOT %d", slot);
success = clusterManagerCheckRedisReply(n, reply, NULL);
if (reply) freeReplyObject(reply);
if (!success) goto cleanup;
success = clusterManagerSetSlot(n, owner, slot, "importing", NULL);
if (!success) goto cleanup;
/* Avoid duplicates. */
clusterManagerRemoveNodeFromList(importing, n);
clusterManagerRemoveNodeFromList(importing, n); //Avoid duplicates
listAddNodeTail(importing, n);
/* Ensure that the node is not in the migrating list. */
clusterManagerRemoveNodeFromList(migrating, n);
}
reply = CLUSTER_MANAGER_COMMAND(owner, "CLUSTER BUMPEPOCH");
success = clusterManagerCheckRedisReply(owner, reply, NULL);
if (reply) freeReplyObject(reply);
if (!success) goto cleanup;
}
int move_opts = CLUSTER_MANAGER_OPT_VERBOSE;
/* Case 1: The slot is in migrating state in one node, and in
* importing state in 1 node. That's trivial to address. */
/* Case 1: The slot is in migrating state in one slot, and in
* importing state in 1 slot. That's trivial to address. */
if (listLength(migrating) == 1 && listLength(importing) == 1) {
clusterManagerNode *src = listFirst(migrating)->value;
clusterManagerNode *dst = listFirst(importing)->value;
clusterManagerLogInfo(">>> Case 1: Moving slot %d from "
"%s:%d to %s:%d\n", slot,
src->ip, src->port, dst->ip, dst->port);
move_opts |= CLUSTER_MANAGER_OPT_UPDATE;
success = clusterManagerMoveSlot(src, dst, slot, move_opts, NULL);
}
/* Case 2: There are multiple nodes that claim the slot as importing,
@@ -4038,7 +3860,7 @@ static int clusterManagerFixOpenSlot(int slot) {
* the slot. In this case we just move all the keys to the owner
* according to the configuration. */
else if (listLength(migrating) == 0 && listLength(importing) > 0) {
clusterManagerLogInfo(">>> Case 2: Moving all the %d slot keys to its "
clusterManagerLogInfo(">>> Moving all the %d slot keys to its "
"owner %s:%d\n", slot, owner->ip, owner->port);
move_opts |= CLUSTER_MANAGER_OPT_COLD;
listRewind(importing, &li);
@@ -4049,117 +3871,38 @@ static int clusterManagerFixOpenSlot(int slot) {
if (!success) goto cleanup;
clusterManagerLogInfo(">>> Setting %d as STABLE in "
"%s:%d\n", slot, n->ip, n->port);
success = clusterManagerClearSlotStatus(n, slot);
redisReply *r = CLUSTER_MANAGER_COMMAND(n, "CLUSTER SETSLOT %d %s",
slot, "STABLE");
success = clusterManagerCheckRedisReply(n, r, NULL);
if (r) freeReplyObject(r);
if (!success) goto cleanup;
}
/* Since the slot has been moved in "cold" mode, ensure that all the
* other nodes update their own configuration about the slot itself. */
listRewind(cluster_manager.nodes, &li);
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n == owner) continue;
if (n->flags & CLUSTER_MANAGER_FLAG_SLAVE) continue;
success = clusterManagerSetSlot(n, owner, slot, "NODE", NULL);
if (!success) goto cleanup;
}
}
/* Case 3: The slot is in migrating state in one node but multiple
* other nodes claim to be in importing state and don't have any key in
* the slot. We search for the importing node having the same ID as
* the destination node of the migrating node.
* In that case we move the slot from the migrating node to this node and
* we close the importing states on all the other importing nodes.
* If no importing node has the same ID as the destination node of the
* migrating node, the slot's state is closed on both the migrating node
* and the importing nodes. */
else if (listLength(migrating) == 1 && listLength(importing) > 1) {
int try_to_fix = 1;
clusterManagerNode *src = listFirst(migrating)->value;
clusterManagerNode *dst = NULL;
sds target_id = NULL;
for (int i = 0; i < src->migrating_count; i += 2) {
sds migrating_slot = src->migrating[i];
if (atoi(migrating_slot) == slot) {
target_id = src->migrating[i + 1];
break;
}
}
assert(target_id != NULL);
listIter li;
listNode *ln;
listRewind(importing, &li);
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
int count = clusterManagerCountKeysInSlot(n, slot);
if (count > 0) {
try_to_fix = 0;
break;
}
if (strcmp(n->name, target_id) == 0) dst = n;
}
if (!try_to_fix) goto unhandled_case;
if (dst != NULL) {
clusterManagerLogInfo(">>> Case 3: Moving slot %d from %s:%d to "
"%s:%d and closing it on all the other "
"importing nodes.\n",
slot, src->ip, src->port,
dst->ip, dst->port);
/* Move the slot to the destination node. */
success = clusterManagerMoveSlot(src, dst, slot, move_opts, NULL);
if (!success) goto cleanup;
/* Close slot on all the other importing nodes. */
listRewind(importing, &li);
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (dst == n) continue;
success = clusterManagerClearSlotStatus(n, slot);
if (!success) goto cleanup;
}
} else {
clusterManagerLogInfo(">>> Case 3: Closing slot %d on both "
"migrating and importing nodes.\n", slot);
/* Close the slot on both the migrating node and the importing
* nodes. */
success = clusterManagerClearSlotStatus(src, slot);
if (!success) goto cleanup;
listRewind(importing, &li);
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
success = clusterManagerClearSlotStatus(n, slot);
if (!success) goto cleanup;
}
}
} else {
int try_to_close_slot = (listLength(importing) == 0 &&
listLength(migrating) == 1);
if (try_to_close_slot) {
clusterManagerNode *n = listFirst(migrating)->value;
if (!owner || owner != n) {
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER GETKEYSINSLOT %d %d", slot, 10);
success = clusterManagerCheckRedisReply(n, r, NULL);
if (r) {
if (success) try_to_close_slot = (r->elements == 0);
freeReplyObject(r);
}
if (!success) goto cleanup;
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER GETKEYSINSLOT %d %d", slot, 10);
success = clusterManagerCheckRedisReply(n, r, NULL);
if (r) {
if (success) try_to_close_slot = (r->elements == 0);
freeReplyObject(r);
}
if (!success) goto cleanup;
}
/* Case 4: There are no slots claiming to be in importing state, but
* there is a migrating node that actually don't have any key or is the
* slot owner. We can just close the slot, probably a reshard
* interrupted in the middle. */
/* Case 3: There are no slots claiming to be in importing state, but
* there is a migrating node that actually don't have any key. We
* can just close the slot, probably a reshard interrupted in the middle. */
if (try_to_close_slot) {
clusterManagerNode *n = listFirst(migrating)->value;
clusterManagerLogInfo(">>> Case 4: Closing slot %d on %s:%d\n",
slot, n->ip, n->port);
redisReply *r = CLUSTER_MANAGER_COMMAND(n, "CLUSTER SETSLOT %d %s",
slot, "STABLE");
success = clusterManagerCheckRedisReply(n, r, NULL);
if (r) freeReplyObject(r);
if (!success) goto cleanup;
} else {
unhandled_case:
success = 0;
clusterManagerLogErr("[ERR] Sorry, redis-cli can't fix this slot "
"yet (work in progress). Slot is set as "
@@ -4177,55 +3920,17 @@ cleanup:
return success;
}
static int clusterManagerFixMultipleSlotOwners(int slot, list *owners) {
clusterManagerLogInfo(">>> Fixing multiple owners for slot %d...\n", slot);
int success = 0;
assert(listLength(owners) > 1);
clusterManagerNode *owner = clusterManagerGetNodeWithMostKeysInSlot(owners,
slot,
NULL);
if (!owner) owner = listFirst(owners)->value;
clusterManagerLogInfo(">>> Setting slot %d owner: %s:%d\n",
slot, owner->ip, owner->port);
/* Set the slot owner. */
if (!clusterManagerSetSlotOwner(owner, slot, 0)) return 0;
listIter li;
listNode *ln;
listRewind(cluster_manager.nodes, &li);
/* Update configuration in all the other master nodes by assigning the slot
* itself to the new owner, and by eventually migrating keys if the node
* has keys for the slot. */
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n == owner) continue;
if (n->flags & CLUSTER_MANAGER_FLAG_SLAVE) continue;
int count = clusterManagerCountKeysInSlot(n, slot);
success = (count >= 0);
if (!success) break;
clusterManagerDelSlot(n, slot, 1);
if (!clusterManagerSetSlot(n, owner, slot, "node", NULL)) return 0;
if (count > 0) {
int opts = CLUSTER_MANAGER_OPT_VERBOSE |
CLUSTER_MANAGER_OPT_COLD;
success = clusterManagerMoveSlot(n, owner, slot, opts, NULL);
if (!success) break;
}
}
return success;
}
static int clusterManagerCheckCluster(int quiet) {
listNode *ln = listFirst(cluster_manager.nodes);
if (!ln) return 0;
int result = 1;
int do_fix = config.cluster_manager_command.flags &
CLUSTER_MANAGER_CMD_FLAG_FIX;
clusterManagerNode *node = ln->value;
clusterManagerLogInfo(">>> Performing Cluster Check (using node %s:%d)\n",
node->ip, node->port);
int result = 1, consistent = 0;
int do_fix = config.cluster_manager_command.flags &
CLUSTER_MANAGER_CMD_FLAG_FIX;
if (!quiet) clusterManagerShowNodes();
consistent = clusterManagerIsConfigConsistent();
if (!consistent) {
if (!clusterManagerIsConfigConsistent()) {
sds err = sdsnew("[ERR] Nodes don't agree about configuration!");
clusterManagerOnError(err);
result = 0;
@@ -4233,7 +3938,7 @@ static int clusterManagerCheckCluster(int quiet) {
clusterManagerLogOk("[OK] All nodes agree about slots "
"configuration.\n");
}
/* Check open slots */
// Check open slots
clusterManagerLogInfo(">>> Check for open slots...\n");
listIter li;
listRewind(cluster_manager.nodes, &li);
@@ -4292,7 +3997,7 @@ static int clusterManagerCheckCluster(int quiet) {
clusterManagerLogErr("%s.\n", (char *) errstr);
sdsfree(errstr);
if (do_fix) {
/* Fix open slots. */
// Fix open slots.
dictReleaseIterator(iter);
iter = dictGetIterator(open_slots);
while ((entry = dictNext(iter)) != NULL) {
@@ -4320,58 +4025,12 @@ static int clusterManagerCheckCluster(int quiet) {
result = 0;
if (do_fix/* && result*/) {
dictType dtype = clusterManagerDictType;
dtype.keyDestructor = dictSdsDestructor;
dtype.valDestructor = dictListDestructor;
clusterManagerUncoveredSlots = dictCreate(&dtype, NULL);
int fixed = clusterManagerFixSlotsCoverage(slots);
if (fixed > 0) result = 1;
}
}
int search_multiple_owners = config.cluster_manager_command.flags &
CLUSTER_MANAGER_CMD_FLAG_CHECK_OWNERS;
if (search_multiple_owners) {
/* Check whether there are multiple owners, even when slots are
* fully covered and there are no open slots. */
clusterManagerLogInfo(">>> Check for multiple slot owners...\n");
int slot = 0;
for (; slot < CLUSTER_MANAGER_SLOTS; slot++) {
listIter li;
listNode *ln;
listRewind(cluster_manager.nodes, &li);
list *owners = listCreate();
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n->flags & CLUSTER_MANAGER_FLAG_SLAVE) continue;
if (n->slots[slot]) listAddNodeTail(owners, n);
else {
/* Nodes having keys for the slot will be considered
* owners too. */
int count = clusterManagerCountKeysInSlot(n, slot);
if (count > 0) listAddNodeTail(owners, n);
}
}
if (listLength(owners) > 1) {
result = 0;
clusterManagerLogErr("[WARNING] Slot %d has %d owners:\n",
slot, listLength(owners));
listRewind(owners, &li);
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
clusterManagerLogErr(" %s:%d\n", n->ip, n->port);
}
if (do_fix) {
result = clusterManagerFixMultipleSlotOwners(slot, owners);
if (!result) {
clusterManagerLogErr("Failed to fix multiple owners "
"for slot %d\n", slot);
listRelease(owners);
break;
}
}
}
listRelease(owners);
}
}
return result;
}
@@ -4402,7 +4061,7 @@ static clusterManagerNode *clusterNodeForResharding(char *id,
static list *clusterManagerComputeReshardTable(list *sources, int numslots) {
list *moved = listCreate();
int src_count = listLength(sources), i = 0, tot_slots = 0, j;
clusterManagerNode **sorted = zmalloc(src_count * sizeof(*sorted));
clusterManagerNode **sorted = zmalloc(src_count * sizeof(**sorted));
listIter li;
listNode *ln;
listRewind(sources, &li);
@@ -5441,13 +5100,10 @@ static int clusterManagerCommandSetTimeout(int argc, char **argv) {
n->port);
ok_count++;
continue;
reply_err:;
int need_free = 0;
reply_err:
if (err == NULL) err = "";
else need_free = 1;
clusterManagerLogErr("ERR setting node-timeot for %s:%d: %s\n", n->ip,
n->port, err);
if (need_free) zfree(err);
err_count++;
}
clusterManagerLogInfo(">>> New node timeout set. %d OK, %d ERR.\n",
@@ -5624,7 +5280,7 @@ static int clusterManagerCommandCall(int argc, char **argv) {
if (status != REDIS_OK || reply == NULL )
printf("%s:%d: Failed!\n", n->ip, n->port);
else {
sds formatted_reply = cliFormatReplyRaw(reply);
sds formatted_reply = cliFormatReplyTTY(reply, "");
printf("%s:%d: %s\n", n->ip, n->port, (char *) formatted_reply);
sdsfree(formatted_reply);
}
@@ -7001,8 +6657,6 @@ int main(int argc, char **argv) {
argc -= firstarg;
argv += firstarg;
parseEnv();
/* Cluster Manager mode */
if (CLUSTER_MANAGER_MODE()) {
clusterManagerCommandProc *proc = validateClusterManagerCommand();
-4
View File
@@ -332,8 +332,6 @@ void REDISMODULE_API_FUNC(RedisModule_GetRandomBytes)(unsigned char *dst, size_t
void REDISMODULE_API_FUNC(RedisModule_GetRandomHexChars)(char *dst, size_t len);
void REDISMODULE_API_FUNC(RedisModule_SetDisconnectCallback)(RedisModuleBlockedClient *bc, RedisModuleDisconnectFunc callback);
void REDISMODULE_API_FUNC(RedisModule_SetClusterFlags)(RedisModuleCtx *ctx, uint64_t flags);
int REDISMODULE_API_FUNC(RedisModule_ExportSharedAPI)(RedisModuleCtx *ctx, const char *apiname, void *func);
void *REDISMODULE_API_FUNC(RedisModule_GetSharedAPI)(RedisModuleCtx *ctx, const char *apiname);
#endif
/* This is included inline inside each Redis module. */
@@ -494,8 +492,6 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(GetRandomBytes);
REDISMODULE_GET_API(GetRandomHexChars);
REDISMODULE_GET_API(SetClusterFlags);
REDISMODULE_GET_API(ExportSharedAPI);
REDISMODULE_GET_API(GetSharedAPI);
#endif
if (RedisModule_IsModuleNameBusy && RedisModule_IsModuleNameBusy(name)) return REDISMODULE_ERR;
-12
View File
@@ -1245,18 +1245,6 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
if (eof_reached) {
int aof_is_enabled = server.aof_state != AOF_OFF;
/* Ensure background save doesn't overwrite synced data */
if (server.rdb_child_pid != -1) {
serverLog(LL_NOTICE,
"Replica is about to load the RDB file received from the "
"master, but there is a pending RDB child running. "
"Killing process %ld and removing its temp file to avoid "
"any race",
(long) server.rdb_child_pid);
kill(server.rdb_child_pid,SIGUSR1);
rdbRemoveTempFile(server.rdb_child_pid);
}
if (rename(server.repl_transfer_tmpfile,server.rdb_filename) == -1) {
serverLog(LL_WARNING,"Failed trying to rename the temp DB into dump.rdb in MASTER <-> REPLICA synchronization: %s", strerror(errno));
cancelReplicationHandshake();
+1 -1
View File
@@ -695,7 +695,7 @@ sds sdscatfmt(sds s, char const *fmt, ...) {
* s = sdstrim(s,"Aa. :");
* printf("%s\n", s);
*
* Output will be just "HelloWorld".
* Output will be just "Hello World".
*/
sds sdstrim(sds s, const char *cset) {
char *start, *end, *sp, *ep;
+3 -18
View File
@@ -452,15 +452,13 @@ struct redisCommand sentinelcmds[] = {
{"info",sentinelInfoCommand,-1,"",0,NULL,0,0,0,0,0},
{"role",sentinelRoleCommand,1,"l",0,NULL,0,0,0,0,0},
{"client",clientCommand,-2,"rs",0,NULL,0,0,0,0,0},
{"shutdown",shutdownCommand,-1,"",0,NULL,0,0,0,0,0},
{"auth",authCommand,2,"sltF",0,NULL,0,0,0,0,0}
{"shutdown",shutdownCommand,-1,"",0,NULL,0,0,0,0,0}
};
/* This function overwrites a few normal Redis config default with Sentinel
* specific defaults. */
void initSentinelConfig(void) {
server.port = REDIS_SENTINEL_PORT;
server.protected_mode = 0; /* Sentinel must be exposed. */
}
/* Perform the Sentinel mode initialization. */
@@ -1943,25 +1941,12 @@ werr:
/* Send the AUTH command with the specified master password if needed.
* Note that for slaves the password set for the master is used.
*
* In case this Sentinel requires a password as well, via the "requirepass"
* configuration directive, we assume we should use the local password in
* order to authenticate when connecting with the other Sentinels as well.
* So basically all the Sentinels share the same password and use it to
* authenticate reciprocally.
*
* We don't check at all if the command was successfully transmitted
* to the instance as if it fails Sentinel will detect the instance down,
* will disconnect and reconnect the link and so forth. */
void sentinelSendAuthIfNeeded(sentinelRedisInstance *ri, redisAsyncContext *c) {
char *auth_pass = NULL;
if (ri->flags & SRI_MASTER) {
auth_pass = ri->auth_pass;
} else if (ri->flags & SRI_SLAVE) {
auth_pass = ri->master->auth_pass;
} else if (ri->flags & SRI_SENTINEL) {
if (server.requirepass) auth_pass = server.requirepass;
}
char *auth_pass = (ri->flags & SRI_MASTER) ? ri->auth_pass :
ri->master->auth_pass;
if (auth_pass) {
if (redisAsyncCommand(c, sentinelDiscardReplyCallback, ri, "%s %s",
+15 -18
View File
@@ -1526,10 +1526,10 @@ void initServerConfig(void) {
server.runid[CONFIG_RUN_ID_SIZE] = '\0';
changeReplicationId();
clearReplicationId2();
server.timezone = getTimeZone(); /* Initialized by tzset(). */
server.timezone = timezone; /* Initialized by tzset(). */
server.configfile = NULL;
server.executable = NULL;
server.hz = server.config_hz = CONFIG_DEFAULT_HZ;
server.config_hz = CONFIG_DEFAULT_HZ;
server.dynamic_hz = CONFIG_DEFAULT_DYNAMIC_HZ;
server.arch_bits = (sizeof(long) == 8) ? 64 : 32;
server.port = CONFIG_DEFAULT_SERVER_PORT;
@@ -1958,13 +1958,9 @@ int listenToPort(int port, int *fds, int *count) {
}
if (fds[*count] == ANET_ERR) {
serverLog(LL_WARNING,
"Could not create server TCP listening socket %s:%d: %s",
"Creating Server TCP listening socket %s:%d: %s",
server.bindaddr[j] ? server.bindaddr[j] : "*",
port, server.neterr);
if (errno == ENOPROTOOPT || errno == EPROTONOSUPPORT ||
errno == ESOCKTNOSUPPORT || errno == EPFNOSUPPORT ||
errno == EAFNOSUPPORT || errno == EADDRNOTAVAIL)
continue;
return C_ERR;
}
anetNonBlock(NULL,fds[*count]);
@@ -2607,23 +2603,24 @@ int processCommand(client *c) {
}
/* Handle the maxmemory directive.
*
* First we try to free some memory if possible (if there are volatile
* keys in the dataset). If there are not the only thing we can do
* is returning an error.
*
* Note that we do not want to reclaim memory if we are here re-entering
* the event loop since there is a busy Lua script running in timeout
* condition, to avoid mixing the propagation of scripts with the
* propagation of DELs due to eviction. */
* condition, to avoid mixing the propagation of scripts with the propagation
* of DELs due to eviction. */
if (server.maxmemory && !server.lua_timedout) {
int out_of_memory = freeMemoryIfNeededAndSafe() == C_ERR;
int out_of_memory = freeMemoryIfNeeded() == C_ERR;
/* freeMemoryIfNeeded may flush slave output buffers. This may result
* into a slave, that may be the active client, to be freed. */
if (server.current_client == NULL) return C_ERR;
/* It was impossible to free enough memory, and the command the client
* is trying to execute is denied during OOM conditions or the client
* is in MULTI/EXEC context? Error. */
if (out_of_memory &&
(c->cmd->flags & CMD_DENYOOM ||
(c->flags & CLIENT_MULTI && c->cmd->proc != execCommand))) {
* is trying to execute is denied during OOM conditions? Error. */
if ((c->cmd->flags & CMD_DENYOOM) && out_of_memory) {
flagTransaction(c);
addReply(c, shared.oomerr);
return C_OK;
@@ -3243,11 +3240,11 @@ sds genRedisInfoString(char *section) {
"allocator_frag_ratio:%.2f\r\n"
"allocator_frag_bytes:%zu\r\n"
"allocator_rss_ratio:%.2f\r\n"
"allocator_rss_bytes:%zd\r\n"
"allocator_rss_bytes:%zu\r\n"
"rss_overhead_ratio:%.2f\r\n"
"rss_overhead_bytes:%zd\r\n"
"rss_overhead_bytes:%zu\r\n"
"mem_fragmentation_ratio:%.2f\r\n"
"mem_fragmentation_bytes:%zd\r\n"
"mem_fragmentation_bytes:%zu\r\n"
"mem_not_counted_for_evict:%zu\r\n"
"mem_replication_backlog:%zu\r\n"
"mem_clients_slaves:%zu\r\n"
+4 -10
View File
@@ -654,9 +654,6 @@ typedef struct multiCmd {
typedef struct multiState {
multiCmd *commands; /* Array of MULTI commands */
int count; /* Total number of MULTI commands */
int cmd_flags; /* The accumulated command flags OR-ed together.
So if at least a command has a given flag, it
will be set in this field. */
int minreplicas; /* MINREPLICAS for synchronous replication */
time_t minreplicas_timeout; /* MINREPLICAS timeout as unixtime. */
} multiState;
@@ -867,11 +864,11 @@ struct redisMemOverhead {
float dataset_perc;
float peak_perc;
float total_frag;
ssize_t total_frag_bytes;
size_t total_frag_bytes;
float allocator_frag;
ssize_t allocator_frag_bytes;
size_t allocator_frag_bytes;
float allocator_rss;
ssize_t allocator_rss_bytes;
size_t allocator_rss_bytes;
float rss_extra;
size_t rss_extra_bytes;
size_t num_dbs;
@@ -954,9 +951,7 @@ struct redisServer {
size_t initial_memory_usage; /* Bytes used after initialization. */
int always_show_logo; /* Show logo even for non-stdout logging. */
/* Modules */
dict *moduleapi; /* Exported core APIs dictionary for modules. */
dict *sharedapi; /* Like moduleapi but containing the APIs that
modules share with each other. */
dict *moduleapi; /* Exported APIs dictionary for modules. */
list *loadmodule_queue; /* List of modules to load at startup. */
int module_blocked_pipe[2]; /* Pipe used to awake the event loop if a
client blocked on a module command needs
@@ -1704,7 +1699,6 @@ int zslLexValueLteMax(sds value, zlexrangespec *spec);
int getMaxmemoryState(size_t *total, size_t *logical, size_t *tofree, float *level);
size_t freeMemoryGetNotCountedMemory();
int freeMemoryIfNeeded(void);
int freeMemoryIfNeededAndSafe(void);
int processCommand(client *c);
void setupSignalHandlers(void);
struct redisCommand *lookupCommand(sds name);
+1 -1
View File
@@ -39,7 +39,7 @@
#include <errno.h> /* errno program_invocation_name program_invocation_short_name */
#if !defined(HAVE_SETPROCTITLE)
#if (defined __NetBSD__ || defined __FreeBSD__ || defined __OpenBSD__ || defined __DragonFly__)
#if (defined __NetBSD__ || defined __FreeBSD__ || defined __OpenBSD__)
#define HAVE_SETPROCTITLE 1
#else
#define HAVE_SETPROCTITLE 0
+34 -52
View File
@@ -37,7 +37,7 @@
* mark the entry as deleted, or having the same field as the "master"
* entry at the start of the listpack> */
#define STREAM_ITEM_FLAG_NONE 0 /* No special flags. */
#define STREAM_ITEM_FLAG_DELETED (1<<0) /* Entry is deleted. Skip it. */
#define STREAM_ITEM_FLAG_DELETED (1<<0) /* Entry is delted. Skip it. */
#define STREAM_ITEM_FLAG_SAMEFIELDS (1<<1) /* Same fields as master entry. */
void streamFreeCG(streamCG *cg);
@@ -165,7 +165,7 @@ int streamCompareID(streamID *a, streamID *b) {
* Returns the new entry ID populating the 'added_id' structure.
*
* If 'use_id' is not NULL, the ID is not auto-generated by the function,
* but instead the passed ID is used to add the new entry. In this case
* but instead the passed ID is uesd to add the new entry. In this case
* adding the entry may fail as specified later in this comment.
*
* The function returns C_OK if the item was added, this is always true
@@ -223,13 +223,13 @@ int streamAppendItem(stream *s, robj **argv, int64_t numfields, streamID *added_
*
* count and deleted just represent respectively the total number of
* entries inside the listpack that are valid, and marked as deleted
* (deleted flag in the entry flags set). So the total number of items
* (delted flag in the entry flags set). So the total number of items
* actually inside the listpack (both deleted and not) is count+deleted.
*
* The real entries will be encoded with an ID that is just the
* millisecond and sequence difference compared to the key stored at
* the radix tree node containing the listpack (delta encoding), and
* if the fields of the entry are the same as the master entry fields, the
* if the fields of the entry are the same as the master enty fields, the
* entry flags will specify this fact and the entry fields and number
* of fields will be omitted (see later in the code of this function).
*
@@ -486,7 +486,7 @@ int64_t streamTrimByLength(stream *s, size_t maxlen, int approx) {
* }
* streamIteratorStop(&myiterator); */
void streamIteratorStart(streamIterator *si, stream *s, streamID *start, streamID *end, int rev) {
/* Initialize the iterator and translates the iteration start/stop
/* Intialize the iterator and translates the iteration start/stop
* elements into a 128 big big-endian number. */
if (start) {
streamEncodeID(si->start_key,start);
@@ -564,7 +564,7 @@ int streamIteratorGetID(streamIterator *si, streamID *id, int64_t *numfields) {
si->lp_ele = lpLast(si->lp);
}
} else if (si->rev) {
/* If we are iterating in the reverse order, and this is not
/* If we are itereating in the reverse order, and this is not
* the first entry emitted for this listpack, then we already
* emitted the current entry, and have to go back to the previous
* one. */
@@ -751,7 +751,7 @@ void streamIteratorRemoveEntry(streamIterator *si, streamID *current) {
}
/* Stop the stream iterator. The only cleanup we need is to free the rax
* iterator, since the stream iterator itself is supposed to be stack
* itereator, since the stream iterator itself is supposed to be stack
* allocated. */
void streamIteratorStop(streamIterator *si) {
raxStop(&si->ri);
@@ -847,15 +847,11 @@ void streamPropagateGroupID(client *c, robj *key, streamCG *group, robj *groupna
decrRefCount(argv[4]);
}
/* Send the stream items in the specified range to the client 'c'. The range
* the client will receive is between start and end inclusive, if 'count' is
* non zero, no more than 'count' elements are sent.
*
* The 'end' pointer can be NULL to mean that we want all the elements from
* 'start' till the end of the stream. If 'rev' is non zero, elements are
* produced in reversed order from end to start.
*
* The function returns the number of entries emitted.
/* Send the specified range to the client 'c'. The range the client will
* receive is between start and end inclusive, if 'count' is non zero, no more
* than 'count' elemnets are sent. The 'end' pointer can be NULL to mean that
* we want all the elements from 'start' till the end of the stream. If 'rev'
* is non zero, elements are produced in reversed order from end to start.
*
* If group and consumer are not NULL, the function performs additional work:
* 1. It updates the last delivered ID in the group in case we are
@@ -867,15 +863,15 @@ void streamPropagateGroupID(client *c, robj *key, streamCG *group, robj *groupna
*
* The behavior may be modified passing non-zero flags:
*
* STREAM_RWR_NOACK: Do not create PEL entries, that is, the point "3" above
* STREAM_RWR_NOACK: Do not craete PEL entries, that is, the point "3" above
* is not performed.
* STREAM_RWR_RAWENTRIES: Do not emit array boundaries, but just the entries,
* and return the number of entries emitted as usually.
* This is used when the function is just used in order
* to emit data and there is some higher level logic.
*
* The final argument 'spi' (stream propagation info pointer) is a structure
* filled with information needed to propagate the command execution to AOF
* The final argument 'spi' (stream propagatino info pointer) is a structure
* filled with information needed to propagte the command execution to AOF
* and slaves, in the case a consumer group was passed: we need to generate
* XCLAIM commands to create the pending list into AOF/slaves in that case.
*
@@ -894,7 +890,6 @@ void streamPropagateGroupID(client *c, robj *key, streamCG *group, robj *groupna
#define STREAM_RWR_NOACK (1<<0) /* Do not create entries in the PEL. */
#define STREAM_RWR_RAWENTRIES (1<<1) /* Do not emit protocol for array
boundaries, just the entries. */
#define STREAM_RWR_HISTORY (1<<2) /* Only serve consumer local PEL. */
size_t streamReplyWithRange(client *c, stream *s, streamID *start, streamID *end, size_t count, int rev, streamCG *group, streamConsumer *consumer, int flags, streamPropInfo *spi) {
void *arraylen_ptr = NULL;
size_t arraylen = 0;
@@ -903,12 +898,15 @@ size_t streamReplyWithRange(client *c, stream *s, streamID *start, streamID *end
streamID id;
int propagate_last_id = 0;
/* If the client is asking for some history, we serve it using a
* different function, so that we return entries *solely* from its
* own PEL. This ensures each consumer will always and only see
* the history of messages delivered to it and not yet confirmed
/* If a group was passed, we check if the request is about messages
* never delivered so far (normally this happens when ">" ID is passed).
*
* If instead the client is asking for some history, we serve it
* using a different function, so that we return entries *solely*
* from its own PEL. This ensures each consumer will always and only
* see the history of messages delivered to it and not yet confirmed
* as delivered. */
if (group && (flags & STREAM_RWR_HISTORY)) {
if (group && streamCompareID(start,&group->last_id) <= 0) {
return streamReplyWithRangeFromConsumerPEL(c,s,start,end,count,
consumer);
}
@@ -1025,7 +1023,7 @@ size_t streamReplyWithRangeFromConsumerPEL(client *c, stream *s, streamID *start
if (end && memcmp(ri.key,end,ri.key_len) > 0) break;
streamID thisid;
streamDecodeID(ri.key,&thisid);
if (streamReplyWithRange(c,s,&thisid,&thisid,1,0,NULL,NULL,
if (streamReplyWithRange(c,s,&thisid,NULL,1,0,NULL,NULL,
STREAM_RWR_RAWENTRIES,NULL) == 0)
{
/* Note that we may have a not acknowledged entry in the PEL
@@ -1138,7 +1136,7 @@ invalid:
}
/* Wrapper for streamGenericParseIDOrReply() with 'strict' argument set to
* 0, to be used when - and + are acceptable IDs. */
* 0, to be used when - and + are accetable IDs. */
int streamParseIDOrReply(client *c, robj *o, streamID *id, uint64_t missing_seq) {
return streamGenericParseIDOrReply(c,o,id,missing_seq,0);
}
@@ -1262,7 +1260,7 @@ void xrangeGenericCommand(client *c, int rev) {
robj *o;
stream *s;
streamID startid, endid;
long long count = -1;
long long count = 0;
robj *startarg = rev ? c->argv[3] : c->argv[2];
robj *endarg = rev ? c->argv[2] : c->argv[3];
@@ -1289,13 +1287,7 @@ void xrangeGenericCommand(client *c, int rev) {
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.emptymultibulk)) == NULL
|| checkType(c,o,OBJ_STREAM)) return;
s = o->ptr;
if (count == 0) {
addReply(c,shared.nullmultibulk);
} else {
if (count == -1) count = 0;
streamReplyWithRange(c,s,&startid,&endid,count,rev,NULL,NULL,0,NULL);
}
streamReplyWithRange(c,s,&startid,&endid,count,rev,NULL,NULL,0,NULL);
}
/* XRANGE key start end [COUNT <n>] */
@@ -1472,10 +1464,8 @@ void xreadCommand(client *c) {
stream *s = o->ptr;
streamID *gt = ids+i; /* ID must be greater than this. */
int serve_synchronously = 0;
int serve_history = 0; /* True for XREADGROUP with ID != ">". */
/* Check if there are the conditions to serve the client
* synchronously. */
/* Check if there are the conditions to serve the client synchronously. */
if (groups) {
/* If the consumer is blocked on a group, we always serve it
* synchronously (serving its local history) if the ID specified
@@ -1484,7 +1474,6 @@ void xreadCommand(client *c) {
gt->seq != UINT64_MAX)
{
serve_synchronously = 1;
serve_history = 1;
} else {
/* We also want to serve a consumer in a consumer group
* synchronously in case the group top item delivered is smaller
@@ -1520,12 +1509,9 @@ void xreadCommand(client *c) {
if (groups) consumer = streamLookupConsumer(groups[i],
consumername->ptr,1);
streamPropInfo spi = {c->argv[i+streams_arg],groupname};
int flags = 0;
if (noack) flags |= STREAM_RWR_NOACK;
if (serve_history) flags |= STREAM_RWR_HISTORY;
streamReplyWithRange(c,s,&start,NULL,count,0,
groups ? groups[i] : NULL,
consumer, flags, &spi);
consumer, noack, &spi);
if (groups) server.dirty++;
}
}
@@ -1829,9 +1815,7 @@ NULL
}
}
/* XSETID <stream> <groupname> <id>
*
* Set the internal "last ID" of a stream. */
/* Set the internal "last ID" of a stream. */
void xsetidCommand(client *c) {
robj *o = lookupKeyWriteOrReply(c,c->argv[1],shared.nokeyerr);
if (o == NULL || checkType(c,o,OBJ_STREAM)) return;
@@ -1908,7 +1892,7 @@ void xackCommand(client *c) {
addReplyLongLong(c,acknowledged);
}
/* XPENDING <key> <group> [<start> <stop> <count> [<consumer>]]
/* XPENDING <key> <group> [<start> <stop> <count>] [<consumer>]
*
* If start and stop are omitted, the command just outputs information about
* the amount of pending messages for the key/group pair, together with
@@ -1937,7 +1921,6 @@ void xpendingCommand(client *c) {
if (c->argc >= 6) {
if (getLongLongFromObjectOrReply(c,c->argv[5],&count,NULL) == C_ERR)
return;
if (count < 0) count = 0;
if (streamParseIDOrReply(c,c->argv[3],&startid,0) == C_ERR)
return;
if (streamParseIDOrReply(c,c->argv[4],&endid,UINT64_MAX) == C_ERR)
@@ -2163,7 +2146,7 @@ void xclaimCommand(client *c) {
/* If we stopped because some IDs cannot be parsed, perhaps they
* are trailing options. */
mstime_t now = mstime();
time_t now = mstime();
streamID last_id = {0,0};
int propagate_last_id = 0;
for (; j < c->argc; j++) {
@@ -2282,9 +2265,8 @@ void xclaimCommand(client *c) {
if (justid) {
addReplyStreamID(c,&id);
} else {
size_t emitted = streamReplyWithRange(c,o->ptr,&id,&id,1,0,
NULL,NULL,STREAM_RWR_RAWENTRIES,NULL);
if (!emitted) addReply(c,shared.nullbulk);
streamReplyWithRange(c,o->ptr,&id,NULL,1,0,NULL,NULL,
STREAM_RWR_RAWENTRIES,NULL);
}
arraylen++;
+7 -5
View File
@@ -301,22 +301,24 @@ void mgetCommand(client *c) {
}
void msetGenericCommand(client *c, int nx) {
int j;
int j, busykeys = 0;
if ((c->argc % 2) == 0) {
addReplyError(c,"wrong number of arguments for MSET");
return;
}
/* Handle the NX flag. The MSETNX semantic is to return zero and don't
* set anything if at least one key alerady exists. */
* set nothing at all if at least one already key exists. */
if (nx) {
for (j = 1; j < c->argc; j += 2) {
if (lookupKeyWrite(c->db,c->argv[j]) != NULL) {
addReply(c, shared.czero);
return;
busykeys++;
}
}
if (busykeys) {
addReply(c, shared.czero);
return;
}
}
for (j = 1; j < c->argc; j += 2) {
+8 -6
View File
@@ -574,12 +574,12 @@ int zslParseLexRangeItem(robj *item, sds *dest, int *ex) {
switch(c[0]) {
case '+':
if (c[1] != '\0') return C_ERR;
*ex = 1;
*ex = 0;
*dest = shared.maxstring;
return C_OK;
case '-':
if (c[1] != '\0') return C_ERR;
*ex = 1;
*ex = 0;
*dest = shared.minstring;
return C_OK;
case '(':
@@ -652,8 +652,9 @@ int zslIsInLexRange(zskiplist *zsl, zlexrangespec *range) {
zskiplistNode *x;
/* Test for ranges that will always be empty. */
int cmp = sdscmplex(range->min,range->max);
if (cmp > 0 || (cmp == 0 && (range->minex || range->maxex)))
if (sdscmplex(range->min,range->max) > 1 ||
(sdscmp(range->min,range->max) == 0 &&
(range->minex || range->maxex)))
return 0;
x = zsl->tail;
if (x == NULL || !zslLexValueGteMin(x->ele,range))
@@ -926,8 +927,9 @@ int zzlIsInLexRange(unsigned char *zl, zlexrangespec *range) {
unsigned char *p;
/* Test for ranges that will always be empty. */
int cmp = sdscmplex(range->min,range->max);
if (cmp > 0 || (cmp == 0 && (range->minex || range->maxex)))
if (sdscmplex(range->min,range->max) > 1 ||
(sdscmp(range->min,range->max) == 0 &&
(range->minex || range->maxex)))
return 0;
p = ziplistIndex(zl,-2); /* Last element. */
+2 -37
View File
@@ -39,7 +39,6 @@
#include <float.h>
#include <stdint.h>
#include <errno.h>
#include <time.h>
#include "util.h"
#include "sha1.h"
@@ -48,7 +47,7 @@
int stringmatchlen(const char *pattern, int patternLen,
const char *string, int stringLen, int nocase)
{
while(patternLen && stringLen) {
while(patternLen) {
switch(pattern[0]) {
case '*':
while (pattern[1] == '*') {
@@ -171,22 +170,6 @@ int stringmatch(const char *pattern, const char *string, int nocase) {
return stringmatchlen(pattern,strlen(pattern),string,strlen(string),nocase);
}
/* Fuzz stringmatchlen() trying to crash it with bad input. */
int stringmatchlen_fuzz_test(void) {
char str[32];
char pat[32];
int cycles = 10000000;
int total_matches = 0;
while(cycles--) {
int strlen = rand() % sizeof(str);
int patlen = rand() % sizeof(pat);
for (int j = 0; j < strlen; j++) str[j] = rand() % 128;
for (int j = 0; j < patlen; j++) pat[j] = rand() % 128;
total_matches += stringmatchlen(pat, patlen, str, strlen, 0);
}
return total_matches;
}
/* Convert a string representing an amount of memory into the number of
* bytes, so for instance memtoll("1Gb") will return 1073741824 that is
* (1024*1024*1024).
@@ -622,7 +605,7 @@ void getRandomHexChars(char *p, size_t len) {
* already, this will be detected and handled correctly.
*
* The function does not try to normalize everything, but only the obvious
* case of one or more "../" appearing at the start of "filename"
* case of one or more "../" appearning at the start of "filename"
* relative path. */
sds getAbsolutePath(char *filename) {
char cwd[1024];
@@ -669,24 +652,6 @@ sds getAbsolutePath(char *filename) {
return abspath;
}
/*
* Gets the proper timezone in a more portable fashion
* i.e timezone variables are linux specific.
*/
unsigned long getTimeZone(void) {
#ifdef __linux__
return timezone;
#else
struct timeval tv;
struct timezone tz;
gettimeofday(&tv, &tz);
return tz.tz_minuteswest * 60UL;
#endif
}
/* Return true if the specified path is just a file basename without any
* relative or absolute path. This function just checks that no / or \
* character exists inside the specified path, that's enough in the
-2
View File
@@ -40,7 +40,6 @@
int stringmatchlen(const char *p, int plen, const char *s, int slen, int nocase);
int stringmatch(const char *p, const char *s, int nocase);
int stringmatchlen_fuzz_test(void);
long long memtoll(const char *p, int *err);
uint32_t digits10(uint64_t v);
uint32_t sdigits10(int64_t v);
@@ -51,7 +50,6 @@ int string2ld(const char *s, size_t slen, long double *dp);
int d2string(char *buf, size_t len, double value);
int ld2string(char *buf, size_t len, long double value, int humanfriendly);
sds getAbsolutePath(char *filename);
unsigned long getTimeZone(void);
int pathIsBaseName(char *path);
#ifdef REDIS_TEST
+1 -1
View File
@@ -1 +1 @@
#define REDIS_VERSION "999.999.999"
#define REDIS_VERSION "5.0.0"
+1
View File
@@ -275,6 +275,7 @@ start_server {tags {"repl"}} {
start_server {} {
test "Master stream is correctly processed while the replica has a script in -BUSY state" {
set slave [srv 0 client]
puts [srv 0 port]
$slave config set lua-time-limit 500
$slave slaveof $master_host $master_port
+1 -1
View File
@@ -4,7 +4,7 @@ set ::valgrind_errors {}
proc start_server_error {config_file error} {
set err {}
append err "Can't start the Redis server\n"
append err "Cant' start the Redis server\n"
append err "CONFIGURATION:"
append err [exec cat $config_file]
append err "\nERROR:"
-8
View File
@@ -91,14 +91,6 @@ proc wait_for_sync r {
}
}
proc wait_for_ofs_sync {r1 r2} {
wait_for_condition 50 100 {
[status $r1 master_repl_offset] eq [status $r2 master_repl_offset]
} else {
fail "replica didn't sync in time"
}
}
# Random integer between 0 and max (excluded).
proc randomInt {max} {
expr {int(rand()*$max)}
-1
View File
@@ -90,7 +90,6 @@ start_server {tags {"defrag"}} {
test "Active defrag big keys" {
r flushdb
r config resetstat
r config set save "" ;# prevent bgsave from interfereing with save below
r config set activedefrag no
r config set active-defrag-max-scan-fields 1000
r config set active-defrag-threshold-lower 5
-89
View File
@@ -108,93 +108,6 @@ start_server {
assert {$c == 5}
}
test {XREADGROUP will not report data on empty history. Bug #5577} {
r del events
r xadd events * a 1
r xadd events * b 2
r xadd events * c 3
r xgroup create events mygroup 0
# Current local PEL should be empty
set res [r xpending events mygroup - + 10]
assert {[llength $res] == 0}
# So XREADGROUP should read an empty history as well
set res [r xreadgroup group mygroup myconsumer count 3 streams events 0]
assert {[llength [lindex $res 0 1]] == 0}
# We should fetch all the elements in the stream asking for >
set res [r xreadgroup group mygroup myconsumer count 3 streams events >]
assert {[llength [lindex $res 0 1]] == 3}
# Now the history is populated with three not acked entries
set res [r xreadgroup group mygroup myconsumer count 3 streams events 0]
assert {[llength [lindex $res 0 1]] == 3}
}
test {XREADGROUP history reporting of deleted entries. Bug #5570} {
r del mystream
r XGROUP CREATE mystream mygroup $ MKSTREAM
r XADD mystream 1 field1 A
r XREADGROUP GROUP mygroup myconsumer STREAMS mystream >
r XADD mystream MAXLEN 1 2 field1 B
r XREADGROUP GROUP mygroup myconsumer STREAMS mystream >
# Now we have two pending entries, however one should be deleted
# and one should be ok (we should only see "B")
set res [r XREADGROUP GROUP mygroup myconsumer STREAMS mystream 0-1]
assert {[lindex $res 0 1 0] == {1-0 {}}}
assert {[lindex $res 0 1 1] == {2-0 {field1 B}}}
}
test {XCLAIM can claim PEL items from another consumer} {
# Add 3 items into the stream, and create a consumer group
r del mystream
set id1 [r XADD mystream * a 1]
set id2 [r XADD mystream * b 2]
set id3 [r XADD mystream * c 3]
r XGROUP CREATE mystream mygroup 0
# Client 1 reads item 1 from the stream without acknowledgements.
# Client 2 then claims pending item 1 from the PEL of client 1
set reply [
r XREADGROUP GROUP mygroup client1 count 1 STREAMS mystream >
]
assert {[llength [lindex $reply 0 1 0 1]] == 2}
assert {[lindex $reply 0 1 0 1] eq {a 1}}
r debug sleep 0.2
set reply [
r XCLAIM mystream mygroup client2 10 $id1
]
assert {[llength [lindex $reply 0 1]] == 2}
assert {[lindex $reply 0 1] eq {a 1}}
# Client 1 reads another 2 items from stream
r XREADGROUP GROUP mygroup client1 count 2 STREAMS mystream >
r debug sleep 0.2
# Delete item 2 from the stream. Now client 1 has PEL that contains
# only item 3. Try to use client 2 to claim the deleted item 2
# from the PEL of client 1, this should return nil
r XDEL mystream $id2
set reply [
r XCLAIM mystream mygroup client2 10 $id2
]
assert {[llength $reply] == 1}
assert_equal "" [lindex $reply 0]
# Delete item 3 from the stream. Now client 1 has PEL that is empty.
# Try to use client 2 to claim the deleted item 3 from the PEL
# of client 1, this should return nil
r debug sleep 0.2
r XDEL mystream $id3
set reply [
r XCLAIM mystream mygroup client2 10 $id3
]
assert {[llength $reply] == 1}
assert_equal "" [lindex $reply 0]
}
start_server {} {
set master [srv -1 client]
set master_host [srv -1 host]
@@ -231,8 +144,6 @@ start_server {
}
}
wait_for_ofs_sync $master $slave
# Turn slave into master
$slave slaveof no one
+1 -17
View File
@@ -388,7 +388,7 @@ start_server {tags {"zset"}} {
0 omega}
}
test "ZRANGEBYLEX/ZREVRANGEBYLEX/ZLEXCOUNT basics" {
test "ZRANGEBYLEX/ZREVRANGEBYLEX/ZCOUNT basics" {
create_default_lex_zset
# inclusive range
@@ -416,22 +416,6 @@ start_server {tags {"zset"}} {
assert_equal {} [r zrevrangebylex zset \[elez \[elex]
assert_equal {} [r zrevrangebylex zset (hill (omega]
}
test "ZLEXCOUNT advanced" {
create_default_lex_zset
assert_equal 9 [r zlexcount zset - +]
assert_equal 0 [r zlexcount zset + -]
assert_equal 0 [r zlexcount zset + \[c]
assert_equal 0 [r zlexcount zset \[c -]
assert_equal 8 [r zlexcount zset \[bar +]
assert_equal 5 [r zlexcount zset \[bar \[foo]
assert_equal 4 [r zlexcount zset \[bar (foo]
assert_equal 4 [r zlexcount zset (bar \[foo]
assert_equal 3 [r zlexcount zset (bar (foo]
assert_equal 5 [r zlexcount zset - (foo]
assert_equal 1 [r zlexcount zset (maxstring +]
}
test "ZRANGEBYSLEX with LIMIT" {
create_default_lex_zset