Compare commits

..
Author SHA1 Message Date
antirez 593ce5b206 Test: processing of master stream in slave -BUSY state.
See #5297.
2018-08-31 16:43:41 +02:00
antirez f2739ca9c4 After slave Lua script leaves busy state, re-process the master buffer.
Technically speaking we don't really need to put the master client in
the clients that need to be processed, since in practice the PING
commands from the master will take care, however it is conceptually more
sane to do so.
2018-08-31 16:07:03 +02:00
antirez e8d35809c4 While the slave is busy, just accumulate master input.
Processing command from the master while the slave is in busy state is
not correct, however we cannot, also, just reply -BUSY to the
replication stream commands from the master. The correct solution is to
stop processing data from the master, but just accumulate the stream
into the buffers and resume the processing later.

Related to #5297.
2018-08-30 17:52:04 +02:00
antirez 4b68dbcbd7 Allow scripts to timeout even if from the master instance.
However the master scripts will be impossible to kill.

Related to #5297.
2018-08-30 11:46:42 +02:00
antirez 17b0d99d9f Allow scripts to timeout on slaves as well.
See reasoning in #5297.
2018-08-29 18:14:46 +02:00
Salvatore SanfilippoandGitHub 476eea95da Merge pull request #4216 from lamby/did-not-received-typos
Correct "did not received" -> "did not receive" typos/grammar.
2018-08-29 16:18:11 +02:00
Salvatore SanfilippoandGitHub bd9259d12d Merge pull request #5282 from soloestoy/remove-duplicate-bind-in-sentinel.conf
remove duplicate bind in sentinel.conf
2018-08-29 16:13:42 +02:00
Salvatore SanfilippoandGitHub ed5cc77ce0 Merge pull request #5296 from soloestoy/command-script-flag2
Supplement to PR #4835, just take info/memory/command as random commands
2018-08-29 12:26:01 +02:00
zhaozhao.zz 7d39c149c4 Supplement to PR #4835, just take info/memory/command as random commands 2018-08-29 18:23:05 +08:00
Salvatore SanfilippoandGitHub 0e21efdb1c Merge pull request #4835 from soloestoy/command-script-flag
some commands' flags should be set correctly, issue #4834
2018-08-29 12:13:50 +02:00
zhaozhao.zz 32844178ac some commands' flags should be set correctly, issue #4834 2018-08-29 18:07:01 +08:00
Salvatore SanfilippoandGitHub 5b4bec9d33 Merge pull request #5265 from oranagra/stabilize_tests
Fix unstable tests on slow machines.
2018-08-27 13:19:31 +02:00
antirez 5941022b52 Document slave-ignore-maxmemory in redis.conf. 2018-08-27 12:34:29 +02:00
antirez e245a2046a Make slave-ignore-maxmemory configurable. 2018-08-27 12:27:17 +02:00
antirez 067647a783 Introduce repl_slave_ignore_maxmemory flag internally.
Note: this breaks backward compatibility with Redis 4, since now slaves
by default are exact copies of masters and do not try to evict keys
independently.
2018-08-27 12:20:27 +02:00
antirez abf52c7cf4 Better variable meaning in processCommand(). 2018-08-27 12:17:34 +02:00
antirez 66b5afdaa4 Re-apply rebased #2358. 2018-08-27 12:17:14 +02:00
antirez c241f51607 Fix build errors caused by #2358. 2018-08-27 12:15:55 +02:00
zhaozhao.zz e711e6f614 remove duplicate bind in sentinel.conf 2018-08-27 12:07:24 +08:00
Salvatore SanfilippoandGitHub 19880ab851 Merge pull request #5248 from soloestoy/rewrite-brpoplpush
rewrite BRPOPLPUSH as RPOPLPUSH to propagate
2018-08-26 16:31:24 +02:00
Salvatore SanfilippoandGitHub 80e1695652 Merge pull request #5244 from soloestoy/optimize-pipeline
pipeline: do not sdsrange querybuf unless all commands processed
2018-08-26 16:30:49 +02:00
Chris Lamb 132be8aed5 Correct "did not received" -> "did not receive" typos/grammar. 2018-08-26 14:45:39 +02:00
Salvatore SanfilippoandGitHub 46d89a9abb Merge pull request #2992 from lamby/source-date-epoch
Use SOURCE_DATE_EPOCH over unreproducible uname + date calls.
2018-08-26 11:25:44 +02:00
Salvatore SanfilippoandGitHub 590717782f Merge pull request #2358 from lamby/config-set-maxmemory-grammar
Tidy grammar in CONFIG SET maxmemory warning.
2018-08-26 11:23:41 +02:00
Salvatore SanfilippoandGitHub 8d519a85f4 Merge pull request #2292 from lamby/sentinel-conf-defaults
Make some defaults explicit in the sentinel.conf for package maintainers
2018-08-26 11:19:51 +02:00
Chris LambandGitHub f63e81c202 Merge branch 'unstable' into config-set-maxmemory-grammar 2018-08-25 21:49:29 +02:00
zhaozhao.zz f2ad89a314 networking: make setProtocolError simple and clear
Function setProtocolError just records proctocol error
details in server log, set client as CLIENT_CLOSE_AFTER_REPLY.
It doesn't care about querybuf sdsrange, because we
will do it after procotol parsing.
2018-08-23 12:21:28 +08:00
Oran Agra c8452ab005 Fix unstable tests on slow machines.
Few tests had borderline thresholds that were adjusted.

The slave buffers test had two issues, preventing the slave buffer from growing:
1) the slave didn't necessarily go to sleep on time, or woke up too early,
   now using SIGSTOP to make sure it goes to sleep exactly when we want.
2) the master disconnected the slave on timeout
2018-08-21 11:46:07 +03:00
zhaozhao.zz 9a65f9cd3e block: format code 2018-08-14 20:59:32 +08:00
zhaozhao.zz 8a1219d93b block: rewrite BRPOPLPUSH as RPOPLPUSH to propagate 2018-08-14 20:58:58 +08:00
zhaozhao.zz ef2a95c461 networking: just move qb_pos instead of sdsrange in processInlineBuffer 2018-08-14 14:50:37 +08:00
zhaozhao.zz e623bd22ba networking: just return C_OK if multibulk processing saw a <= 0 length. 2018-08-14 13:55:30 +08:00
zhaozhao.zz b89302c462 adjust qbuf to 26 in test case for client list 2018-08-14 00:57:22 +08:00
zhaozhao.zz 14c4ddb5a6 pipeline: do not sdsrange querybuf unless all commands processed
This is an optimization for processing pipeline, we discussed a
problem in issue #5229: clients may be paused if we apply `CLIENT
PAUSE` command, and then querybuf may grow too large, the cost of
memmove in sdsrange after parsing a completed command will be
horrible. The optimization is that parsing all commands in queyrbuf
, after that we can just call sdsrange only once.
2018-08-14 00:43:42 +08:00
Salvatore SanfilippoandGitHub 39c70e728b Merge pull request #5146 from 0xtonyxia/fix-xclaim-id-parse
Streams: ID of xclaim command should start from the sixth argument.
2018-08-03 13:45:27 +02:00
Salvatore SanfilippoandGitHub 7b5e7f3ed0 Merge pull request #5151 from shenlongxing/fix-stream
Fix stream command paras
2018-08-03 13:39:32 +02:00
shenlongxing fe56c67405 Fix stream command paras 2018-08-03 19:01:15 +08:00
antirez 0ce8323c0d Fix AOF comment to report the current behavior.
Realted to #5201.
2018-08-03 12:46:06 +02:00
antirez 3d56311f0c Test suite: add --loop option.
Very useful with --stop in order to catch heisenbugs.
2018-08-02 19:07:17 +02:00
antirez 26897c03e2 Test suite: new --stop option.
It pauses the test execution once the first failure is found.
2018-08-02 18:49:49 +02:00
antirez d506334b67 Test: new sorted set skiplist order consistency.
This should be able to find new bugs and regressions about the new
sorted set update function when ZADD is used to update an element
already existing.

The test is able to find the bug fixed at 2f282aee immediately.
2018-08-02 14:15:53 +02:00
antirez 2f282aee0b Fix zslUpdateScore() edge case.
When the element new score is the same of prev/next node, the
lexicographical order kicks in, so we can safely update the node in
place only when the new score is strictly between the adjacent nodes
but never equal to one of them.

Technically speaking we could do extra checks to make sure that even if the
score is the same as one of the adjacent nodes, we can still update on
place, but this rarely happens, so probably not a good deal to make it
more complex.

Related to #5179.
2018-08-01 19:04:53 +02:00
antirez 29226a3919 More commenting of zslUpdateScore(). 2018-08-01 18:54:15 +02:00
antirez 12ff0c0d79 Explain what's the point of zslUpdateScore() in top comment. 2018-08-01 18:53:09 +02:00
antirez e3e94ec35c Remove old commented zslUpdateScore() from source. 2018-08-01 18:50:49 +02:00
antirez 0b800c4332 Optimize zslUpdateScore() as asked in #5179. 2018-08-01 18:50:31 +02:00
antirez 6dd0d6f7dd zsetAdd() refactored adding zslUpdateScore(). 2018-08-01 18:27:56 +02:00
antirez 2f2987ffc5 Streams IDs parsing refactoring.
Related to #5184.
2018-07-31 18:08:52 +02:00
Salvatore SanfilippoandGitHub 187fa78637 Merge pull request #5184 from 0xtonyxia/streams-id-opt
Streams: rearrange the usage of '-' and '+' IDs in stream commands.
2018-07-31 17:58:13 +02:00
antirez 7a30be1237 Minor improvements to PR #5187. 2018-07-31 17:30:12 +02:00
Salvatore SanfilippoandGitHub 7b41701eb8 Merge pull request #5187 from oranagra/testsuite_improvements
Testsuite improvements
2018-07-31 17:20:09 +02:00
antirez e245ed9a44 Cluster cron announce IP minor refactoring. 2018-07-31 17:13:03 +02:00
Salvatore SanfilippoandGitHub 093adb7ef9 Merge pull request #5191 from shenlongxing/fix-memory-leak
Fix cluster-announce-ip memory leak
2018-07-31 17:11:16 +02:00
antirez b0392e75ec Tranfer -> transfer typo fixed. 2018-07-31 16:41:33 +02:00
Salvatore SanfilippoandGitHub 92b39a0abf Merge pull request #5189 from soloestoy/refactor-dbOverwrite
refactor dbOverwrite to make lazyfree work
2018-07-31 16:40:35 +02:00
antirez db693be00d Refactoring: replace low-level checks with writeCommandsDeniedByDiskError(). 2018-07-31 13:16:43 +02:00
antirez 11dd3f4b03 Fix writeCommandsDeniedByDiskError() inverted return value. 2018-07-31 13:14:24 +02:00
antirez 0e49938b68 Better top comment for writeCommandsDeniedByDiskError(). 2018-07-31 13:10:37 +02:00
antirez 5401fe7fb9 Introduce writeCommandsDeniedByDiskError(). 2018-07-31 13:09:38 +02:00
shenlongxing 35ca670060 Fix cluster-announce-ip memory leak 2018-07-31 16:01:44 +08:00
zhaozhao.zz fddeeae724 refactor dbOverwrite to make lazyfree work 2018-07-31 12:07:57 +08:00
Oran Agra 7829287691 test suite conveniency improvements
* allowing --single to be repeated
* adding --only so that only a specific test inside a unit can be run
* adding --skiptill useful to resume a test that crashed passed the problematic unit.
  useful together with --clients 1
* adding --skipfile to use a file containing list of tests names to skip
* printing the names of the tests that are skiped by skipfile or denytags
* adding --config to add config file options from command line
2018-07-30 19:13:15 +03:00
Salvatore SanfilippoandGitHub ac3c012a7f Merge pull request #5153 from trevor211/fixLuaScript
Consider aof write error as well as rdb in lua script.
2018-07-30 18:10:06 +02:00
Salvatore Sanfilippo 4ebb4d54c6 Merge pull request #5168 from rpv-tomsk/issue-5033
INFO CPU: higher precision of reported values
2018-07-30 18:06:54 +02:00
Pavel Rochnyackandantirez 1e394b73b7 INFO CPU: higher precision of reported values
Closes: #5033
2018-07-30 18:06:22 +02:00
Salvatore SanfilippoandGitHub 1465755e38 Merge pull request #5183 from gkorland/patch-1
Few typo fixes
2018-07-30 17:59:12 +02:00
antirez 3c19ae941d Add year in log.
User: "is there a reason why redis server logs  are missing the year in
the "date time"?"

Me: "I guess I did not imagine it would be stable enough to run for
several years".
2018-07-30 17:42:30 +02:00
dejun.xdj 6491717c88 Streams: rearrange the usage of '-' and '+' IDs in stream commands. 2018-07-30 21:33:01 +08:00
dejun.xdj d6f5ec6f09 Streams: add mmid_supp argument in streamParseIDOrReply().
If 'mmid_supp' is set to 0, "-" and "+" will be
treated as an invalid ID.
2018-07-30 21:32:07 +08:00
Guy KorlandandGitHub acaa18f1d1 Few typo fixes 2018-07-30 16:18:56 +03:00
antirez 15be174cf1 Document dynamic-hz in the example redis.conf. 2018-07-30 13:50:15 +02:00
antirez be28050ac0 Make dynamic hz actually configurable. 2018-07-30 13:44:52 +02:00
antirez c426d85c4c Control dynamic HZ via server configuration. 2018-07-30 13:37:30 +02:00
antirez 4e9c30a6ca Merge branch 'dynamic-hz' into unstable 2018-07-30 13:31:23 +02:00
Oran Agra 4a30adde31 add DEBUG LOG, to to assist test suite debugging 2018-07-30 12:23:51 +03:00
antirez fd174cca23 Remove useless conditional from emptyDb().
Related to #4852.
2018-07-25 16:34:57 +02:00
antirez 1c2352f0dd Make emptyDb() change introduced in #4852 simpler to read. 2018-07-25 16:32:52 +02:00
Salvatore SanfilippoandGitHub fe408ad23f Merge pull request #4852 from soloestoy/optimize-flushall
optimize flushdb, avoid useless loops
2018-07-25 16:30:11 +02:00
zhaozhao.zz dd071ff6c9 optimize flushdb, avoid useless loops 2018-07-25 18:13:34 +08:00
Salvatore SanfilippoandGitHub 73cbc33a0a Merge pull request #5133 from soloestoy/fix-xdel-memory-leak
Streams: fix xdel memory leak
2018-07-24 18:57:05 +02:00
zhaozhao.zz eb0e5fe4e3 Streams: fix xdel memory leak 2018-07-25 00:06:27 +08:00
antirez 0bdeb861f4 Example the magic +1 in migrateCommand().
Related to #5154.
2018-07-24 17:31:43 +02:00
antirez 53d46fa712 Make changes of PR #5154 hopefully simpler. 2018-07-24 17:27:43 +02:00
Salvatore SanfilippoandGitHub e52a036ac6 Merge pull request #5154 from trevor211/optimizeMigrate
Do not migrate already expired keys.
2018-07-24 17:24:37 +02:00
antirez 8513f31be0 Streams: refactoring of next entry seek in the iterator.
After #5161 the code could be made a bit more obvious for newcomers.
2018-07-24 11:07:27 +02:00
Salvatore SanfilippoandGitHub 41a886605a Merge pull request #5161 from soloestoy/optimize-stream-iterator
Streams: skip master fileds only when we are going forward in streamIteratorGetID
2018-07-24 10:59:55 +02:00
Salvatore SanfilippoandGitHub dfc82fca54 Merge pull request #5163 from oranagra/fix_slave_buffer_test
fix slave buffer test suite false positives
2018-07-24 10:28:48 +02:00
antirez f4ac796c34 string2ll(): better commenting. 2018-07-24 10:27:20 +02:00
Oran Agra d4ae76d1a6 fix slave buffer test suite false positives
it looks like on slow machines we're getting:
[err]: slave buffer are counted correctly in tests/unit/maxmemory.tcl
Expected condition '$slave_buf > 2*1024*1024' to be true (16914 > 2*1024*1024)

this is a result of the slave waking up too early and eating the
slave buffer before the traffic and the test ends.
2018-07-24 11:24:27 +03:00
antirez b9a8e56439 Merge branch 'removeRedundantCheck' of https://github.com/dsomeshwar/redis into unstable 2018-07-24 10:24:24 +02:00
antirez b28cbe90e1 Restore string2ll() to original version.
See PR #5157.
2018-07-24 10:22:12 +02:00
antirez 1a8d7cd9c2 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2018-07-24 00:21:49 +02:00
antirez e8f5d21940 string2ll(): remove duplicated check for special case.
Related to #5157. The PR author correctly indentified that the check was
duplicated, but removing the second one introduces a bug that was fixed
in the past (hence the duplication). Instead we can remove the first
instance of the check without issues.
2018-07-24 00:20:20 +02:00
antirez 1daee8c748 string2ll(): test for NULL pointer in all the cases. 2018-07-24 00:18:39 +02:00
Salvatore SanfilippoandGitHub ab1d5fe11a Merge pull request #5160 from oranagra/fix_zmalloc_usable
fix recursion typo in zmalloc_usable
2018-07-24 00:07:59 +02:00
antirez 9a375e5408 Change 42 to 1000 as warning level for cached scripts.
Related to #4883.
2018-07-23 18:44:58 +02:00
Salvatore SanfilippoandGitHub 445a2a2b1b Merge pull request #4883 from itamarhaber/lua_scripts-in-info-memory
Adds memory information about the scripts' cache to INFO
2018-07-23 18:43:05 +02:00
antirez b65ddfb16a Dynamic HZ: adapt cron frequency to number of clients. 2018-07-23 14:21:04 +02:00
antirez e6ea603ad3 Dynamic HZ: separate hz from the configured hz.
This way we can remember what the user configured HZ is, but change the
actual HZ dynamically if needed in the dynamic HZ feature
implementation.
2018-07-23 14:13:58 +02:00
Itamar Haber 993716c351 Adds Lua overheads to MEMORY STATS, smartens the MEMORY DOCTOR 2018-07-22 21:16:00 +03:00
zhaozhao.zz 9674ede85a Streams: skip master fileds only when we are going forward in streamIteratorGetID 2018-07-22 17:52:09 +08:00
Oran Agra 780815dd6e fix recursion typo in zmalloc_usable 2018-07-22 10:17:35 +03:00
dsomeshwar bb5b8b3a6f removing redundant check 2018-07-21 23:42:08 +05:30
WuYunlong 4017a11144 Do not migrate already expired keys. 2018-07-21 10:00:32 +08:00
WuYunlong dd760bd5e6 Consider aof write error as well as rdb in lua script. 2018-07-21 08:48:51 +08:00
dejun.xdj de5ca516ae Streams: ID of xclaim command starts from the sixth argument. 2018-07-19 20:48:08 +08:00
Itamar Haber 49890c8ee9 Adds memory information about the script's cache to INFO
Implementation notes: as INFO is "already broken", I didn't want to break it further. Instead of computing the server.lua_script dict size on every call, I'm keeping a running sum of the body's length and dict overheads.

This implementation is naive as it **does not** take into consideration dict rehashing, but that inaccuracy pays off in speed ;)

Demo time:

```bash
$ redis-cli info memory | grep "script"
used_memory_scripts:96
used_memory_scripts_human:96B
number_of_cached_scripts:0
$ redis-cli eval "" 0 ; redis-cli info memory | grep "script"
(nil)
used_memory_scripts:120
used_memory_scripts_human:120B
number_of_cached_scripts:1
$ redis-cli script flush ; redis-cli info memory | grep "script"
OK
used_memory_scripts:96
used_memory_scripts_human:96B
number_of_cached_scripts:0
$ redis-cli eval "return('Hello, Script Cache :)')" 0 ; redis-cli info memory | grep "script"
"Hello, Script Cache :)"
used_memory_scripts:152
used_memory_scripts_human:152B
number_of_cached_scripts:1
$ redis-cli eval "return redis.sha1hex(\"return('Hello, Script Cache :)')\")" 0 ; redis-cli info memory | grep "script"
"1be72729d43da5114929c1260a749073732dc822"
used_memory_scripts:232
used_memory_scripts_human:232B
number_of_cached_scripts:2
✔ 19:03:54 redis [lua_scripts-in-info-memory L ✚…⚑] $ redis-cli evalsha 1be72729d43da5114929c1260a749073732dc822 0
"Hello, Script Cache :)"
```
2018-04-30 19:33:01 +03:00
Chris Lamb 82ae4f30ed Use SOURCE_DATE_EPOCH over unreproducible uname + date calls.
See <https://reproducible-builds.org/specs/source-date-epoch/> for more
details.

Signed-off-by: Chris Lamb <chris@chris-lamb.co.uk>
2016-04-29 16:45:53 +01:00
Chris Lamb eaeba1b2c8 Tidy grammar in CONFIG SET maxmemory warning.
Signed-off-by: Chris Lamb <chris@chris-lamb.co.uk>
2015-02-04 18:37:34 +00:00
Chris Lamb 3447062d5e Make some defaults explicit in the sentinel.conf for package maintainers
This may look a little pointless (and it is a complete no-op change here)
but as package maintainers need to modify these lines to actually
daemonize (etc. etc) but it's far preferable if the diff is restricted to
actually changing just that bit, not adding docs, etc. The less diff the
better, in general.

Signed-off-by: Chris Lamb <chris@chris-lamb.co.uk>
2015-01-16 09:03:00 +00:00
66 changed files with 749 additions and 5127 deletions
+11 -2354
View File
File diff suppressed because it is too large Load Diff
+9 -9
View File
@@ -119,7 +119,7 @@ parameter (the path of the configuration file):
It is possible to alter the Redis configuration by passing parameters directly
as options using the command line. Examples:
% ./redis-server --port 9999 --replicaof 127.0.0.1 6379
% ./redis-server --port 9999 --slaveof 127.0.0.1 6379
% ./redis-server /etc/redis/6379.conf --loglevel debug
All the options in redis.conf are also supported as options using the command
@@ -216,7 +216,7 @@ Inside the root are the following important directories:
* `src`: contains the Redis implementation, written in C.
* `tests`: contains the unit tests, implemented in Tcl.
* `deps`: contains libraries Redis uses. Everything needed to compile Redis is inside this directory; your system just needs to provide `libc`, a POSIX compatible interface and a C compiler. Notably `deps` contains a copy of `jemalloc`, which is the default allocator of Redis under Linux. Note that under `deps` there are also things which started with the Redis project, but for which the main repository is not `antirez/redis`. An exception to this rule is `deps/geohash-int` which is the low level geocoding library used by Redis: it originated from a different project, but at this point it diverged so much that it is developed as a separated entity directly inside the Redis repository.
* `deps`: contains libraries Redis uses. Everything needed to compile Redis is inside this directory; your system just needs to provide `libc`, a POSIX compatible interface and a C compiler. Notably `deps` contains a copy of `jemalloc`, which is the default allocator of Redis under Linux. Note that under `deps` there are also things which started with the Redis project, but for which the main repository is not `anitrez/redis`. An exception to this rule is `deps/geohash-int` which is the low level geocoding library used by Redis: it originated from a different project, but at this point it diverged so much that it is developed as a separated entity directly inside the Redis repository.
There are a few more directories but they are not very important for our goals
here. We'll focus mostly on `src`, where the Redis implementation is contained,
@@ -227,7 +227,7 @@ of complexity incrementally.
Note: lately Redis was refactored quite a bit. Function names and file
names have been changed, so you may find that this documentation reflects the
`unstable` branch more closely. For instance in Redis 3.0 the `server.c`
and `server.h` files were named `redis.c` and `redis.h`. However the overall
and `server.h` files were named to `redis.c` and `redis.h`. However the overall
structure is the same. Keep in mind that all the new developments and pull
requests should be performed against the `unstable` branch.
@@ -245,7 +245,7 @@ A few important fields in this structure are:
* `server.db` is an array of Redis databases, where data is stored.
* `server.commands` is the command table.
* `server.clients` is a linked list of clients connected to the server.
* `server.master` is a special client, the master, if the instance is a replica.
* `server.master` is a special client, the master, if the instance is a slave.
There are tons of other fields. Most fields are commented directly inside
the structure definition.
@@ -323,7 +323,7 @@ Inside server.c you can find code that handles other vital things of the Redis s
networking.c
---
This file defines all the I/O functions with clients, masters and replicas
This file defines all the I/O functions with clients, masters and slaves
(which in Redis are just special clients):
* `createClient()` allocates and initializes a new client.
@@ -390,16 +390,16 @@ replication.c
This is one of the most complex files inside Redis, it is recommended to
approach it only after getting a bit familiar with the rest of the code base.
In this file there is the implementation of both the master and replica role
In this file there is the implementation of both the master and slave role
of Redis.
One of the most important functions inside this file is `replicationFeedSlaves()` that writes commands to the clients representing replica instances connected
to our master, so that the replicas can get the writes performed by the clients:
One of the most important functions inside this file is `replicationFeedSlaves()` that writes commands to the clients representing slave instances connected
to our master, so that the slaves can get the writes performed by the clients:
this way their data set will remain synchronized with the one in the master.
This file also implements both the `SYNC` and `PSYNC` commands that are
used in order to perform the first synchronization between masters and
replicas, or to continue the replication after a disconnection.
slaves, or to continue the replication after a disconnection.
Other C files
---
+129 -134
View File
@@ -264,64 +264,59 @@ dir ./
################################# REPLICATION #################################
# Master-Replica replication. Use replicaof to make a Redis instance a copy of
# Master-Slave replication. Use slaveof to make a Redis instance a copy of
# another Redis server. A few things to understand ASAP about Redis replication.
#
# +------------------+ +---------------+
# | Master | ---> | Replica |
# | (receive writes) | | (exact copy) |
# +------------------+ +---------------+
#
# 1) Redis replication is asynchronous, but you can configure a master to
# stop accepting writes if it appears to be not connected with at least
# a given number of replicas.
# 2) Redis replicas are able to perform a partial resynchronization with the
# a given number of slaves.
# 2) Redis slaves are able to perform a partial resynchronization with the
# master if the replication link is lost for a relatively small amount of
# time. You may want to configure the replication backlog size (see the next
# sections of this file) with a sensible value depending on your needs.
# 3) Replication is automatic and does not need user intervention. After a
# network partition replicas automatically try to reconnect to masters
# network partition slaves automatically try to reconnect to masters
# and resynchronize with them.
#
# replicaof <masterip> <masterport>
# slaveof <masterip> <masterport>
# If the master is password protected (using the "requirepass" configuration
# directive below) it is possible to tell the replica to authenticate before
# directive below) it is possible to tell the slave to authenticate before
# starting the replication synchronization process, otherwise the master will
# refuse the replica request.
# refuse the slave request.
#
# masterauth <master-password>
# When a replica loses its connection with the master, or when the replication
# is still in progress, the replica can act in two different ways:
# When a slave loses its connection with the master, or when the replication
# is still in progress, the slave can act in two different ways:
#
# 1) if replica-serve-stale-data is set to 'yes' (the default) the replica will
# 1) if slave-serve-stale-data is set to 'yes' (the default) the slave will
# still reply to client requests, possibly with out of date data, or the
# data set may just be empty if this is the first synchronization.
#
# 2) if replica-serve-stale-data is set to 'no' the replica will reply with
# 2) if slave-serve-stale-data is set to 'no' the slave will reply with
# an error "SYNC with master in progress" to all the kind of commands
# but to INFO, replicaOF, AUTH, PING, SHUTDOWN, REPLCONF, ROLE, CONFIG,
# but to INFO, SLAVEOF, AUTH, PING, SHUTDOWN, REPLCONF, ROLE, CONFIG,
# SUBSCRIBE, UNSUBSCRIBE, PSUBSCRIBE, PUNSUBSCRIBE, PUBLISH, PUBSUB,
# COMMAND, POST, HOST: and LATENCY.
#
replica-serve-stale-data yes
slave-serve-stale-data yes
# You can configure a replica instance to accept writes or not. Writing against
# a replica instance may be useful to store some ephemeral data (because data
# written on a replica will be easily deleted after resync with the master) but
# You can configure a slave instance to accept writes or not. Writing against
# a slave instance may be useful to store some ephemeral data (because data
# written on a slave will be easily deleted after resync with the master) but
# may also cause problems if clients are writing to it because of a
# misconfiguration.
#
# Since Redis 2.6 by default replicas are read-only.
# Since Redis 2.6 by default slaves are read-only.
#
# Note: read only replicas are not designed to be exposed to untrusted clients
# Note: read only slaves are not designed to be exposed to untrusted clients
# on the internet. It's just a protection layer against misuse of the instance.
# Still a read only replica exports by default all the administrative commands
# Still a read only slave exports by default all the administrative commands
# such as CONFIG, DEBUG, and so forth. To a limited extent you can improve
# security of read only replicas using 'rename-command' to shadow all the
# security of read only slaves using 'rename-command' to shadow all the
# administrative / dangerous commands.
replica-read-only yes
slave-read-only yes
# Replication SYNC strategy: disk or socket.
#
@@ -329,25 +324,25 @@ replica-read-only yes
# WARNING: DISKLESS REPLICATION IS EXPERIMENTAL CURRENTLY
# -------------------------------------------------------
#
# New replicas and reconnecting replicas that are not able to continue the replication
# New slaves and reconnecting slaves that are not able to continue the replication
# process just receiving differences, need to do what is called a "full
# synchronization". An RDB file is transmitted from the master to the replicas.
# synchronization". An RDB file is transmitted from the master to the slaves.
# The transmission can happen in two different ways:
#
# 1) Disk-backed: The Redis master creates a new process that writes the RDB
# file on disk. Later the file is transferred by the parent
# process to the replicas incrementally.
# process to the slaves incrementally.
# 2) Diskless: The Redis master creates a new process that directly writes the
# RDB file to replica sockets, without touching the disk at all.
# RDB file to slave sockets, without touching the disk at all.
#
# With disk-backed replication, while the RDB file is generated, more replicas
# With disk-backed replication, while the RDB file is generated, more slaves
# can be queued and served with the RDB file as soon as the current child producing
# the RDB file finishes its work. With diskless replication instead once
# the transfer starts, new replicas arriving will be queued and a new transfer
# the transfer starts, new slaves arriving will be queued and a new transfer
# will start when the current one terminates.
#
# When diskless replication is used, the master waits a configurable amount of
# time (in seconds) before starting the transfer in the hope that multiple replicas
# time (in seconds) before starting the transfer in the hope that multiple slaves
# will arrive and the transfer can be parallelized.
#
# With slow disks and fast (large bandwidth) networks, diskless replication
@@ -356,140 +351,140 @@ repl-diskless-sync no
# When diskless replication is enabled, it is possible to configure the delay
# the server waits in order to spawn the child that transfers the RDB via socket
# to the replicas.
# to the slaves.
#
# This is important since once the transfer starts, it is not possible to serve
# new replicas arriving, that will be queued for the next RDB transfer, so the server
# waits a delay in order to let more replicas arrive.
# new slaves arriving, that will be queued for the next RDB transfer, so the server
# waits a delay in order to let more slaves arrive.
#
# The delay is specified in seconds, and by default is 5 seconds. To disable
# it entirely just set it to 0 seconds and the transfer will start ASAP.
repl-diskless-sync-delay 5
# Replicas send PINGs to server in a predefined interval. It's possible to change
# this interval with the repl_ping_replica_period option. The default value is 10
# Slaves send PINGs to server in a predefined interval. It's possible to change
# this interval with the repl_ping_slave_period option. The default value is 10
# seconds.
#
# repl-ping-replica-period 10
# repl-ping-slave-period 10
# The following option sets the replication timeout for:
#
# 1) Bulk transfer I/O during SYNC, from the point of view of replica.
# 2) Master timeout from the point of view of replicas (data, pings).
# 3) Replica timeout from the point of view of masters (REPLCONF ACK pings).
# 1) Bulk transfer I/O during SYNC, from the point of view of slave.
# 2) Master timeout from the point of view of slaves (data, pings).
# 3) Slave timeout from the point of view of masters (REPLCONF ACK pings).
#
# It is important to make sure that this value is greater than the value
# specified for repl-ping-replica-period otherwise a timeout will be detected
# every time there is low traffic between the master and the replica.
# specified for repl-ping-slave-period otherwise a timeout will be detected
# every time there is low traffic between the master and the slave.
#
# repl-timeout 60
# Disable TCP_NODELAY on the replica socket after SYNC?
# Disable TCP_NODELAY on the slave socket after SYNC?
#
# If you select "yes" Redis will use a smaller number of TCP packets and
# less bandwidth to send data to replicas. But this can add a delay for
# the data to appear on the replica side, up to 40 milliseconds with
# less bandwidth to send data to slaves. But this can add a delay for
# the data to appear on the slave side, up to 40 milliseconds with
# Linux kernels using a default configuration.
#
# If you select "no" the delay for data to appear on the replica side will
# If you select "no" the delay for data to appear on the slave side will
# be reduced but more bandwidth will be used for replication.
#
# By default we optimize for low latency, but in very high traffic conditions
# or when the master and replicas are many hops away, turning this to "yes" may
# or when the master and slaves are many hops away, turning this to "yes" may
# be a good idea.
repl-disable-tcp-nodelay no
# Set the replication backlog size. The backlog is a buffer that accumulates
# replica data when replicas are disconnected for some time, so that when a replica
# slave data when slaves are disconnected for some time, so that when a slave
# wants to reconnect again, often a full resync is not needed, but a partial
# resync is enough, just passing the portion of data the replica missed while
# resync is enough, just passing the portion of data the slave missed while
# disconnected.
#
# The bigger the replication backlog, the longer the time the replica can be
# The bigger the replication backlog, the longer the time the slave can be
# disconnected and later be able to perform a partial resynchronization.
#
# The backlog is only allocated once there is at least a replica connected.
# The backlog is only allocated once there is at least a slave connected.
#
# repl-backlog-size 1mb
# After a master has no longer connected replicas for some time, the backlog
# After a master has no longer connected slaves for some time, the backlog
# will be freed. The following option configures the amount of seconds that
# need to elapse, starting from the time the last replica disconnected, for
# need to elapse, starting from the time the last slave disconnected, for
# the backlog buffer to be freed.
#
# Note that replicas never free the backlog for timeout, since they may be
# Note that slaves never free the backlog for timeout, since they may be
# promoted to masters later, and should be able to correctly "partially
# resynchronize" with the replicas: hence they should always accumulate backlog.
# resynchronize" with the slaves: hence they should always accumulate backlog.
#
# A value of 0 means to never release the backlog.
#
# repl-backlog-ttl 3600
# The replica priority is an integer number published by Redis in the INFO output.
# It is used by Redis Sentinel in order to select a replica to promote into a
# The slave priority is an integer number published by Redis in the INFO output.
# It is used by Redis Sentinel in order to select a slave to promote into a
# master if the master is no longer working correctly.
#
# A replica with a low priority number is considered better for promotion, so
# for instance if there are three replicas with priority 10, 100, 25 Sentinel will
# A slave with a low priority number is considered better for promotion, so
# for instance if there are three slaves with priority 10, 100, 25 Sentinel will
# pick the one with priority 10, that is the lowest.
#
# However a special priority of 0 marks the replica as not able to perform the
# role of master, so a replica with priority of 0 will never be selected by
# However a special priority of 0 marks the slave as not able to perform the
# role of master, so a slave with priority of 0 will never be selected by
# Redis Sentinel for promotion.
#
# By default the priority is 100.
replica-priority 100
slave-priority 100
# It is possible for a master to stop accepting writes if there are less than
# N replicas connected, having a lag less or equal than M seconds.
# N slaves connected, having a lag less or equal than M seconds.
#
# The N replicas need to be in "online" state.
# The N slaves need to be in "online" state.
#
# The lag in seconds, that must be <= the specified value, is calculated from
# the last ping received from the replica, that is usually sent every second.
# the last ping received from the slave, that is usually sent every second.
#
# This option does not GUARANTEE that N replicas will accept the write, but
# will limit the window of exposure for lost writes in case not enough replicas
# will limit the window of exposure for lost writes in case not enough slaves
# are available, to the specified number of seconds.
#
# For example to require at least 3 replicas with a lag <= 10 seconds use:
# For example to require at least 3 slaves with a lag <= 10 seconds use:
#
# min-replicas-to-write 3
# min-replicas-max-lag 10
# min-slaves-to-write 3
# min-slaves-max-lag 10
#
# Setting one or the other to 0 disables the feature.
#
# By default min-replicas-to-write is set to 0 (feature disabled) and
# min-replicas-max-lag is set to 10.
# By default min-slaves-to-write is set to 0 (feature disabled) and
# min-slaves-max-lag is set to 10.
# A Redis master is able to list the address and port of the attached
# replicas in different ways. For example the "INFO replication" section
# slaves in different ways. For example the "INFO replication" section
# offers this information, which is used, among other tools, by
# Redis Sentinel in order to discover replica instances.
# Redis Sentinel in order to discover slave instances.
# Another place where this info is available is in the output of the
# "ROLE" command of a master.
#
# The listed IP and address normally reported by a replica is obtained
# The listed IP and address normally reported by a slave is obtained
# in the following way:
#
# IP: The address is auto detected by checking the peer address
# of the socket used by the replica to connect with the master.
# of the socket used by the slave to connect with the master.
#
# Port: The port is communicated by the replica during the replication
# handshake, and is normally the port that the replica is using to
# listen for connections.
# Port: The port is communicated by the slave during the replication
# handshake, and is normally the port that the slave is using to
# list for connections.
#
# However when port forwarding or Network Address Translation (NAT) is
# used, the replica may be actually reachable via different IP and port
# pairs. The following two options can be used by a replica in order to
# used, the slave may be actually reachable via different IP and port
# pairs. The following two options can be used by a slave in order to
# report to its master a specific set of IP and port, so that both INFO
# and ROLE will report those values.
#
# There is no need to use both the options if you need to override just
# the port or the IP address.
#
# replica-announce-ip 5.5.5.5
# replica-announce-port 1234
# slave-announce-ip 5.5.5.5
# slave-announce-port 1234
################################## SECURITY ###################################
@@ -523,7 +518,7 @@ replica-priority 100
# rename-command CONFIG ""
#
# Please note that changing the name of commands that are logged into the
# AOF file or transmitted to replicas may cause problems.
# AOF file or transmitted to slaves may cause problems.
################################### CLIENTS ####################################
@@ -552,15 +547,15 @@ replica-priority 100
# This option is usually useful when using Redis as an LRU or LFU cache, or to
# set a hard memory limit for an instance (using the 'noeviction' policy).
#
# WARNING: If you have replicas attached to an instance with maxmemory on,
# the size of the output buffers needed to feed the replicas are subtracted
# WARNING: If you have slaves attached to an instance with maxmemory on,
# the size of the output buffers needed to feed the slaves are subtracted
# from the used memory count, so that network problems / resyncs will
# not trigger a loop where keys are evicted, and in turn the output
# buffer of replicas is full with DELs of keys evicted triggering the deletion
# buffer of slaves is full with DELs of keys evicted triggering the deletion
# of more keys, and so forth until the database is completely emptied.
#
# In short... if you have replicas attached it is suggested that you set a lower
# limit for maxmemory so that there is some free RAM on the system for replica
# In short... if you have slaves attached it is suggested that you set a lower
# limit for maxmemory so that there is some free RAM on the system for slave
# output buffers (but this is not needed if the policy is 'noeviction').
#
# maxmemory <bytes>
@@ -607,25 +602,25 @@ replica-priority 100
#
# maxmemory-samples 5
# Starting from Redis 5, by default a replica will ignore its maxmemory setting
# Starting from Redis 5, by default a slave will ignore its maxmemory setting
# (unless it is promoted to master after a failover or manually). It means
# that the eviction of keys will be just handled by the master, sending the
# DEL commands to the replica as keys evict in the master side.
# DEL commands to the slave as keys evict in the master side.
#
# This behavior ensures that masters and replicas stay consistent, and is usually
# what you want, however if your replica is writable, or you want the replica to have
# This behavior ensures that masters and slaves stay consistent, and is usually
# what you want, however if your slave is writable, or you want the slave to have
# a different memory setting, and you are sure all the writes performed to the
# replica are idempotent, then you may change this default (but be sure to understand
# slave are idempotent, then you may change this default (but be sure to understand
# what you are doing).
#
# Note that since the replica by default does not evict, it may end using more
# Note that since the slave by default does not evict, it may end using more
# memory than the one set via maxmemory (there are certain buffers that may
# be larger on the replica, or data structures may sometimes take more memory and so
# forth). So make sure you monitor your replicas and make sure they have enough
# be larger on the slave, or data structures may sometimes take more memory and so
# forth). So make sure you monitor your slaves and make sure they have enough
# memory to never hit a real out-of-memory condition before the master hits
# the configured maxmemory setting.
#
# replica-ignore-maxmemory yes
# slave-ingore-maxmemory yes
############################# LAZY FREEING ####################################
@@ -662,7 +657,7 @@ replica-priority 100
# or SORT with STORE option may delete existing keys. The SET command
# itself removes any old content of the specified key in order to replace
# it with the specified string.
# 4) During replication, when a replica performs a full resynchronization with
# 4) During replication, when a slave performs a full resynchronization with
# its master, the content of the whole database is removed in order to
# load the RDB file just transferred.
#
@@ -674,7 +669,7 @@ replica-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
slave-lazy-flush no
############################## APPEND ONLY MODE ###############################
@@ -851,42 +846,42 @@ lua-time-limit 5000
#
# cluster-node-timeout 15000
# A replica of a failing master will avoid to start a failover if its data
# A slave of a failing master will avoid to start a failover if its data
# looks too old.
#
# There is no simple way for a replica to actually have an exact measure of
# There is no simple way for a slave to actually have an exact measure of
# its "data age", so the following two checks are performed:
#
# 1) If there are multiple replicas able to failover, they exchange messages
# in order to try to give an advantage to the replica with the best
# 1) If there are multiple slaves able to failover, they exchange messages
# in order to try to give an advantage to the slave with the best
# replication offset (more data from the master processed).
# Replicas will try to get their rank by offset, and apply to the start
# Slaves will try to get their rank by offset, and apply to the start
# of the failover a delay proportional to their rank.
#
# 2) Every single replica computes the time of the last interaction with
# 2) Every single slave computes the time of the last interaction with
# its master. This can be the last ping or command received (if the master
# is still in the "connected" state), or the time that elapsed since the
# disconnection with the master (if the replication link is currently down).
# If the last interaction is too old, the replica will not try to failover
# If the last interaction is too old, the slave will not try to failover
# at all.
#
# The point "2" can be tuned by user. Specifically a replica will not perform
# The point "2" can be tuned by user. Specifically a slave will not perform
# the failover if, since the last interaction with the master, the time
# elapsed is greater than:
#
# (node-timeout * replica-validity-factor) + repl-ping-replica-period
# (node-timeout * slave-validity-factor) + repl-ping-slave-period
#
# So for example if node-timeout is 30 seconds, and the replica-validity-factor
# is 10, and assuming a default repl-ping-replica-period of 10 seconds, the
# replica will not try to failover if it was not able to talk with the master
# So for example if node-timeout is 30 seconds, and the slave-validity-factor
# is 10, and assuming a default repl-ping-slave-period of 10 seconds, the
# slave will not try to failover if it was not able to talk with the master
# for longer than 310 seconds.
#
# A large replica-validity-factor may allow replicas with too old data to failover
# A large slave-validity-factor may allow slaves with too old data to failover
# a master, while a too small value may prevent the cluster from being able to
# elect a replica at all.
# elect a slave at all.
#
# For maximum availability, it is possible to set the replica-validity-factor
# to a value of 0, which means, that replicas will always try to failover the
# For maximum availability, it is possible to set the slave-validity-factor
# to a value of 0, which means, that slaves will always try to failover the
# master regardless of the last time they interacted with the master.
# (However they'll always try to apply a delay proportional to their
# offset rank).
@@ -894,22 +889,22 @@ lua-time-limit 5000
# Zero is the only value able to guarantee that when all the partitions heal
# the cluster will always be able to continue.
#
# cluster-replica-validity-factor 10
# cluster-slave-validity-factor 10
# Cluster replicas are able to migrate to orphaned masters, that are masters
# that are left without working replicas. This improves the cluster ability
# Cluster slaves are able to migrate to orphaned masters, that are masters
# that are left without working slaves. This improves the cluster ability
# to resist to failures as otherwise an orphaned master can't be failed over
# in case of failure if it has no working replicas.
# in case of failure if it has no working slaves.
#
# Replicas migrate to orphaned masters only if there are still at least a
# given number of other working replicas for their old master. This number
# is the "migration barrier". A migration barrier of 1 means that a replica
# will migrate only if there is at least 1 other working replica for its master
# and so forth. It usually reflects the number of replicas you want for every
# Slaves migrate to orphaned masters only if there are still at least a
# given number of other working slaves for their old master. This number
# is the "migration barrier". A migration barrier of 1 means that a slave
# will migrate only if there is at least 1 other working slave for its master
# and so forth. It usually reflects the number of slaves you want for every
# master in your cluster.
#
# Default is 1 (replicas migrate only if their masters remain with at least
# one replica). To disable migration just set it to a very large value.
# Default is 1 (slaves migrate only if their masters remain with at least
# one slave). To disable migration just set it to a very large value.
# A value of 0 can be set but is useful only for debugging and dangerous
# in production.
#
@@ -928,7 +923,7 @@ lua-time-limit 5000
#
# cluster-require-full-coverage yes
# This option, when set to yes, prevents replicas from trying to failover its
# This option, when set to yes, prevents slaves from trying to failover its
# master during master failures. However the master can still perform a
# manual failover, if forced to do so.
#
@@ -936,7 +931,7 @@ lua-time-limit 5000
# data center operations, where we want one side to never be promoted if not
# in the case of a total DC failure.
#
# cluster-replica-no-failover no
# cluster-slave-no-failover no
# In order to setup your cluster make sure to read the documentation
# available at http://redis.io web site.
@@ -1170,7 +1165,7 @@ activerehashing yes
# The limit can be set differently for the three different classes of clients:
#
# normal -> normal clients including MONITOR clients
# replica -> replica clients
# slave -> slave clients
# pubsub -> clients subscribed to at least one pubsub channel or pattern
#
# The syntax of every client-output-buffer-limit directive is the following:
@@ -1191,12 +1186,12 @@ activerehashing yes
# asynchronous clients may create a scenario where data is requested faster
# than it can read.
#
# Instead there is a default limit for pubsub and replica clients, since
# subscribers and replicas receive data in a push fashion.
# Instead there is a default limit for pubsub and slave clients, since
# subscribers and slaves receive data in a push fashion.
#
# Both the hard or the soft limit can be disabled by setting them to zero.
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
# Client query buffers accumulate new commands. They are limited to a fixed
+19 -24
View File
@@ -73,11 +73,11 @@ dir /tmp
# be elected by the majority of the known Sentinels in order to
# start a failover, so no failover can be performed in minority.
#
# Replicas are auto-discovered, so you don't need to specify replicas in
# Slaves are auto-discovered, so you don't need to specify slaves in
# any way. Sentinel itself will rewrite this configuration file adding
# the replicas using additional configuration options.
# the slaves using additional configuration options.
# Also note that the configuration file is rewritten when a
# replica is promoted to master.
# slave is promoted to master.
#
# Note: master name should not include special characters or spaces.
# The valid charset is A-z 0-9 and the three characters ".-_".
@@ -85,11 +85,11 @@ sentinel monitor mymaster 127.0.0.1 6379 2
# sentinel auth-pass <master-name> <password>
#
# Set the password to use to authenticate with the master and replicas.
# Set the password to use to authenticate with the master and slaves.
# Useful if there is a password set in the Redis instances to monitor.
#
# Note that the master password is also used for replicas, so it is not
# possible to set a different password in masters and replicas instances
# Note that the master password is also used for slaves, so it is not
# possible to set a different password in masters and slaves instances
# if you want to be able to monitor these instances with Sentinel.
#
# However you can have Redis instances without the authentication enabled
@@ -104,7 +104,7 @@ sentinel monitor mymaster 127.0.0.1 6379 2
# sentinel down-after-milliseconds <master-name> <milliseconds>
#
# Number of milliseconds the master (or any attached replica or sentinel) should
# Number of milliseconds the master (or any attached slave or sentinel) should
# be unreachable (as in, not acceptable reply to PING, continuously, for the
# specified period) in order to consider it in S_DOWN state (Subjectively
# Down).
@@ -112,11 +112,11 @@ sentinel monitor mymaster 127.0.0.1 6379 2
# Default is 30 seconds.
sentinel down-after-milliseconds mymaster 30000
# sentinel parallel-syncs <master-name> <numreplicas>
# sentinel parallel-syncs <master-name> <numslaves>
#
# How many replicas we can reconfigure to point to the new replica simultaneously
# during the failover. Use a low number if you use the replicas to serve query
# to avoid that all the replicas will be unreachable at about the same
# How many slaves we can reconfigure to point to the new slave simultaneously
# during the failover. Use a low number if you use the slaves to serve query
# to avoid that all the slaves will be unreachable at about the same
# time while performing the synchronization with the master.
sentinel parallel-syncs mymaster 1
@@ -128,18 +128,18 @@ sentinel parallel-syncs mymaster 1
# already tried against the same master by a given Sentinel, is two
# times the failover timeout.
#
# - The time needed for a replica replicating to a wrong master according
# - The time needed for a slave replicating to a wrong master according
# to a Sentinel current configuration, to be forced to replicate
# with the right master, is exactly the failover timeout (counting since
# the moment a Sentinel detected the misconfiguration).
#
# - The time needed to cancel a failover that is already in progress but
# did not produced any configuration change (SLAVEOF NO ONE yet not
# acknowledged by the promoted replica).
# acknowledged by the promoted slave).
#
# - The maximum time a failover in progress waits for all the replicas to be
# reconfigured as replicas of the new master. However even after this time
# the replicas will be reconfigured by the Sentinels anyway, but not with
# - The maximum time a failover in progress waits for all the slaves to be
# reconfigured as slaves of the new master. However even after this time
# the slaves will be reconfigured by the Sentinels anyway, but not with
# the exact parallel-syncs progression as specified.
#
# Default is 3 minutes.
@@ -200,7 +200,7 @@ sentinel failover-timeout mymaster 180000
# <role> is either "leader" or "observer"
#
# The arguments from-ip, from-port, to-ip, to-port are used to communicate
# the old address of the master and the new address of the elected replica
# the old address of the master and the new address of the elected slave
# (now a master).
#
# This script should be resistant to multiple invocations.
@@ -228,17 +228,12 @@ sentinel deny-scripts-reconfig yes
#
# In such case it is possible to tell Sentinel to use different command names
# instead of the normal ones. For example if the master "mymaster", and the
# associated replicas, have "CONFIG" all renamed to "GUESSME", I could use:
# associated slaves, have "CONFIG" all renamed to "GUESSME", I could use:
#
# SENTINEL rename-command mymaster CONFIG GUESSME
# sentinel rename-command mymaster CONFIG GUESSME
#
# After such configuration is set, every time Sentinel would use CONFIG it will
# use GUESSME instead. Note that there is no actual need to respect the command
# case, so writing "config guessme" is the same in the example above.
#
# SENTINEL SET can also be used in order to perform this configuration at runtime.
#
# In order to set a command back to its original name (undo the renaming), it
# is possible to just rename a command to itsef:
#
# SENTINEL rename-command mymaster CONFIG CONFIG
+1 -5
View File
@@ -41,10 +41,6 @@ endif
# To get ARM stack traces if Redis crashes we need a special C flag.
ifneq (,$(filter aarch64 armv,$(uname_M)))
CFLAGS+=-funwind-tables
else
ifneq (,$(findstring armv,$(uname_M)))
CFLAGS+=-funwind-tables
endif
endif
# Backwards compatibility for selecting an allocator
@@ -148,7 +144,7 @@ endif
REDIS_SERVER_NAME=redis-server
REDIS_SENTINEL_NAME=redis-sentinel
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o
REDIS_CLI_NAME=redis-cli
REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o anet.o ae.o crc64.o siphash.o crc16.o
REDIS_BENCHMARK_NAME=redis-benchmark
+18 -58
View File
@@ -677,7 +677,6 @@ int loadAppendOnlyFile(char *filename) {
int old_aof_state = server.aof_state;
long loops = 0;
off_t valid_up_to = 0; /* Offset of latest well-formed command loaded. */
off_t valid_before_multi = 0; /* Offset before MULTI command loaded. */
if (fp == NULL) {
serverLog(LL_WARNING,"Fatal error: can't open the append log file for reading: %s",strerror(errno));
@@ -778,28 +777,16 @@ int loadAppendOnlyFile(char *filename) {
/* Command lookup */
cmd = lookupCommand(argv[0]->ptr);
if (!cmd) {
serverLog(LL_WARNING,
"Unknown command '%s' reading the append only file",
(char*)argv[0]->ptr);
serverLog(LL_WARNING,"Unknown command '%s' reading the append only file", (char*)argv[0]->ptr);
exit(1);
}
if (cmd == server.multiCommand) valid_before_multi = valid_up_to;
/* Run the command in the context of a fake client */
fakeClient->cmd = cmd;
if (fakeClient->flags & CLIENT_MULTI &&
fakeClient->cmd->proc != execCommand)
{
queueMultiCommand(fakeClient);
} else {
cmd->proc(fakeClient);
}
cmd->proc(fakeClient);
/* The fake client should not have a reply */
serverAssert(fakeClient->bufpos == 0 &&
listLength(fakeClient->reply) == 0);
serverAssert(fakeClient->bufpos == 0 && listLength(fakeClient->reply) == 0);
/* The fake client should never get blocked */
serverAssert((fakeClient->flags & CLIENT_BLOCKED) == 0);
@@ -814,12 +801,7 @@ int loadAppendOnlyFile(char *filename) {
* If the client is in the middle of a MULTI/EXEC, handle it as it was
* a short read, even if technically the protocol is correct: we want
* to remove the unprocessed tail and continue. */
if (fakeClient->flags & CLIENT_MULTI) {
serverLog(LL_WARNING,
"Revert incomplete MULTI/EXEC transaction in AOF file");
valid_up_to = valid_before_multi;
goto uxeof;
}
if (fakeClient->flags & CLIENT_MULTI) goto uxeof;
loaded_ok: /* DB loaded, cleanup and return C_OK to the caller. */
fclose(fp);
@@ -1139,47 +1121,25 @@ int rewriteStreamObject(rio *r, robj *key, robj *o) {
streamID id;
int64_t numfields;
if (s->length) {
/* Reconstruct the stream data using XADD commands. */
while(streamIteratorGetID(&si,&id,&numfields)) {
/* Emit a two elements array for each item. The first is
* the ID, the second is an array of field-value pairs. */
/* Reconstruct the stream data using XADD commands. */
while(streamIteratorGetID(&si,&id,&numfields)) {
/* Emit a two elements array for each item. The first is
* the ID, the second is an array of field-value pairs. */
/* Emit the XADD <key> <id> ...fields... command. */
if (rioWriteBulkCount(r,'*',3+numfields*2) == 0) return 0;
if (rioWriteBulkString(r,"XADD",4) == 0) return 0;
if (rioWriteBulkObject(r,key) == 0) return 0;
if (rioWriteBulkStreamID(r,&id) == 0) return 0;
while(numfields--) {
unsigned char *field, *value;
int64_t field_len, value_len;
streamIteratorGetField(&si,&field,&value,&field_len,&value_len);
if (rioWriteBulkString(r,(char*)field,field_len) == 0) return 0;
if (rioWriteBulkString(r,(char*)value,value_len) == 0) return 0;
}
}
} else {
/* Use the XADD MAXLEN 0 trick to generate an empty stream if
* the key we are serializing is an empty string, which is possible
* for the Stream type. */
if (rioWriteBulkCount(r,'*',7) == 0) return 0;
/* Emit the XADD <key> <id> ...fields... command. */
if (rioWriteBulkCount(r,'*',3+numfields*2) == 0) return 0;
if (rioWriteBulkString(r,"XADD",4) == 0) return 0;
if (rioWriteBulkObject(r,key) == 0) return 0;
if (rioWriteBulkString(r,"MAXLEN",6) == 0) return 0;
if (rioWriteBulkString(r,"0",1) == 0) return 0;
if (rioWriteBulkStreamID(r,&s->last_id) == 0) return 0;
if (rioWriteBulkString(r,"x",1) == 0) return 0;
if (rioWriteBulkString(r,"y",1) == 0) return 0;
if (rioWriteBulkStreamID(r,&id) == 0) return 0;
while(numfields--) {
unsigned char *field, *value;
int64_t field_len, value_len;
streamIteratorGetField(&si,&field,&value,&field_len,&value_len);
if (rioWriteBulkString(r,(char*)field,field_len) == 0) return 0;
if (rioWriteBulkString(r,(char*)value,value_len) == 0) return 0;
}
}
/* Append XSETID after XADD, make sure lastid is correct,
* in case of XDEL lastid. */
if (rioWriteBulkCount(r,'*',3) == 0) return 0;
if (rioWriteBulkString(r,"XSETID",6) == 0) return 0;
if (rioWriteBulkObject(r,key) == 0) return 0;
if (rioWriteBulkStreamID(r,&s->last_id) == 0) return 0;
/* Create all the stream consumer groups. */
if (s->cgroups) {
raxIterator ri;
+1 -1
View File
@@ -1,7 +1,7 @@
/* This file implements atomic counters using __atomic or __sync macros if
* available, otherwise synchronizing different threads using a mutex.
*
* The exported interface is composed of three macros:
* The exported interaface is composed of three macros:
*
* atomicIncr(var,count) -- Increment the atomic counter
* atomicGetIncr(var,oldvalue_var,count) -- Get and increment the atomic counter
+4 -4
View File
@@ -17,7 +17,7 @@
*
* The design is trivial, we have a structure representing a job to perform
* and a different thread and job queue for every job type.
* Every thread waits for new jobs in its queue, and process every job
* Every thread wait for new jobs in its queue, and process every job
* sequentially.
*
* Jobs of the same type are guaranteed to be processed from the least
@@ -204,14 +204,14 @@ void *bioProcessBackgroundJobs(void *arg) {
}
zfree(job);
/* Unblock threads blocked on bioWaitStepOfType() if any. */
pthread_cond_broadcast(&bio_step_cond[type]);
/* Lock again before reiterating the loop, if there are no longer
* jobs to process we'll block again in pthread_cond_wait(). */
pthread_mutex_lock(&bio_mutex[type]);
listDelNode(bio_jobs[type],ln);
bio_pending[type]--;
/* Unblock threads blocked on bioWaitStepOfType() if any. */
pthread_cond_broadcast(&bio_step_cond[type]);
}
}
+8 -28
View File
@@ -126,37 +126,12 @@ void processUnblockedClients(void) {
* the code is conceptually more correct this way. */
if (!(c->flags & CLIENT_BLOCKED)) {
if (c->querybuf && sdslen(c->querybuf) > 0) {
processInputBufferAndReplicate(c);
processInputBuffer(c);
}
}
}
}
/* This function will schedule the client for reprocessing at a safe time.
*
* This is useful when a client was blocked for some reason (blocking opeation,
* CLIENT PAUSE, or whatever), because it may end with some accumulated query
* buffer that needs to be processed ASAP:
*
* 1. When a client is blocked, its readable handler is still active.
* 2. However in this case it only gets data into the query buffer, but the
* query is not parsed or executed once there is enough to proceed as
* usually (because the client is blocked... so we can't execute commands).
* 3. When the client is unblocked, without this function, the client would
* have to write some query in order for the readable handler to finally
* call processQueryBuffer*() on it.
* 4. With this function instead we can put the client in a queue that will
* process it for queries ready to be executed at a safe time.
*/
void queueClientForReprocessing(client *c) {
/* The client may already be into the unblocked list because of a previous
* blocking operation, don't add back it into the list multiple times. */
if (!(c->flags & CLIENT_UNBLOCKED)) {
c->flags |= CLIENT_UNBLOCKED;
listAddNodeTail(server.unblocked_clients,c);
}
}
/* Unblock a client calling the right function depending on the kind
* of operation the client is blocking for. */
void unblockClient(client *c) {
@@ -177,7 +152,12 @@ void unblockClient(client *c) {
server.blocked_clients_by_type[c->btype]--;
c->flags &= ~CLIENT_BLOCKED;
c->btype = BLOCKED_NONE;
queueClientForReprocessing(c);
/* The client may already be into the unblocked list because of a previous
* blocking operation, don't add back it into the list multiple times. */
if (!(c->flags & CLIENT_UNBLOCKED)) {
c->flags |= CLIENT_UNBLOCKED;
listAddNodeTail(server.unblocked_clients,c);
}
}
/* This function gets called when a blocked client timed out in order to
@@ -215,7 +195,7 @@ void disconnectAllBlockedClients(void) {
if (c->flags & CLIENT_BLOCKED) {
addReplySds(c,sdsnew(
"-UNBLOCKED force unblock from blocking operation, "
"instance state changed (master -> replica?)\r\n"));
"instance state changed (master -> slave?)\r\n"));
unblockClient(c);
c->flags |= CLIENT_CLOSE_AFTER_REPLY;
}
+19 -49
View File
@@ -1230,7 +1230,7 @@ void clearNodeFailureIfNeeded(clusterNode *node) {
serverLog(LL_NOTICE,
"Clear FAIL state for node %.40s: %s is reachable again.",
node->name,
nodeIsSlave(node) ? "replica" : "master without slots");
nodeIsSlave(node) ? "slave" : "master without slots");
node->flags &= ~CLUSTER_NODE_FAIL;
clusterDoBeforeSleep(CLUSTER_TODO_UPDATE_STATE|CLUSTER_TODO_SAVE_CONFIG);
}
@@ -1589,12 +1589,6 @@ void clusterUpdateSlotsConfigWith(clusterNode *sender, uint64_t senderConfigEpoc
}
}
/* After updating the slots configuration, don't do any actual change
* in the state of the server if a module disabled Redis Cluster
* keys redirections. */
if (server.cluster_module_flags & CLUSTER_MODULE_FLAG_NO_REDIRECTION)
return;
/* If at least one slot was reassigned from a node to another node
* with a greater configEpoch, it is possible that:
* 1) We are a master left without slots. This means that we were
@@ -2065,7 +2059,7 @@ int clusterProcessPacket(clusterLink *link) {
server.cluster->mf_end = mstime() + CLUSTER_MF_TIMEOUT;
server.cluster->mf_slave = sender;
pauseClients(mstime()+(CLUSTER_MF_TIMEOUT*2));
serverLog(LL_WARNING,"Manual failover requested by replica %.40s.",
serverLog(LL_WARNING,"Manual failover requested by slave %.40s.",
sender->name);
} else if (type == CLUSTERMSG_TYPE_UPDATE) {
clusterNode *n; /* The node the update is about. */
@@ -2879,7 +2873,7 @@ void clusterLogCantFailover(int reason) {
switch(reason) {
case CLUSTER_CANT_FAILOVER_DATA_AGE:
msg = "Disconnected from master for longer than allowed. "
"Please check the 'cluster-replica-validity-factor' configuration "
"Please check the 'cluster-slave-validity-factor' configuration "
"option.";
break;
case CLUSTER_CANT_FAILOVER_WAITING_DELAY:
@@ -3060,7 +3054,7 @@ void clusterHandleSlaveFailover(void) {
server.cluster->failover_auth_time += added_delay;
server.cluster->failover_auth_rank = newrank;
serverLog(LL_WARNING,
"Replica rank updated to #%d, added %lld milliseconds of delay.",
"Slave rank updated to #%d, added %lld milliseconds of delay.",
newrank, added_delay);
}
}
@@ -3216,8 +3210,7 @@ void clusterHandleSlaveMigration(int max_slaves) {
* the natural slaves of this instance to advertise their switch from
* the old master to the new one. */
if (target && candidate == myself &&
(mstime()-target->orphaned_time) > CLUSTER_SLAVE_MIGRATION_DELAY &&
!(server.cluster_module_flags & CLUSTER_MODULE_FLAG_NO_FAILOVER))
(mstime()-target->orphaned_time) > CLUSTER_SLAVE_MIGRATION_DELAY)
{
serverLog(LL_WARNING,"Migrating to orphaned master %.40s",
target->name);
@@ -3570,8 +3563,7 @@ void clusterCron(void) {
if (nodeIsSlave(myself)) {
clusterHandleManualFailover();
if (!(server.cluster_module_flags & CLUSTER_MODULE_FLAG_NO_FAILOVER))
clusterHandleSlaveFailover();
clusterHandleSlaveFailover();
/* If there are orphaned slaves, and we are a slave among the masters
* with the max number of non-failing slaves, consider migrating to
* the orphaned masters. Note that it does not make sense to try
@@ -3877,11 +3869,6 @@ int verifyClusterConfigWithData(void) {
int j;
int update_config = 0;
/* Return ASAP if a module disabled cluster redirections. In that case
* every master can store keys about every possible hash slot. */
if (server.cluster_module_flags & CLUSTER_MODULE_FLAG_NO_REDIRECTION)
return C_OK;
/* If this node is a slave, don't perform the check at all as we
* completely depend on the replication stream. */
if (nodeIsSlave(myself)) return C_OK;
@@ -4200,7 +4187,7 @@ void clusterCommand(client *c) {
"COUNT-failure-reports <node-id> -- Return number of failure reports for <node-id>.",
"COUNTKEYSINSLOT <slot> - Return the number of keys in <slot>.",
"DELSLOTS <slot> [slot ...] -- Delete slots information from current node.",
"FAILOVER [force|takeover] -- Promote current replica node to being a master.",
"FAILOVER [force|takeover] -- Promote current slave node to being a master.",
"FORGET <node-id> -- Remove a node from the cluster.",
"GETKEYSINSLOT <slot> <count> -- Return key names stored by current node in a slot.",
"FLUSHSLOTS -- Delete current node own slots information.",
@@ -4210,11 +4197,11 @@ void clusterCommand(client *c) {
"MYID -- Return the node id.",
"NODES -- Return cluster configuration seen by node. Output format:",
" <id> <ip:port> <flags> <master> <pings> <pongs> <epoch> <link> <slot> ... <slot>",
"REPLICATE <node-id> -- Configure current node as replica to <node-id>.",
"REPLICATE <node-id> -- Configure current node as slave to <node-id>.",
"RESET [hard|soft] -- Reset current node (default: soft).",
"SET-config-epoch <epoch> - Set config epoch of current node.",
"SETSLOT <slot> (importing|migrating|stable|node <node-id>) -- Set slot state.",
"REPLICAS <node-id> -- Return <node-id> replicas.",
"SLAVES <node-id> -- Return <node-id> slaves.",
"SLOTS -- Return information about slots range mappings. Each range is made of:",
" start, end, master and replicas IP addresses, ports and ids",
NULL
@@ -4591,7 +4578,7 @@ NULL
/* Can't replicate a slave. */
if (nodeIsSlave(n)) {
addReplyError(c,"I can only replicate a master, not a replica.");
addReplyError(c,"I can only replicate a master, not a slave.");
return;
}
@@ -4610,8 +4597,7 @@ NULL
clusterSetMaster(n);
clusterDoBeforeSleep(CLUSTER_TODO_UPDATE_STATE|CLUSTER_TODO_SAVE_CONFIG);
addReply(c,shared.ok);
} else if ((!strcasecmp(c->argv[1]->ptr,"slaves") ||
!strcasecmp(c->argv[1]->ptr,"replicas")) && c->argc == 3) {
} else if (!strcasecmp(c->argv[1]->ptr,"slaves") && c->argc == 3) {
/* CLUSTER SLAVES <NODE ID> */
clusterNode *n = clusterLookupNode(c->argv[2]->ptr);
int j;
@@ -4665,10 +4651,10 @@ NULL
/* Check preconditions. */
if (nodeIsMaster(myself)) {
addReplyError(c,"You should send CLUSTER FAILOVER to a replica");
addReplyError(c,"You should send CLUSTER FAILOVER to a slave");
return;
} else if (myself->slaveof == NULL) {
addReplyError(c,"I'm a replica but my master is unknown to me");
addReplyError(c,"I'm a slave but my master is unknown to me");
return;
} else if (!force &&
(nodeFailed(myself->slaveof) ||
@@ -5164,10 +5150,10 @@ try_again:
serverAssertWithInfo(c,NULL,rioWriteBulkLongLong(&cmd,dbid));
}
int non_expired = 0; /* Number of keys that we'll find non expired.
Note that serializing large keys may take some time
so certain keys that were found non expired by the
lookupKey() function, may be expired later. */
int expired = 0; /* Number of keys that we'll find already expired.
Note that serializing large keys may take some time
so certain keys that were found non expired by the
lookupKey() function, may be expired later. */
/* Create RESTORE payload and generate the protocol to call the command. */
for (j = 0; j < num_keys; j++) {
@@ -5177,16 +5163,11 @@ try_again:
if (expireat != -1) {
ttl = expireat-mstime();
if (ttl < 0) {
expired++;
continue;
}
if (ttl < 1) ttl = 1;
}
/* Relocate valid (non expired) keys into the array in successive
* positions to remove holes created by the keys that were present
* in the first lookup but are now expired after the second lookup. */
kv[non_expired++] = kv[j];
serverAssertWithInfo(c,NULL,
rioWriteBulkCount(&cmd,'*',replace ? 5 : 4));
@@ -5214,9 +5195,6 @@ try_again:
serverAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,"REPLACE",7));
}
/* Fix the actual number of keys we are migrating. */
num_keys = non_expired;
/* Transfer the query to the other node in 64K chunks. */
errno = 0;
{
@@ -5258,7 +5236,7 @@ try_again:
* command name itself. */
if (!copy) newargv = zmalloc(sizeof(robj*)*(num_keys+1));
for (j = 0; j < num_keys; j++) {
for (j = 0; j < num_keys-expired; j++) {
if (syncReadLine(cs->fd, buf2, sizeof(buf2), timeout) <= 0) {
socket_error = 1;
break;
@@ -5456,17 +5434,9 @@ clusterNode *getNodeByQuery(client *c, struct redisCommand *cmd, robj **argv, in
multiCmd mc;
int i, slot = 0, migrating_slot = 0, importing_slot = 0, missing_keys = 0;
/* Allow any key to be set if a module disabled cluster redirections. */
if (server.cluster_module_flags & CLUSTER_MODULE_FLAG_NO_REDIRECTION)
return myself;
/* Set error code optimistically for the base case. */
if (error_code) *error_code = CLUSTER_REDIR_NONE;
/* Modules can turn off Redis Cluster redirection: this is useful
* when writing a module that implements a completely different
* distributed system. */
/* We handle all the cases as if they were EXEC commands, so we have
* a common code path for everything */
if (cmd->proc == execCommand) {
-7
View File
@@ -100,13 +100,6 @@ typedef struct clusterLink {
#define CLUSTERMSG_TYPE_MODULE 9 /* Module cluster API message. */
#define CLUSTERMSG_TYPE_COUNT 10 /* Total number of message types. */
/* Flags that a module can set in order to prevent certain Redis Cluster
* features to be enabled. Useful when implementing a different distributed
* system on top of Redis Cluster message bus, using modules. */
#define CLUSTER_MODULE_FLAG_NONE 0
#define CLUSTER_MODULE_FLAG_NO_FAILOVER (1<<1)
#define CLUSTER_MODULE_FLAG_NO_REDIRECTION (1<<2)
/* This structure represent elements of node->fail_reports. */
typedef struct clusterNodeFailReport {
struct clusterNode *node; /* Node reporting the failure condition. */
+40 -133
View File
@@ -120,7 +120,7 @@ const char *configEnumGetName(configEnum *ce, int val) {
return NULL;
}
/* Wrapper for configEnumGetName() returning "unknown" instead of NULL if
/* Wrapper for configEnumGetName() returning "unknown" insetad of NULL if
* there is no match. */
const char *configEnumGetNameOrUnknown(configEnum *ce, int val) {
const char *name = configEnumGetName(ce,val);
@@ -344,19 +344,15 @@ void loadServerConfigFromString(char *config) {
err = "lfu-decay-time must be 0 or greater";
goto loaderr;
}
} else if ((!strcasecmp(argv[0],"slaveof") ||
!strcasecmp(argv[0],"replicaof")) && argc == 3) {
} else if (!strcasecmp(argv[0],"slaveof") && argc == 3) {
slaveof_linenum = linenum;
server.masterhost = sdsnew(argv[1]);
server.masterport = atoi(argv[2]);
server.repl_state = REPL_STATE_CONNECT;
} else if ((!strcasecmp(argv[0],"repl-ping-slave-period") ||
!strcasecmp(argv[0],"repl-ping-replica-period")) &&
argc == 2)
{
} else if (!strcasecmp(argv[0],"repl-ping-slave-period") && argc == 2) {
server.repl_ping_slave_period = atoi(argv[1]);
if (server.repl_ping_slave_period <= 0) {
err = "repl-ping-replica-period must be 1 or greater";
err = "repl-ping-slave-period must be 1 or greater";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"repl-timeout") && argc == 2) {
@@ -395,24 +391,15 @@ void loadServerConfigFromString(char *config) {
} else if (!strcasecmp(argv[0],"masterauth") && argc == 2) {
zfree(server.masterauth);
server.masterauth = argv[1][0] ? zstrdup(argv[1]) : NULL;
} else if ((!strcasecmp(argv[0],"slave-serve-stale-data") ||
!strcasecmp(argv[0],"replica-serve-stale-data"))
&& argc == 2)
{
} else if (!strcasecmp(argv[0],"slave-serve-stale-data") && argc == 2) {
if ((server.repl_serve_stale_data = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if ((!strcasecmp(argv[0],"slave-read-only") ||
!strcasecmp(argv[0],"replica-read-only"))
&& argc == 2)
{
} else if (!strcasecmp(argv[0],"slave-read-only") && argc == 2) {
if ((server.repl_slave_ro = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if ((!strcasecmp(argv[0],"slave-ignore-maxmemory") ||
!strcasecmp(argv[0],"replica-ignore-maxmemory"))
&& argc == 2)
{
} else if (!strcasecmp(argv[0],"slave-ignore-maxmemory") && argc == 2) {
if ((server.repl_slave_ignore_maxmemory = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
@@ -440,9 +427,7 @@ void loadServerConfigFromString(char *config) {
if ((server.lazyfree_lazy_server_del = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if ((!strcasecmp(argv[0],"slave-lazy-flush") ||
!strcasecmp(argv[0],"replica-lazy-flush")) && argc == 2)
{
} else if (!strcasecmp(argv[0],"slave-lazy-flush") && argc == 2) {
if ((server.repl_slave_lazy_flush = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
@@ -674,17 +659,15 @@ void loadServerConfigFromString(char *config) {
err = "cluster migration barrier must zero or positive";
goto loaderr;
}
} else if ((!strcasecmp(argv[0],"cluster-slave-validity-factor") ||
!strcasecmp(argv[0],"cluster-replica-validity-factor"))
} else if (!strcasecmp(argv[0],"cluster-slave-validity-factor")
&& argc == 2)
{
server.cluster_slave_validity_factor = atoi(argv[1]);
if (server.cluster_slave_validity_factor < 0) {
err = "cluster replica validity factor must be zero or positive";
err = "cluster slave validity factor must be zero or positive";
goto loaderr;
}
} else if ((!strcasecmp(argv[0],"cluster-slave-no-failover") ||
!strcasecmp(argv[0],"cluster-replica-no-failover")) &&
} else if (!strcasecmp(argv[0],"cluster-slave-no-failover") &&
argc == 2)
{
server.cluster_slave_no_failover = yesnotoi(argv[1]);
@@ -694,8 +677,6 @@ void loadServerConfigFromString(char *config) {
}
} else if (!strcasecmp(argv[0],"lua-time-limit") && argc == 2) {
server.lua_time_limit = strtoll(argv[1],NULL,10);
} else if (!strcasecmp(argv[0],"lua-replicate-commands") && argc == 2) {
server.lua_always_replicate_commands = yesnotoi(argv[1]);
} else if (!strcasecmp(argv[0],"slowlog-log-slower-than") &&
argc == 2)
{
@@ -737,37 +718,27 @@ void loadServerConfigFromString(char *config) {
if ((server.stop_writes_on_bgsave_err = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if ((!strcasecmp(argv[0],"slave-priority") ||
!strcasecmp(argv[0],"replica-priority")) && argc == 2)
{
} else if (!strcasecmp(argv[0],"slave-priority") && argc == 2) {
server.slave_priority = atoi(argv[1]);
} else if ((!strcasecmp(argv[0],"slave-announce-ip") ||
!strcasecmp(argv[0],"replica-announce-ip")) && argc == 2)
{
} else if (!strcasecmp(argv[0],"slave-announce-ip") && argc == 2) {
zfree(server.slave_announce_ip);
server.slave_announce_ip = zstrdup(argv[1]);
} else if ((!strcasecmp(argv[0],"slave-announce-port") ||
!strcasecmp(argv[0],"replica-announce-port")) && argc == 2)
{
} else if (!strcasecmp(argv[0],"slave-announce-port") && argc == 2) {
server.slave_announce_port = atoi(argv[1]);
if (server.slave_announce_port < 0 ||
server.slave_announce_port > 65535)
{
err = "Invalid port"; goto loaderr;
}
} else if ((!strcasecmp(argv[0],"min-slaves-to-write") ||
!strcasecmp(argv[0],"min-replicas-to-write")) && argc == 2)
{
} else if (!strcasecmp(argv[0],"min-slaves-to-write") && argc == 2) {
server.repl_min_slaves_to_write = atoi(argv[1]);
if (server.repl_min_slaves_to_write < 0) {
err = "Invalid value for min-replicas-to-write."; goto loaderr;
err = "Invalid value for min-slaves-to-write."; goto loaderr;
}
} else if ((!strcasecmp(argv[0],"min-slaves-max-lag") ||
!strcasecmp(argv[0],"min-replicas-max-lag")) && argc == 2)
{
} else if (!strcasecmp(argv[0],"min-slaves-max-lag") && argc == 2) {
server.repl_min_slaves_max_lag = atoi(argv[1]);
if (server.repl_min_slaves_max_lag < 0) {
err = "Invalid value for min-replicas-max-lag."; goto loaderr;
err = "Invalid value for min-slaves-max-lag."; goto loaderr;
}
} else if (!strcasecmp(argv[0],"notify-keyspace-events") && argc == 2) {
int flags = keyspaceEventsStringToFlags(argv[1]);
@@ -809,7 +780,7 @@ void loadServerConfigFromString(char *config) {
if (server.cluster_enabled && server.masterhost) {
linenum = slaveof_linenum;
i = linenum-1;
err = "replicaof directive not allowed in cluster mode";
err = "slaveof directive not allowed in cluster mode";
goto loaderr;
}
@@ -893,10 +864,6 @@ void loadServerConfig(char *filename, char *options) {
#define config_set_special_field(_name) \
} else if (!strcasecmp(c->argv[2]->ptr,_name)) {
#define config_set_special_field_with_alias(_name1,_name2) \
} else if (!strcasecmp(c->argv[2]->ptr,_name1) || \
!strcasecmp(c->argv[2]->ptr,_name2)) {
#define config_set_else } else
void configSetCommand(client *c) {
@@ -1056,9 +1023,7 @@ void configSetCommand(client *c) {
if (flags == -1) goto badfmt;
server.notify_keyspace_events = flags;
} config_set_special_field_with_alias("slave-announce-ip",
"replica-announce-ip")
{
} config_set_special_field("slave-announce-ip") {
zfree(server.slave_announce_ip);
server.slave_announce_ip = ((char*)o->ptr)[0] ? zstrdup(o->ptr) : NULL;
@@ -1074,8 +1039,6 @@ void configSetCommand(client *c) {
"cluster-require-full-coverage",server.cluster_require_full_coverage) {
} config_set_bool_field(
"cluster-slave-no-failover",server.cluster_slave_no_failover) {
} config_set_bool_field(
"cluster-replica-no-failover",server.cluster_slave_no_failover) {
} config_set_bool_field(
"aof-rewrite-incremental-fsync",server.aof_rewrite_incremental_fsync) {
} config_set_bool_field(
@@ -1086,16 +1049,10 @@ void configSetCommand(client *c) {
"aof-use-rdb-preamble",server.aof_use_rdb_preamble) {
} config_set_bool_field(
"slave-serve-stale-data",server.repl_serve_stale_data) {
} config_set_bool_field(
"replica-serve-stale-data",server.repl_serve_stale_data) {
} config_set_bool_field(
"slave-read-only",server.repl_slave_ro) {
} config_set_bool_field(
"replica-read-only",server.repl_slave_ro) {
} config_set_bool_field(
"slave-ignore-maxmemory",server.repl_slave_ignore_maxmemory) {
} config_set_bool_field(
"replica-ignore-maxmemory",server.repl_slave_ignore_maxmemory) {
} config_set_bool_field(
"activerehashing",server.activerehashing) {
} config_set_bool_field(
@@ -1123,8 +1080,6 @@ void configSetCommand(client *c) {
"lazyfree-lazy-server-del",server.lazyfree_lazy_server_del) {
} config_set_bool_field(
"slave-lazy-flush",server.repl_slave_lazy_flush) {
} config_set_bool_field(
"replica-lazy-flush",server.repl_slave_lazy_flush) {
} config_set_bool_field(
"no-appendfsync-on-rewrite",server.aof_no_fsync_on_rewrite) {
} config_set_bool_field(
@@ -1188,8 +1143,6 @@ void configSetCommand(client *c) {
"latency-monitor-threshold",server.latency_monitor_threshold,0,LLONG_MAX){
} config_set_numerical_field(
"repl-ping-slave-period",server.repl_ping_slave_period,1,INT_MAX) {
} config_set_numerical_field(
"repl-ping-replica-period",server.repl_ping_slave_period,1,INT_MAX) {
} config_set_numerical_field(
"repl-timeout",server.repl_timeout,1,INT_MAX) {
} config_set_numerical_field(
@@ -1198,24 +1151,14 @@ void configSetCommand(client *c) {
"repl-diskless-sync-delay",server.repl_diskless_sync_delay,0,INT_MAX) {
} config_set_numerical_field(
"slave-priority",server.slave_priority,0,INT_MAX) {
} config_set_numerical_field(
"replica-priority",server.slave_priority,0,INT_MAX) {
} config_set_numerical_field(
"slave-announce-port",server.slave_announce_port,0,65535) {
} config_set_numerical_field(
"replica-announce-port",server.slave_announce_port,0,65535) {
} config_set_numerical_field(
"min-slaves-to-write",server.repl_min_slaves_to_write,0,INT_MAX) {
refreshGoodSlavesCount();
} config_set_numerical_field(
"min-replicas-to-write",server.repl_min_slaves_to_write,0,INT_MAX) {
refreshGoodSlavesCount();
} config_set_numerical_field(
"min-slaves-max-lag",server.repl_min_slaves_max_lag,0,INT_MAX) {
refreshGoodSlavesCount();
} config_set_numerical_field(
"min-replicas-max-lag",server.repl_min_slaves_max_lag,0,INT_MAX) {
refreshGoodSlavesCount();
} config_set_numerical_field(
"cluster-node-timeout",server.cluster_node_timeout,0,LLONG_MAX) {
} config_set_numerical_field(
@@ -1226,8 +1169,6 @@ void configSetCommand(client *c) {
"cluster-migration-barrier",server.cluster_migration_barrier,0,INT_MAX){
} config_set_numerical_field(
"cluster-slave-validity-factor",server.cluster_slave_validity_factor,0,INT_MAX) {
} config_set_numerical_field(
"cluster-replica-validity-factor",server.cluster_slave_validity_factor,0,INT_MAX) {
} config_set_numerical_field(
"hz",server.config_hz,0,INT_MAX) {
/* Hz is more an hint from the user, so we accept values out of range
@@ -1339,7 +1280,6 @@ void configGetCommand(client *c) {
config_get_string_field("logfile",server.logfile);
config_get_string_field("pidfile",server.pidfile);
config_get_string_field("slave-announce-ip",server.slave_announce_ip);
config_get_string_field("replica-announce-ip",server.slave_announce_ip);
/* Numerical values */
config_get_numerical_field("maxmemory",server.maxmemory);
@@ -1392,25 +1332,19 @@ void configGetCommand(client *c) {
config_get_numerical_field("tcp-backlog",server.tcp_backlog);
config_get_numerical_field("databases",server.dbnum);
config_get_numerical_field("repl-ping-slave-period",server.repl_ping_slave_period);
config_get_numerical_field("repl-ping-replica-period",server.repl_ping_slave_period);
config_get_numerical_field("repl-timeout",server.repl_timeout);
config_get_numerical_field("repl-backlog-size",server.repl_backlog_size);
config_get_numerical_field("repl-backlog-ttl",server.repl_backlog_time_limit);
config_get_numerical_field("maxclients",server.maxclients);
config_get_numerical_field("watchdog-period",server.watchdog_period);
config_get_numerical_field("slave-priority",server.slave_priority);
config_get_numerical_field("replica-priority",server.slave_priority);
config_get_numerical_field("slave-announce-port",server.slave_announce_port);
config_get_numerical_field("replica-announce-port",server.slave_announce_port);
config_get_numerical_field("min-slaves-to-write",server.repl_min_slaves_to_write);
config_get_numerical_field("min-replicas-to-write",server.repl_min_slaves_to_write);
config_get_numerical_field("min-slaves-max-lag",server.repl_min_slaves_max_lag);
config_get_numerical_field("min-replicas-max-lag",server.repl_min_slaves_max_lag);
config_get_numerical_field("hz",server.config_hz);
config_get_numerical_field("cluster-node-timeout",server.cluster_node_timeout);
config_get_numerical_field("cluster-migration-barrier",server.cluster_migration_barrier);
config_get_numerical_field("cluster-slave-validity-factor",server.cluster_slave_validity_factor);
config_get_numerical_field("cluster-replica-validity-factor",server.cluster_slave_validity_factor);
config_get_numerical_field("repl-diskless-sync-delay",server.repl_diskless_sync_delay);
config_get_numerical_field("tcp-keepalive",server.tcpkeepalive);
@@ -1419,22 +1353,14 @@ void configGetCommand(client *c) {
server.cluster_require_full_coverage);
config_get_bool_field("cluster-slave-no-failover",
server.cluster_slave_no_failover);
config_get_bool_field("cluster-replica-no-failover",
server.cluster_slave_no_failover);
config_get_bool_field("no-appendfsync-on-rewrite",
server.aof_no_fsync_on_rewrite);
config_get_bool_field("slave-serve-stale-data",
server.repl_serve_stale_data);
config_get_bool_field("replica-serve-stale-data",
server.repl_serve_stale_data);
config_get_bool_field("slave-read-only",
server.repl_slave_ro);
config_get_bool_field("replica-read-only",
server.repl_slave_ro);
config_get_bool_field("slave-ignore-maxmemory",
server.repl_slave_ignore_maxmemory);
config_get_bool_field("replica-ignore-maxmemory",
server.repl_slave_ignore_maxmemory);
config_get_bool_field("stop-writes-on-bgsave-error",
server.stop_writes_on_bgsave_err);
config_get_bool_field("daemonize", server.daemonize);
@@ -1463,8 +1389,6 @@ void configGetCommand(client *c) {
server.lazyfree_lazy_server_del);
config_get_bool_field("slave-lazy-flush",
server.repl_slave_lazy_flush);
config_get_bool_field("replica-lazy-flush",
server.repl_slave_lazy_flush);
config_get_bool_field("dynamic-hz",
server.dynamic_hz);
@@ -1538,14 +1462,10 @@ void configGetCommand(client *c) {
addReplyBulkCString(c,buf);
matches++;
}
if (stringmatch(pattern,"slaveof",1) ||
stringmatch(pattern,"replicaof",1))
{
char *optname = stringmatch(pattern,"slaveof",1) ?
"slaveof" : "replicaof";
if (stringmatch(pattern,"slaveof",1)) {
char buf[256];
addReplyBulkCString(c,optname);
addReplyBulkCString(c,"slaveof");
if (server.masterhost)
snprintf(buf,sizeof(buf),"%s %d",
server.masterhost, server.masterport);
@@ -1701,20 +1621,8 @@ struct rewriteConfigState *rewriteConfigReadOldFile(char *path) {
/* Now we populate the state according to the content of this line.
* Append the line and populate the option -> line numbers map. */
rewriteConfigAppendLine(state,line);
/* Translate options using the word "slave" to the corresponding name
* "replica", before adding such option to the config name -> lines
* mapping. */
char *p = strstr(argv[0],"slave");
if (p) {
sds alt = sdsempty();
alt = sdscatlen(alt,argv[0],p-argv[0]);;
alt = sdscatlen(alt,"replica",7);
alt = sdscatlen(alt,p+5,strlen(p+5));
sdsfree(argv[0]);
argv[0] = alt;
}
rewriteConfigAddLineNumberToOption(state,argv[0],linenum);
sdsfreesplitres(argv,argc);
}
fclose(fp);
@@ -1901,14 +1809,15 @@ void rewriteConfigDirOption(struct rewriteConfigState *state) {
}
/* Rewrite the slaveof option. */
void rewriteConfigSlaveofOption(struct rewriteConfigState *state, char *option) {
void rewriteConfigSlaveofOption(struct rewriteConfigState *state) {
char *option = "slaveof";
sds line;
/* If this is a master, we want all the slaveof config options
* in the file to be removed. Note that if this is a cluster instance
* we don't want a slaveof directive inside redis.conf. */
if (server.cluster_enabled || server.masterhost == NULL) {
rewriteConfigMarkAsProcessed(state,option);
rewriteConfigMarkAsProcessed(state,"slaveof");
return;
}
line = sdscatprintf(sdsempty(),"%s %s %d", option,
@@ -1950,10 +1859,8 @@ void rewriteConfigClientoutputbufferlimitOption(struct rewriteConfigState *state
rewriteConfigFormatMemory(soft,sizeof(soft),
server.client_obuf_limits[j].soft_limit_bytes);
char *typename = getClientTypeName(j);
if (!strcmp(typename,"slave")) typename = "replica";
line = sdscatprintf(sdsempty(),"%s %s %s %s %ld",
option, typename, hard, soft,
option, getClientTypeName(j), hard, soft,
(long) server.client_obuf_limits[j].soft_limit_seconds);
rewriteConfigRewriteLine(state,option,line,force);
}
@@ -2131,7 +2038,7 @@ int rewriteConfig(char *path) {
rewriteConfigOctalOption(state,"unixsocketperm",server.unixsocketperm,CONFIG_DEFAULT_UNIX_SOCKET_PERM);
rewriteConfigNumericalOption(state,"timeout",server.maxidletime,CONFIG_DEFAULT_CLIENT_TIMEOUT);
rewriteConfigNumericalOption(state,"tcp-keepalive",server.tcpkeepalive,CONFIG_DEFAULT_TCP_KEEPALIVE);
rewriteConfigNumericalOption(state,"replica-announce-port",server.slave_announce_port,CONFIG_DEFAULT_SLAVE_ANNOUNCE_PORT);
rewriteConfigNumericalOption(state,"slave-announce-port",server.slave_announce_port,CONFIG_DEFAULT_SLAVE_ANNOUNCE_PORT);
rewriteConfigEnumOption(state,"loglevel",server.verbosity,loglevel_enum,CONFIG_DEFAULT_VERBOSITY);
rewriteConfigStringOption(state,"logfile",server.logfile,CONFIG_DEFAULT_LOGFILE);
rewriteConfigYesNoOption(state,"syslog-enabled",server.syslog_enabled,CONFIG_DEFAULT_SYSLOG_ENABLED);
@@ -2144,23 +2051,23 @@ int rewriteConfig(char *path) {
rewriteConfigYesNoOption(state,"rdbchecksum",server.rdb_checksum,CONFIG_DEFAULT_RDB_CHECKSUM);
rewriteConfigStringOption(state,"dbfilename",server.rdb_filename,CONFIG_DEFAULT_RDB_FILENAME);
rewriteConfigDirOption(state);
rewriteConfigSlaveofOption(state,"replicaof");
rewriteConfigStringOption(state,"replica-announce-ip",server.slave_announce_ip,CONFIG_DEFAULT_SLAVE_ANNOUNCE_IP);
rewriteConfigSlaveofOption(state);
rewriteConfigStringOption(state,"slave-announce-ip",server.slave_announce_ip,CONFIG_DEFAULT_SLAVE_ANNOUNCE_IP);
rewriteConfigStringOption(state,"masterauth",server.masterauth,NULL);
rewriteConfigStringOption(state,"cluster-announce-ip",server.cluster_announce_ip,NULL);
rewriteConfigYesNoOption(state,"replica-serve-stale-data",server.repl_serve_stale_data,CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA);
rewriteConfigYesNoOption(state,"replica-read-only",server.repl_slave_ro,CONFIG_DEFAULT_SLAVE_READ_ONLY);
rewriteConfigYesNoOption(state,"replica-ignore-maxmemory",server.repl_slave_ignore_maxmemory,CONFIG_DEFAULT_SLAVE_IGNORE_MAXMEMORY);
rewriteConfigNumericalOption(state,"repl-ping-replica-period",server.repl_ping_slave_period,CONFIG_DEFAULT_REPL_PING_SLAVE_PERIOD);
rewriteConfigYesNoOption(state,"slave-serve-stale-data",server.repl_serve_stale_data,CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA);
rewriteConfigYesNoOption(state,"slave-read-only",server.repl_slave_ro,CONFIG_DEFAULT_SLAVE_READ_ONLY);
rewriteConfigYesNoOption(state,"slave-ignore-maxmemory",server.repl_slave_ignore_maxmemory,CONFIG_DEFAULT_SLAVE_IGNORE_MAXMEMORY);
rewriteConfigNumericalOption(state,"repl-ping-slave-period",server.repl_ping_slave_period,CONFIG_DEFAULT_REPL_PING_SLAVE_PERIOD);
rewriteConfigNumericalOption(state,"repl-timeout",server.repl_timeout,CONFIG_DEFAULT_REPL_TIMEOUT);
rewriteConfigBytesOption(state,"repl-backlog-size",server.repl_backlog_size,CONFIG_DEFAULT_REPL_BACKLOG_SIZE);
rewriteConfigBytesOption(state,"repl-backlog-ttl",server.repl_backlog_time_limit,CONFIG_DEFAULT_REPL_BACKLOG_TIME_LIMIT);
rewriteConfigYesNoOption(state,"repl-disable-tcp-nodelay",server.repl_disable_tcp_nodelay,CONFIG_DEFAULT_REPL_DISABLE_TCP_NODELAY);
rewriteConfigYesNoOption(state,"repl-diskless-sync",server.repl_diskless_sync,CONFIG_DEFAULT_REPL_DISKLESS_SYNC);
rewriteConfigNumericalOption(state,"repl-diskless-sync-delay",server.repl_diskless_sync_delay,CONFIG_DEFAULT_REPL_DISKLESS_SYNC_DELAY);
rewriteConfigNumericalOption(state,"replica-priority",server.slave_priority,CONFIG_DEFAULT_SLAVE_PRIORITY);
rewriteConfigNumericalOption(state,"min-replicas-to-write",server.repl_min_slaves_to_write,CONFIG_DEFAULT_MIN_SLAVES_TO_WRITE);
rewriteConfigNumericalOption(state,"min-replicas-max-lag",server.repl_min_slaves_max_lag,CONFIG_DEFAULT_MIN_SLAVES_MAX_LAG);
rewriteConfigNumericalOption(state,"slave-priority",server.slave_priority,CONFIG_DEFAULT_SLAVE_PRIORITY);
rewriteConfigNumericalOption(state,"min-slaves-to-write",server.repl_min_slaves_to_write,CONFIG_DEFAULT_MIN_SLAVES_TO_WRITE);
rewriteConfigNumericalOption(state,"min-slaves-max-lag",server.repl_min_slaves_max_lag,CONFIG_DEFAULT_MIN_SLAVES_MAX_LAG);
rewriteConfigStringOption(state,"requirepass",server.requirepass,NULL);
rewriteConfigNumericalOption(state,"maxclients",server.maxclients,CONFIG_DEFAULT_MAX_CLIENTS);
rewriteConfigBytesOption(state,"maxmemory",server.maxmemory,CONFIG_DEFAULT_MAXMEMORY);
@@ -2186,10 +2093,10 @@ int rewriteConfig(char *path) {
rewriteConfigYesNoOption(state,"cluster-enabled",server.cluster_enabled,0);
rewriteConfigStringOption(state,"cluster-config-file",server.cluster_configfile,CONFIG_DEFAULT_CLUSTER_CONFIG_FILE);
rewriteConfigYesNoOption(state,"cluster-require-full-coverage",server.cluster_require_full_coverage,CLUSTER_DEFAULT_REQUIRE_FULL_COVERAGE);
rewriteConfigYesNoOption(state,"cluster-replica-no-failover",server.cluster_slave_no_failover,CLUSTER_DEFAULT_SLAVE_NO_FAILOVER);
rewriteConfigYesNoOption(state,"cluster-slave-no-failover",server.cluster_slave_no_failover,CLUSTER_DEFAULT_SLAVE_NO_FAILOVER);
rewriteConfigNumericalOption(state,"cluster-node-timeout",server.cluster_node_timeout,CLUSTER_DEFAULT_NODE_TIMEOUT);
rewriteConfigNumericalOption(state,"cluster-migration-barrier",server.cluster_migration_barrier,CLUSTER_DEFAULT_MIGRATION_BARRIER);
rewriteConfigNumericalOption(state,"cluster-replica-validity-factor",server.cluster_slave_validity_factor,CLUSTER_DEFAULT_SLAVE_VALIDITY);
rewriteConfigNumericalOption(state,"cluster-slave-validity-factor",server.cluster_slave_validity_factor,CLUSTER_DEFAULT_SLAVE_VALIDITY);
rewriteConfigNumericalOption(state,"slowlog-log-slower-than",server.slowlog_log_slower_than,CONFIG_DEFAULT_SLOWLOG_LOG_SLOWER_THAN);
rewriteConfigNumericalOption(state,"latency-monitor-threshold",server.latency_monitor_threshold,CONFIG_DEFAULT_LATENCY_MONITOR_THRESHOLD);
rewriteConfigNumericalOption(state,"slowlog-max-len",server.slowlog_max_len,CONFIG_DEFAULT_SLOWLOG_MAX_LEN);
@@ -2217,7 +2124,7 @@ int rewriteConfig(char *path) {
rewriteConfigYesNoOption(state,"lazyfree-lazy-eviction",server.lazyfree_lazy_eviction,CONFIG_DEFAULT_LAZYFREE_LAZY_EVICTION);
rewriteConfigYesNoOption(state,"lazyfree-lazy-expire",server.lazyfree_lazy_expire,CONFIG_DEFAULT_LAZYFREE_LAZY_EXPIRE);
rewriteConfigYesNoOption(state,"lazyfree-lazy-server-del",server.lazyfree_lazy_server_del,CONFIG_DEFAULT_LAZYFREE_LAZY_SERVER_DEL);
rewriteConfigYesNoOption(state,"replica-lazy-flush",server.repl_slave_lazy_flush,CONFIG_DEFAULT_SLAVE_LAZY_FLUSH);
rewriteConfigYesNoOption(state,"slave-lazy-flush",server.repl_slave_lazy_flush,CONFIG_DEFAULT_SLAVE_LAZY_FLUSH);
rewriteConfigYesNoOption(state,"dynamic-hz",server.dynamic_hz,CONFIG_DEFAULT_DYNAMIC_HZ);
/* Rewrite Sentinel config if in Sentinel mode. */
+21 -31
View File
@@ -38,8 +38,6 @@
* C-level DB API
*----------------------------------------------------------------------------*/
int keyIsExpired(redisDb *db, robj *key);
/* Update LFU when an object is accessed.
* Firstly, decrement the counter if the decrement time is reached.
* Then logarithmically increment the counter, and update the access time. */
@@ -104,10 +102,7 @@ robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
/* Key expired. If we are in the context of a master, expireIfNeeded()
* returns 0 only when the key does not exist at all, so it's safe
* to return NULL ASAP. */
if (server.masterhost == NULL) {
server.stat_keyspace_misses++;
return NULL;
}
if (server.masterhost == NULL) return NULL;
/* However if we are in the context of a slave, expireIfNeeded() will
* not really try to expire the key, it only returns information
@@ -126,7 +121,6 @@ robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
server.current_client->cmd &&
server.current_client->cmd->flags & CMD_READONLY)
{
server.stat_keyspace_misses++;
return NULL;
}
}
@@ -549,7 +543,7 @@ void keysCommand(client *c) {
if (allkeys || stringmatchlen(pattern,plen,key,sdslen(key),0)) {
keyobj = createStringObject(key,sdslen(key));
if (!keyIsExpired(c->db,keyobj)) {
if (expireIfNeeded(c->db,keyobj) == 0) {
addReplyBulk(c,keyobj);
numkeys++;
}
@@ -1126,25 +1120,6 @@ void propagateExpire(redisDb *db, robj *key, int lazy) {
decrRefCount(argv[1]);
}
/* Check if the key is expired. */
int keyIsExpired(redisDb *db, robj *key) {
mstime_t when = getExpire(db,key);
if (when < 0) return 0; /* No expire for this key */
/* Don't expire anything while loading. It will be done later. */
if (server.loading) return 0;
/* If we are in the context of a Lua script, we pretend that time is
* blocked to when the Lua script started. This way a key can expire
* only the first time it is accessed and not in the middle of the
* script execution, making propagation to slaves / AOF consistent.
* See issue #1525 on Github for more information. */
mstime_t now = server.lua_caller ? server.lua_time_start : mstime();
return now > when;
}
/* This function is called when we are going to perform some operation
* in a given key, but such key may be already logically expired even if
* it still exists in the database. The main way this function is called
@@ -1165,17 +1140,32 @@ int keyIsExpired(redisDb *db, robj *key) {
* The return value of the function is 0 if the key is still valid,
* otherwise the function returns 1 if the key is expired. */
int expireIfNeeded(redisDb *db, robj *key) {
if (!keyIsExpired(db,key)) return 0;
mstime_t when = getExpire(db,key);
mstime_t now;
/* If we are running in the context of a slave, instead of
* evicting the expired key from the database, we return ASAP:
if (when < 0) return 0; /* No expire for this key */
/* Don't expire anything while loading. It will be done later. */
if (server.loading) return 0;
/* If we are in the context of a Lua script, we pretend that time is
* blocked to when the Lua script started. This way a key can expire
* only the first time it is accessed and not in the middle of the
* script execution, making propagation to slaves / AOF consistent.
* See issue #1525 on Github for more information. */
now = server.lua_caller ? server.lua_time_start : mstime();
/* If we are running in the context of a slave, return ASAP:
* the slave key expiration is controlled by the master that will
* send us synthesized DEL operations for expired keys.
*
* Still we try to return the right information to the caller,
* that is, 0 if we think the key should be still valid, 1 if
* we think the key is expired at this time. */
if (server.masterhost != NULL) return 1;
if (server.masterhost != NULL) return now > when;
/* Return when this key has not expired */
if (now <= when) return 0;
/* Delete the key */
server.stat_expiredkeys++;
+2 -8
View File
@@ -345,10 +345,7 @@ NULL
return;
}
emptyDb(-1,EMPTYDB_NO_FLAGS,NULL);
protectClient(c);
int ret = rdbLoad(server.rdb_filename,NULL);
unprotectClient(c);
if (ret != C_OK) {
if (rdbLoad(server.rdb_filename,NULL) != C_OK) {
addReplyError(c,"Error trying to load the RDB dump");
return;
}
@@ -357,10 +354,7 @@ NULL
} else if (!strcasecmp(c->argv[1]->ptr,"loadaof")) {
if (server.aof_state != AOF_OFF) flushAppendOnlyFile(1);
emptyDb(-1,EMPTYDB_NO_FLAGS,NULL);
protectClient(c);
int ret = loadAppendOnlyFile(server.aof_filename);
unprotectClient(c);
if (ret != C_OK) {
if (loadAppendOnlyFile(server.aof_filename) != C_OK) {
addReply(c,shared.err);
return;
}
+1 -1
View File
@@ -364,7 +364,7 @@ size_t freeMemoryGetNotCountedMemory(void) {
}
}
if (server.aof_state != AOF_OFF) {
overhead += sdsalloc(server.aof_buf)+aofRewriteBufferSize();
overhead += sdslen(server.aof_buf)+aofRewriteBufferSize();
}
return overhead;
}
+2 -2
View File
@@ -127,8 +127,8 @@ int geohashEncode(const GeoHashRange *long_range, const GeoHashRange *lat_range,
/* Return an error when trying to index outside the supported
* constraints. */
if (longitude > GEO_LONG_MAX || longitude < GEO_LONG_MIN ||
latitude > GEO_LAT_MAX || latitude < GEO_LAT_MIN) return 0;
if (longitude > 180 || longitude < -180 ||
latitude > 85.05112878 || latitude < -85.05112878) return 0;
hash->bits = 0;
hash->step = step;
+7 -57
View File
@@ -98,11 +98,6 @@ struct commandHelp {
"Get the current connection name",
9,
"2.6.9" },
{ "CLIENT ID",
"-",
"Returns the client ID for the current connection",
9,
"5.0.0" },
{ "CLIENT KILL",
"[ip:port] [ID client-id] [TYPE normal|master|slave|pubsub] [ADDR ip:port] [SKIPME yes/no]",
"Kill the connection of a client",
@@ -128,11 +123,6 @@ struct commandHelp {
"Set the current connection name",
9,
"2.6.9" },
{ "CLIENT UNBLOCK",
"client-id [TIMEOUT|ERROR]",
"Unblock a client blocked in a blocking command from a different connection",
9,
"5.0.0" },
{ "CLUSTER ADDSLOTS",
"slot [slot ...]",
"Assign new hash slots to receiving node",
@@ -155,7 +145,7 @@ struct commandHelp {
"3.0.0" },
{ "CLUSTER FAILOVER",
"[FORCE|TAKEOVER]",
"Forces a replica to perform a manual failover of its master.",
"Forces a slave to perform a manual failover of its master.",
12,
"3.0.0" },
{ "CLUSTER FORGET",
@@ -188,14 +178,9 @@ struct commandHelp {
"Get Cluster config for the node",
12,
"3.0.0" },
{ "CLUSTER REPLICAS",
"node-id",
"List replica nodes of the specified master node",
12,
"5.0.0" },
{ "CLUSTER REPLICATE",
"node-id",
"Reconfigure a node as a replica of the specified master node",
"Reconfigure a node as a slave of the specified master node",
12,
"3.0.0" },
{ "CLUSTER RESET",
@@ -220,7 +205,7 @@ struct commandHelp {
"3.0.0" },
{ "CLUSTER SLAVES",
"node-id",
"List replica nodes of the specified master node",
"List slave nodes of the specified master node",
12,
"3.0.0" },
{ "CLUSTER SLOTS",
@@ -705,12 +690,12 @@ struct commandHelp {
"1.0.0" },
{ "READONLY",
"-",
"Enables read queries for a connection to a cluster replica node",
"Enables read queries for a connection to a cluster slave node",
12,
"3.0.0" },
{ "READWRITE",
"-",
"Disables read queries for a connection to a cluster replica node",
"Disables read queries for a connection to a cluster slave node",
12,
"3.0.0" },
{ "RENAME",
@@ -723,11 +708,6 @@ struct commandHelp {
"Rename a key, only if the new key does not exist",
0,
"1.0.0" },
{ "REPLICAOF",
"host port",
"Make the server a replica of another instance, or promote it as master.",
9,
"5.0.0" },
{ "RESTORE",
"key ttl serialized-value [REPLACE]",
"Create a key using the provided serialized value, previously obtained using DUMP.",
@@ -865,7 +845,7 @@ struct commandHelp {
"1.0.0" },
{ "SLAVEOF",
"host port",
"Make the server a replica of another instance, or promote it as master. Deprecated starting with Redis 5. Use REPLICAOF instead.",
"Make the server a slave of another instance, or promote it as master",
9,
"1.0.0" },
{ "SLOWLOG",
@@ -974,7 +954,7 @@ struct commandHelp {
7,
"2.2.0" },
{ "WAIT",
"numreplicas timeout",
"numslaves timeout",
"Wait for the synchronous replication of all the write commands sent in the context of the current connection",
0,
"3.0.0" },
@@ -983,36 +963,11 @@ struct commandHelp {
"Watch the given keys to determine execution of the MULTI/EXEC block",
7,
"2.2.0" },
{ "XACK",
"key group ID [ID ...]",
"Marks a pending message as correctly processed, effectively removing it from the pending entries list of the consumer group. Return value of the command is the number of messages successfully acknowledged, that is, the IDs we were actually able to resolve in the PEL.",
14,
"5.0.0" },
{ "XADD",
"key ID field string [field string ...]",
"Appends a new entry to a stream",
14,
"5.0.0" },
{ "XCLAIM",
"key group consumer min-idle-time ID [ID ...] [IDLE ms] [TIME ms-unix-time] [RETRYCOUNT count] [force] [justid]",
"Changes (or acquires) ownership of a message in a consumer group, as if the message was delivered to the specified consumer.",
14,
"5.0.0" },
{ "XDEL",
"key ID [ID ...]",
"Removes the specified entries from the stream. Returns the number of items actually deleted, that may be different from the number of IDs passed in case certain IDs do not exist.",
14,
"5.0.0" },
{ "XGROUP",
"[CREATE key groupname id-or-$] [SETID key id-or-$] [DESTROY key groupname] [DELCONSUMER key groupname consumername]",
"Create, destroy, and manage consumer groups.",
14,
"5.0.0" },
{ "XINFO",
"[CONSUMERS key groupname] [GROUPS key] [STREAM key] [HELP]",
"Get information on streams and consumer groups",
14,
"5.0.0" },
{ "XLEN",
"key",
"Return the number of entires in a stream",
@@ -1043,11 +998,6 @@ struct commandHelp {
"Return a range of elements in a stream, with IDs matching the specified IDs interval, in reverse order (from greater to smaller IDs) compared to XRANGE",
14,
"5.0.0" },
{ "XTRIM",
"key MAXLEN [~] count",
"Trims the stream to (approximately if '~' is passed) a certain size",
14,
"5.0.0" },
{ "ZADD",
"key [NX|XX] [CH] [INCR] score member [score member ...]",
"Add one or more members to a sorted set, or update its score if it already exists",
+1 -1
View File
@@ -123,7 +123,7 @@ static uint8_t intsetSearch(intset *is, int64_t value, uint32_t *pos) {
} else {
/* Check for the case where we know we cannot find the value,
* but do know the insert position. */
if (value > _intsetGet(is,max)) {
if (value > _intsetGet(is,intrev32ifbe(is->length)-1)) {
if (pos) *pos = intrev32ifbe(is->length);
return 0;
} else if (value < _intsetGet(is,0)) {
+3 -16
View File
@@ -560,23 +560,12 @@ sds latencyCommandGenSparkeline(char *event, struct latencyTimeSeries *ts) {
/* LATENCY command implementations.
*
* LATENCY HISTORY: return time-latency samples for the specified event.
* LATENCY SAMPLES: return time-latency samples for the specified event.
* LATENCY LATEST: return the latest latency for all the events classes.
* LATENCY DOCTOR: returns a human readable analysis of instance latency.
* LATENCY DOCTOR: returns an human readable analysis of instance latency.
* LATENCY GRAPH: provide an ASCII graph of the latency of the specified event.
* LATENCY RESET: reset data of a specified event or all the data if no event provided.
*/
void latencyCommand(client *c) {
const char *help[] = {
"DOCTOR -- Returns a human readable latency analysis report.",
"GRAPH <event> -- Returns an ASCII latency graph for the event class.",
"HISTORY <event> -- Returns time-latency samples for the event class.",
"LATEST -- Returns the latest latency samples for all events.",
"RESET [event ...] -- Resets latency data of one or more event classes.",
" (default: reset all data for all event classes)",
"HELP -- Prints this help.",
NULL
};
struct latencyTimeSeries *ts;
if (!strcasecmp(c->argv[1]->ptr,"history") && c->argc == 3) {
@@ -621,10 +610,8 @@ NULL
resets += latencyResetEvent(c->argv[j]->ptr);
addReplyLongLong(c,resets);
}
} else if (!strcasecmp(c->argv[1]->ptr,"help") && c->argc >= 2) {
addReplyHelp(c, help);
} else {
addReplySubcommandSyntaxError(c);
addReply(c,shared.syntaxerr);
}
return;
-20
View File
@@ -707,26 +707,6 @@ unsigned char *lpInsert(unsigned char *lp, unsigned char *ele, uint32_t size, un
}
}
lpSetTotalBytes(lp,new_listpack_bytes);
#if 0
/* This code path is normally disabled: what it does is to force listpack
* to return *always* a new pointer after performing some modification to
* the listpack, even if the previous allocation was enough. This is useful
* in order to spot bugs in code using listpacks: by doing so we can find
* if the caller forgets to set the new pointer where the listpack reference
* is stored, after an update. */
unsigned char *oldlp = lp;
lp = lp_malloc(new_listpack_bytes);
memcpy(lp,oldlp,new_listpack_bytes);
if (newp) {
unsigned long offset = (*newp)-oldlp;
*newp = lp + offset;
}
/* Make sure the old allocation contains garbage. */
memset(oldlp,'A',new_listpack_bytes);
lp_free(oldlp);
#endif
return lp;
}
-56
View File
@@ -1,56 +0,0 @@
/*
* Copyright (c) 2018, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*
* ----------------------------------------------------------------------------
*
* This file implements the LOLWUT command. The command should do something
* fun and interesting, and should be replaced by a new implementation at
* each new version of Redis.
*/
#include "server.h"
void lolwut5Command(client *c);
/* The default target for LOLWUT if no matching version was found.
* This is what unstable versions of Redis will display. */
void lolwutUnstableCommand(client *c) {
sds rendered = sdsnew("Redis ver. ");
rendered = sdscat(rendered,REDIS_VERSION);
rendered = sdscatlen(rendered,"\n",1);
addReplyBulkSds(c,rendered);
}
void lolwutCommand(client *c) {
char *v = REDIS_VERSION;
if ((v[0] == '5' && v[1] == '.') ||
(v[0] == '4' && v[1] == '.' && v[2] == '9'))
lolwut5Command(c);
else
lolwutUnstableCommand(c);
}
-282
View File
@@ -1,282 +0,0 @@
/*
* Copyright (c) 2018, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*
* ----------------------------------------------------------------------------
*
* This file implements the LOLWUT command. The command should do something
* fun and interesting, and should be replaced by a new implementation at
* each new version of Redis.
*/
#include "server.h"
#include <math.h>
/* This structure represents our canvas. Drawing functions will take a pointer
* to a canvas to write to it. Later the canvas can be rendered to a string
* suitable to be printed on the screen, using unicode Braille characters. */
typedef struct lwCanvas {
int width;
int height;
char *pixels;
} lwCanvas;
/* Translate a group of 8 pixels (2x4 vertical rectangle) to the corresponding
* braille character. The byte should correspond to the pixels arranged as
* follows, where 0 is the least significant bit, and 7 the most significant
* bit:
*
* 0 3
* 1 4
* 2 5
* 6 7
*
* The corresponding utf8 encoded character is set into the three bytes
* pointed by 'output'.
*/
#include <stdio.h>
void lwTranslatePixelsGroup(int byte, char *output) {
int code = 0x2800 + byte;
/* Convert to unicode. This is in the U0800-UFFFF range, so we need to
* emit it like this in three bytes:
* 1110xxxx 10xxxxxx 10xxxxxx. */
output[0] = 0xE0 | (code >> 12); /* 1110-xxxx */
output[1] = 0x80 | ((code >> 6) & 0x3F); /* 10-xxxxxx */
output[2] = 0x80 | (code & 0x3F); /* 10-xxxxxx */
}
/* Allocate and return a new canvas of the specified size. */
lwCanvas *lwCreateCanvas(int width, int height) {
lwCanvas *canvas = zmalloc(sizeof(*canvas));
canvas->width = width;
canvas->height = height;
canvas->pixels = zmalloc(width*height);
memset(canvas->pixels,0,width*height);
return canvas;
}
/* Free the canvas created by lwCreateCanvas(). */
void lwFreeCanvas(lwCanvas *canvas) {
zfree(canvas->pixels);
zfree(canvas);
}
/* Set a pixel to the specified color. Color is 0 or 1, where zero means no
* dot will be displyed, and 1 means dot will be displayed.
* Coordinates are arranged so that left-top corner is 0,0. You can write
* out of the size of the canvas without issues. */
void lwDrawPixel(lwCanvas *canvas, int x, int y, int color) {
if (x < 0 || x >= canvas->width ||
y < 0 || y >= canvas->height) return;
canvas->pixels[x+y*canvas->width] = color;
}
/* Return the value of the specified pixel on the canvas. */
int lwGetPixel(lwCanvas *canvas, int x, int y) {
if (x < 0 || x >= canvas->width ||
y < 0 || y >= canvas->height) return 0;
return canvas->pixels[x+y*canvas->width];
}
/* Draw a line from x1,y1 to x2,y2 using the Bresenham algorithm. */
void lwDrawLine(lwCanvas *canvas, int x1, int y1, int x2, int y2, int color) {
int dx = abs(x2-x1);
int dy = abs(y2-y1);
int sx = (x1 < x2) ? 1 : -1;
int sy = (y1 < y2) ? 1 : -1;
int err = dx-dy, e2;
while(1) {
lwDrawPixel(canvas,x1,y1,color);
if (x1 == x2 && y1 == y2) break;
e2 = err*2;
if (e2 > -dy) {
err -= dy;
x1 += sx;
}
if (e2 < dx) {
err += dx;
y1 += sy;
}
}
}
/* Draw a square centered at the specified x,y coordinates, with the specified
* rotation angle and size. In order to write a rotated square, we use the
* trivial fact that the parametric equation:
*
* x = sin(k)
* y = cos(k)
*
* Describes a circle for values going from 0 to 2*PI. So basically if we start
* at 45 degrees, that is k = PI/4, with the first point, and then we find
* the other three points incrementing K by PI/2 (90 degrees), we'll have the
* points of the square. In order to rotate the square, we just start with
* k = PI/4 + rotation_angle, and we are done.
*
* Of course the vanilla equations above will describe the square inside a
* circle of radius 1, so in order to draw larger squares we'll have to
* multiply the obtained coordinates, and then translate them. However this
* is much simpler than implementing the abstract concept of 2D shape and then
* performing the rotation/translation transformation, so for LOLWUT it's
* a good approach. */
void lwDrawSquare(lwCanvas *canvas, int x, int y, float size, float angle) {
int px[4], py[4];
/* Adjust the desired size according to the fact that the square inscribed
* into a circle of radius 1 has the side of length SQRT(2). This way
* size becomes a simple multiplication factor we can use with our
* coordinates to magnify them. */
size /= 1.4142135623;
size = round(size);
/* Compute the four points. */
float k = M_PI/4 + angle;
for (int j = 0; j < 4; j++) {
px[j] = round(sin(k) * size + x);
py[j] = round(cos(k) * size + y);
k += M_PI/2;
}
/* Draw the square. */
for (int j = 0; j < 4; j++)
lwDrawLine(canvas,px[j],py[j],px[(j+1)%4],py[(j+1)%4],1);
}
/* Schotter, the output of LOLWUT of Redis 5, is a computer graphic art piece
* generated by Georg Nees in the 60s. It explores the relationship between
* caos and order.
*
* The function creates the canvas itself, depending on the columns available
* in the output display and the number of squares per row and per column
* requested by the caller. */
lwCanvas *lwDrawSchotter(int console_cols, int squares_per_row, int squares_per_col) {
/* Calculate the canvas size. */
int canvas_width = console_cols*2;
int padding = canvas_width > 4 ? 2 : 0;
float square_side = (float)(canvas_width-padding*2) / squares_per_row;
int canvas_height = square_side * squares_per_col + padding*2;
lwCanvas *canvas = lwCreateCanvas(canvas_width, canvas_height);
for (int y = 0; y < squares_per_col; y++) {
for (int x = 0; x < squares_per_row; x++) {
int sx = x * square_side + square_side/2 + padding;
int sy = y * square_side + square_side/2 + padding;
/* Rotate and translate randomly as we go down to lower
* rows. */
float angle = 0;
if (y > 1) {
float r1 = (float)rand() / RAND_MAX / squares_per_col * y;
float r2 = (float)rand() / RAND_MAX / squares_per_col * y;
float r3 = (float)rand() / RAND_MAX / squares_per_col * y;
if (rand() % 2) r1 = -r1;
if (rand() % 2) r2 = -r2;
if (rand() % 2) r3 = -r3;
angle = r1;
sx += r2*square_side/3;
sy += r3*square_side/3;
}
lwDrawSquare(canvas,sx,sy,square_side,angle);
}
}
return canvas;
}
/* Converts the canvas to an SDS string representing the UTF8 characters to
* print to the terminal in order to obtain a graphical representaiton of the
* logical canvas. The actual returned string will require a terminal that is
* width/2 large and height/4 tall in order to hold the whole image without
* overflowing or scrolling, since each Barille character is 2x4. */
sds lwRenderCanvas(lwCanvas *canvas) {
sds text = sdsempty();
for (int y = 0; y < canvas->height; y += 4) {
for (int x = 0; x < canvas->width; x += 2) {
/* We need to emit groups of 8 bits according to a specific
* arrangement. See lwTranslatePixelsGroup() for more info. */
int byte = 0;
if (lwGetPixel(canvas,x,y)) byte |= (1<<0);
if (lwGetPixel(canvas,x,y+1)) byte |= (1<<1);
if (lwGetPixel(canvas,x,y+2)) byte |= (1<<2);
if (lwGetPixel(canvas,x+1,y)) byte |= (1<<3);
if (lwGetPixel(canvas,x+1,y+1)) byte |= (1<<4);
if (lwGetPixel(canvas,x+1,y+2)) byte |= (1<<5);
if (lwGetPixel(canvas,x,y+3)) byte |= (1<<6);
if (lwGetPixel(canvas,x+1,y+3)) byte |= (1<<7);
char unicode[3];
lwTranslatePixelsGroup(byte,unicode);
text = sdscatlen(text,unicode,3);
}
if (y != canvas->height-1) text = sdscatlen(text,"\n",1);
}
return text;
}
/* The LOLWUT command:
*
* LOLWUT [terminal columns] [squares-per-row] [squares-per-col]
*
* By default the command uses 66 columns, 8 squares per row, 12 squares
* per column.
*/
void lolwut5Command(client *c) {
long cols = 66;
long squares_per_row = 8;
long squares_per_col = 12;
/* Parse the optional arguments if any. */
if (c->argc > 1 &&
getLongFromObjectOrReply(c,c->argv[1],&cols,NULL) != C_OK)
return;
if (c->argc > 2 &&
getLongFromObjectOrReply(c,c->argv[2],&squares_per_row,NULL) != C_OK)
return;
if (c->argc > 3 &&
getLongFromObjectOrReply(c,c->argv[3],&squares_per_col,NULL) != C_OK)
return;
/* Limits. We want LOLWUT to be always reasonably fast and cheap to execute
* so we have maximum number of columns, rows, and output resulution. */
if (cols < 1) cols = 1;
if (cols > 1000) cols = 1000;
if (squares_per_row < 1) squares_per_row = 1;
if (squares_per_row > 200) squares_per_row = 200;
if (squares_per_col < 1) squares_per_col = 1;
if (squares_per_col > 200) squares_per_col = 200;
/* Generate some computer art and reply. */
lwCanvas *canvas = lwDrawSchotter(cols,squares_per_row,squares_per_col);
sds rendered = lwRenderCanvas(canvas);
rendered = sdscat(rendered,
"\nGeorg Nees - schotter, plotter on paper, 1968. Redis ver. ");
rendered = sdscat(rendered,REDIS_VERSION);
rendered = sdscatlen(rendered,"\n",1);
addReplyBulkSds(c,rendered);
lwFreeCanvas(canvas);
}
+4 -7
View File
@@ -52,10 +52,6 @@
#endif
#endif
#if defined(__GNUC__) && __GNUC__ >= 5
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wimplicit-fallthrough"
#endif
unsigned int
lzf_decompress (const void *const in_data, unsigned int in_len,
void *out_data, unsigned int out_len)
@@ -90,6 +86,8 @@ lzf_decompress (const void *const in_data, unsigned int in_len,
#ifdef lzf_movsb
lzf_movsb (op, ip, ctrl);
#else
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wimplicit-fallthrough"
switch (ctrl)
{
case 32: *op++ = *ip++; case 31: *op++ = *ip++; case 30: *op++ = *ip++; case 29: *op++ = *ip++;
@@ -101,6 +99,7 @@ lzf_decompress (const void *const in_data, unsigned int in_len,
case 8: *op++ = *ip++; case 7: *op++ = *ip++; case 6: *op++ = *ip++; case 5: *op++ = *ip++;
case 4: *op++ = *ip++; case 3: *op++ = *ip++; case 2: *op++ = *ip++; case 1: *op++ = *ip++;
}
#pragma GCC diagnostic pop
#endif
}
else /* back reference */
@@ -186,6 +185,4 @@ lzf_decompress (const void *const in_data, unsigned int in_len,
return op - (u8 *)out_data;
}
#if defined(__GNUC__) && __GNUC__ >= 5
#pragma GCC diagnostic pop
#endif
+29 -373
View File
@@ -64,7 +64,6 @@ struct AutoMemEntry {
#define REDISMODULE_AM_STRING 1
#define REDISMODULE_AM_REPLY 2
#define REDISMODULE_AM_FREED 3 /* Explicitly freed by user already. */
#define REDISMODULE_AM_DICT 4
/* The pool allocator block. Redis Modules can allocate memory via this special
* allocator that will automatically release it all once the callback returns.
@@ -242,21 +241,9 @@ typedef struct RedisModuleKeyspaceSubscriber {
/* The module keyspace notification subscribers list */
static list *moduleKeyspaceSubscribers;
/* Static client recycled for when we need to provide a context with a client
* in a situation where there is no client to provide. This avoidsallocating
* a new client per round. For instance this is used in the keyspace
* notifications, timers and cluster messages callbacks. */
static client *moduleFreeContextReusedClient;
/* Data structures related to the exported dictionary data structure. */
typedef struct RedisModuleDict {
rax *rax; /* The radix tree. */
} RedisModuleDict;
typedef struct RedisModuleDictIter {
RedisModuleDict *dict;
raxIterator ri;
} RedisModuleDictIter;
/* Static client recycled for all notification clients, to avoid allocating
* per round. */
static client *moduleKeyspaceSubscribersClient;
/* --------------------------------------------------------------------------
* Prototypes
@@ -269,7 +256,6 @@ robj **moduleCreateArgvFromUserFormat(const char *cmdname, const char *fmt, int
void moduleReplicateMultiIfNeeded(RedisModuleCtx *ctx);
void RM_ZsetRangeStop(RedisModuleKey *kp);
static void zsetKeyReset(RedisModuleKey *key);
void RM_FreeDict(RedisModuleCtx *ctx, RedisModuleDict *d);
/* --------------------------------------------------------------------------
* Heap allocation raw functions
@@ -488,7 +474,7 @@ void moduleHandlePropagationAfterCommandCallback(RedisModuleCtx *ctx) {
if (c->flags & CLIENT_LUA) return;
/* Handle the replication of the final EXEC, since whatever a command
* emits is always wrapped around MULTI/EXEC. */
* emits is always wrappered around MULTI/EXEC. */
if (ctx->flags & REDISMODULE_CTX_MULTI_EMITTED) {
robj *propargv[1];
propargv[0] = createStringObject("EXEC",4);
@@ -791,7 +777,6 @@ void autoMemoryCollect(RedisModuleCtx *ctx) {
case REDISMODULE_AM_STRING: decrRefCount(ptr); break;
case REDISMODULE_AM_REPLY: RM_FreeCallReply(ptr); break;
case REDISMODULE_AM_KEY: RM_CloseKey(ptr); break;
case REDISMODULE_AM_DICT: RM_FreeDict(NULL,ptr); break;
}
}
ctx->flags |= REDISMODULE_CTX_AUTO_MEMORY;
@@ -809,26 +794,19 @@ void autoMemoryCollect(RedisModuleCtx *ctx) {
* with RedisModule_FreeString(), unless automatic memory is enabled.
*
* The string is created by copying the `len` bytes starting
* at `ptr`. No reference is retained to the passed buffer.
*
* The module context 'ctx' is optional and may be NULL if you want to create
* a string out of the context scope. However in that case, the automatic
* memory management will not be available, and the string memory must be
* managed manually. */
* at `ptr`. No reference is retained to the passed buffer. */
RedisModuleString *RM_CreateString(RedisModuleCtx *ctx, const char *ptr, size_t len) {
RedisModuleString *o = createStringObject(ptr,len);
if (ctx != NULL) autoMemoryAdd(ctx,REDISMODULE_AM_STRING,o);
autoMemoryAdd(ctx,REDISMODULE_AM_STRING,o);
return o;
}
/* Create a new module string object from a printf format and arguments.
* The returned string must be freed with RedisModule_FreeString(), unless
* automatic memory is enabled.
*
* The string is created using the sds formatter function sdscatvprintf().
*
* The passed context 'ctx' may be NULL if necessary, see the
* RedisModule_CreateString() documentation for more info. */
* The string is created using the sds formatter function sdscatvprintf(). */
RedisModuleString *RM_CreateStringPrintf(RedisModuleCtx *ctx, const char *fmt, ...) {
sds s = sdsempty();
@@ -838,7 +816,7 @@ RedisModuleString *RM_CreateStringPrintf(RedisModuleCtx *ctx, const char *fmt, .
va_end(ap);
RedisModuleString *o = createObject(OBJ_STRING, s);
if (ctx != NULL) autoMemoryAdd(ctx,REDISMODULE_AM_STRING,o);
autoMemoryAdd(ctx,REDISMODULE_AM_STRING,o);
return o;
}
@@ -848,10 +826,7 @@ RedisModuleString *RM_CreateStringPrintf(RedisModuleCtx *ctx, const char *fmt, .
* integer instead of taking a buffer and its length.
*
* The returned string must be released with RedisModule_FreeString() or by
* enabling automatic memory management.
*
* The passed context 'ctx' may be NULL if necessary, see the
* RedisModule_CreateString() documentation for more info. */
* enabling automatic memory management. */
RedisModuleString *RM_CreateStringFromLongLong(RedisModuleCtx *ctx, long long ll) {
char buf[LONG_STR_SIZE];
size_t len = ll2string(buf,sizeof(buf),ll);
@@ -862,13 +837,10 @@ RedisModuleString *RM_CreateStringFromLongLong(RedisModuleCtx *ctx, long long ll
* RedisModuleString.
*
* The returned string must be released with RedisModule_FreeString() or by
* enabling automatic memory management.
*
* The passed context 'ctx' may be NULL if necessary, see the
* RedisModule_CreateString() documentation for more info. */
* enabling automatic memory management. */
RedisModuleString *RM_CreateStringFromString(RedisModuleCtx *ctx, const RedisModuleString *str) {
RedisModuleString *o = dupStringObject(str);
if (ctx != NULL) autoMemoryAdd(ctx,REDISMODULE_AM_STRING,o);
autoMemoryAdd(ctx,REDISMODULE_AM_STRING,o);
return o;
}
@@ -877,16 +849,10 @@ RedisModuleString *RM_CreateStringFromString(RedisModuleCtx *ctx, const RedisMod
*
* It is possible to call this function even when automatic memory management
* is enabled. In that case the string will be released ASAP and removed
* from the pool of string to release at the end.
*
* If the string was created with a NULL context 'ctx', it is also possible to
* pass ctx as NULL when releasing the string (but passing a context will not
* create any issue). Strings created with a context should be freed also passing
* the context, so if you want to free a string out of context later, make sure
* to create it using a NULL context. */
* from the pool of string to release at the end. */
void RM_FreeString(RedisModuleCtx *ctx, RedisModuleString *str) {
decrRefCount(str);
if (ctx != NULL) autoMemoryFreed(ctx,REDISMODULE_AM_STRING,str);
autoMemoryFreed(ctx,REDISMODULE_AM_STRING,str);
}
/* Every call to this function, will make the string 'str' requiring
@@ -910,11 +876,9 @@ void RM_FreeString(RedisModuleCtx *ctx, RedisModuleString *str) {
* Note that when memory management is turned off, you don't need
* any call to RetainString() since creating a string will always result
* into a string that lives after the callback function returns, if
* no FreeString() call is performed.
*
* It is possible to call this function with a NULL context. */
* no FreeString() call is performed. */
void RM_RetainString(RedisModuleCtx *ctx, RedisModuleString *str) {
if (ctx == NULL || !autoMemoryFreed(ctx,REDISMODULE_AM_STRING,str)) {
if (!autoMemoryFreed(ctx,REDISMODULE_AM_STRING,str)) {
/* Increment the string reference counting only if we can't
* just remove the object from the list of objects that should
* be reclaimed. Why we do that, instead of just incrementing
@@ -2186,9 +2150,7 @@ int RM_ZsetRangePrev(RedisModuleKey *key) {
*
* The function is variadic and the user must specify pairs of field
* names and values, both as RedisModuleString pointers (unless the
* CFIELD option is set, see later). At the end of the field/value-ptr pairs,
* NULL must be specified as last argument to signal the end of the arguments
* in the variadic function.
* CFIELD option is set, see later).
*
* Example to set the hash argv[1] to the value argv[2]:
*
@@ -3025,7 +2987,7 @@ int RM_ModuleTypeSetValue(RedisModuleKey *key, moduleType *mt, void *value) {
}
/* Assuming RedisModule_KeyType() returned REDISMODULE_KEYTYPE_MODULE on
* the key, returns the module type pointer of the value stored at key.
* the key, returns the moduel type pointer of the value stored at key.
*
* If the key is NULL, is not associated with a module type, or is empty,
* then NULL is returned instead. */
@@ -3325,7 +3287,7 @@ void RM_DigestAddLongLong(RedisModuleDigest *md, long long ll) {
mixDigest(md->o,buf,len);
}
/* See the documentation for `RedisModule_DigestAddElement()`. */
/* See the doucmnetation for `RedisModule_DigestAddElement()`. */
void RM_DigestEndSequence(RedisModuleDigest *md) {
xorDigest(md->x,md->o,sizeof(md->o));
memset(md->o,0,sizeof(md->o));
@@ -3522,7 +3484,7 @@ void unblockClientFromModule(client *c) {
* reply_timeout: called when the timeout is reached in order to send an
* error to the client.
*
* free_privdata: called in order to free the private data that is passed
* free_privdata: called in order to free the privata data that is passed
* by RedisModule_UnblockClient() call.
*/
RedisModuleBlockedClient *RM_BlockClient(RedisModuleCtx *ctx, RedisModuleCmdFunc reply_callback, RedisModuleCmdFunc timeout_callback, void (*free_privdata)(RedisModuleCtx*,void*), long long timeout_ms) {
@@ -3719,7 +3681,7 @@ void moduleBlockedClientTimedOut(client *c) {
bc->timeout_callback(&ctx,(void**)c->argv,c->argc);
moduleFreeContext(&ctx);
/* For timeout events, we do not want to call the disconnect callback,
* because the blocked client will be automatically disconnected in
* because the blocekd client will be automatically disconnected in
* this case, and the user can still hook using the timeout callback. */
bc->disconnect_callback = NULL;
}
@@ -3736,7 +3698,7 @@ int RM_IsBlockedTimeoutRequest(RedisModuleCtx *ctx) {
return (ctx->flags & REDISMODULE_CTX_BLOCKED_TIMEOUT) != 0;
}
/* Get the private data set by RedisModule_UnblockClient() */
/* Get the privata data set by RedisModule_UnblockClient() */
void *RM_GetBlockedClientPrivateData(RedisModuleCtx *ctx) {
return ctx->blocked_privdata;
}
@@ -3911,7 +3873,7 @@ void moduleNotifyKeyspaceEvent(int type, const char *event, robj *key, int dbid)
if ((sub->event_mask & type) && sub->active == 0) {
RedisModuleCtx ctx = REDISMODULE_CTX_INIT;
ctx.module = sub->module;
ctx.client = moduleFreeContextReusedClient;
ctx.client = moduleKeyspaceSubscribersClient;
selectDb(ctx.client, dbid);
/* mark the handler as active to avoid reentrant loops.
@@ -3974,8 +3936,6 @@ void moduleCallClusterReceivers(const char *sender_id, uint64_t module_id, uint8
if (r->module_id == module_id) {
RedisModuleCtx ctx = REDISMODULE_CTX_INIT;
ctx.module = r->module;
ctx.client = moduleFreeContextReusedClient;
selectDb(ctx.client, 0);
r->callback(&ctx,sender_id,type,payload,len);
moduleFreeContext(&ctx);
return;
@@ -4124,7 +4084,7 @@ size_t RM_GetClusterSize(void) {
*
* * REDISMODULE_NODE_MYSELF This node
* * REDISMODULE_NODE_MASTER The node is a master
* * REDISMODULE_NODE_SLAVE The node is a replica
* * REDISMODULE_NODE_SLAVE The ndoe is a slave
* * REDISMODULE_NODE_PFAIL We see the node as failing
* * REDISMODULE_NODE_FAIL The cluster agrees the node is failing
* * REDISMODULE_NODE_NOFAILOVER The slave is configured to never failover
@@ -4166,32 +4126,6 @@ int RM_GetClusterNodeInfo(RedisModuleCtx *ctx, const char *id, char *ip, char *m
return REDISMODULE_OK;
}
/* Set Redis Cluster flags in order to change the normal behavior of
* Redis Cluster, especially with the goal of disabling certain functions.
* This is useful for modules that use the Cluster API in order to create
* a different distributed system, but still want to use the Redis Cluster
* message bus. Flags that can be set:
*
* CLUSTER_MODULE_FLAG_NO_FAILOVER
* CLUSTER_MODULE_FLAG_NO_REDIRECTION
*
* With the following effects:
*
* NO_FAILOVER: prevent Redis Cluster slaves to failover a failing master.
* Also disables the replica migration feature.
*
* NO_REDIRECTION: Every node will accept any key, without trying to perform
* partitioning according to the user Redis Cluster algorithm.
* Slots informations will still be propagated across the
* cluster, but without effects. */
void RM_SetClusterFlags(RedisModuleCtx *ctx, uint64_t flags) {
UNUSED(ctx);
if (flags & REDISMODULE_CLUSTER_FLAG_NO_FAILOVER)
server.cluster_module_flags |= CLUSTER_MODULE_FLAG_NO_FAILOVER;
if (flags & REDISMODULE_CLUSTER_FLAG_NO_REDIRECTION)
server.cluster_module_flags |= CLUSTER_MODULE_FLAG_NO_REDIRECTION;
}
/* --------------------------------------------------------------------------
* Modules Timers API
*
@@ -4221,7 +4155,6 @@ typedef struct RedisModuleTimer {
RedisModule *module; /* Module reference. */
RedisModuleTimerProc callback; /* The callback to invoke on expire. */
void *data; /* Private data for the callback. */
int dbid; /* Database number selected by the original client. */
} RedisModuleTimer;
/* This is the timer handler that is called by the main event loop. We schedule
@@ -4247,8 +4180,6 @@ int moduleTimerHandler(struct aeEventLoop *eventLoop, long long id, void *client
RedisModuleCtx ctx = REDISMODULE_CTX_INIT;
ctx.module = timer->module;
ctx.client = moduleFreeContextReusedClient;
selectDb(ctx.client, timer->dbid);
timer->callback(&ctx,timer->data);
moduleFreeContext(&ctx);
raxRemove(Timers,(unsigned char*)ri.key,ri.key_len,NULL);
@@ -4273,7 +4204,6 @@ RedisModuleTimerID RM_CreateTimer(RedisModuleCtx *ctx, mstime_t period, RedisMod
timer->module = ctx->module;
timer->callback = callback;
timer->data = data;
timer->dbid = ctx->client->db->id;
uint64_t expiretime = ustime()+period*1000;
uint64_t key;
@@ -4313,7 +4243,7 @@ RedisModuleTimerID RM_CreateTimer(RedisModuleCtx *ctx, mstime_t period, RedisMod
}
/* Stop a timer, returns REDISMODULE_OK if the timer was found, belonged to the
* calling module, and was stopped, otherwise REDISMODULE_ERR is returned.
* calling module, and was stoped, otherwise REDISMODULE_ERR is returned.
* If not NULL, the data pointer is set to the value of the data argument when
* the timer was created. */
int RM_StopTimer(RedisModuleCtx *ctx, RedisModuleTimerID id, void **data) {
@@ -4330,7 +4260,7 @@ int RM_StopTimer(RedisModuleCtx *ctx, RedisModuleTimerID id, void **data) {
* (in milliseconds), and the private data pointer associated with the timer.
* If the timer specified does not exist or belongs to a different module
* no information is returned and the function returns REDISMODULE_ERR, otherwise
* REDISMODULE_OK is returned. The arguments remaining or data can be NULL if
* REDISMODULE_OK is returned. The argumnets remaining or data can be NULL if
* the caller does not need certain information. */
int RM_GetTimerInfo(RedisModuleCtx *ctx, RedisModuleTimerID id, uint64_t *remaining, void **data) {
RedisModuleTimer *timer = raxFind(Timers,(unsigned char*)&id,sizeof(id));
@@ -4345,257 +4275,6 @@ int RM_GetTimerInfo(RedisModuleCtx *ctx, RedisModuleTimerID id, uint64_t *remain
return REDISMODULE_OK;
}
/* --------------------------------------------------------------------------
* Modules Dictionary API
*
* Implements a sorted dictionary (actually backed by a radix tree) with
* the usual get / set / del / num-items API, together with an iterator
* capable of going back and forth.
* -------------------------------------------------------------------------- */
/* Create a new dictionary. The 'ctx' pointer can be the current module context
* or NULL, depending on what you want. Please follow the following rules:
*
* 1. Use a NULL context if you plan to retain a reference to this dictionary
* that will survive the time of the module callback where you created it.
* 2. Use a NULL context if no context is available at the time you are creating
* the dictionary (of course...).
* 3. However use the current callback context as 'ctx' argument if the
* dictionary time to live is just limited to the callback scope. In this
* case, if enabled, you can enjoy the automatic memory management that will
* reclaim the dictionary memory, as well as the strings returned by the
* Next / Prev dictionary iterator calls.
*/
RedisModuleDict *RM_CreateDict(RedisModuleCtx *ctx) {
struct RedisModuleDict *d = zmalloc(sizeof(*d));
d->rax = raxNew();
if (ctx != NULL) autoMemoryAdd(ctx,REDISMODULE_AM_DICT,d);
return d;
}
/* Free a dictionary created with RM_CreateDict(). You need to pass the
* context pointer 'ctx' only if the dictionary was created using the
* context instead of passing NULL. */
void RM_FreeDict(RedisModuleCtx *ctx, RedisModuleDict *d) {
if (ctx != NULL) autoMemoryFreed(ctx,REDISMODULE_AM_DICT,d);
raxFree(d->rax);
zfree(d);
}
/* Return the size of the dictionary (number of keys). */
uint64_t RM_DictSize(RedisModuleDict *d) {
return raxSize(d->rax);
}
/* Store the specified key into the dictionary, setting its value to the
* pointer 'ptr'. If the key was added with success, since it did not
* already exist, REDISMODULE_OK is returned. Otherwise if the key already
* exists the function returns REDISMODULE_ERR. */
int RM_DictSetC(RedisModuleDict *d, void *key, size_t keylen, void *ptr) {
int retval = raxTryInsert(d->rax,key,keylen,ptr,NULL);
return (retval == 1) ? REDISMODULE_OK : REDISMODULE_ERR;
}
/* Like RedisModule_DictSetC() but will replace the key with the new
* value if the key already exists. */
int RM_DictReplaceC(RedisModuleDict *d, void *key, size_t keylen, void *ptr) {
int retval = raxInsert(d->rax,key,keylen,ptr,NULL);
return (retval == 1) ? REDISMODULE_OK : REDISMODULE_ERR;
}
/* Like RedisModule_DictSetC() but takes the key as a RedisModuleString. */
int RM_DictSet(RedisModuleDict *d, RedisModuleString *key, void *ptr) {
return RM_DictSetC(d,key->ptr,sdslen(key->ptr),ptr);
}
/* Like RedisModule_DictReplaceC() but takes the key as a RedisModuleString. */
int RM_DictReplace(RedisModuleDict *d, RedisModuleString *key, void *ptr) {
return RM_DictReplaceC(d,key->ptr,sdslen(key->ptr),ptr);
}
/* Return the value stored at the specified key. The function returns NULL
* both in the case the key does not exist, or if you actually stored
* NULL at key. So, optionally, if the 'nokey' pointer is not NULL, it will
* be set by reference to 1 if the key does not exist, or to 0 if the key
* exists. */
void *RM_DictGetC(RedisModuleDict *d, void *key, size_t keylen, int *nokey) {
void *res = raxFind(d->rax,key,keylen);
if (nokey) *nokey = (res == raxNotFound);
return (res == raxNotFound) ? NULL : res;
}
/* Like RedisModule_DictGetC() but takes the key as a RedisModuleString. */
void *RM_DictGet(RedisModuleDict *d, RedisModuleString *key, int *nokey) {
return RM_DictGetC(d,key->ptr,sdslen(key->ptr),nokey);
}
/* Remove the specified key from the dictionary, returning REDISMODULE_OK if
* the key was found and delted, or REDISMODULE_ERR if instead there was
* no such key in the dictionary. When the operation is successful, if
* 'oldval' is not NULL, then '*oldval' is set to the value stored at the
* key before it was deleted. Using this feature it is possible to get
* a pointer to the value (for instance in order to release it), without
* having to call RedisModule_DictGet() before deleting the key. */
int RM_DictDelC(RedisModuleDict *d, void *key, size_t keylen, void *oldval) {
int retval = raxRemove(d->rax,key,keylen,oldval);
return retval ? REDISMODULE_OK : REDISMODULE_ERR;
}
/* Like RedisModule_DictDelC() but gets the key as a RedisModuleString. */
int RM_DictDel(RedisModuleDict *d, RedisModuleString *key, void *oldval) {
return RM_DictDelC(d,key->ptr,sdslen(key->ptr),oldval);
}
/* Return an interator, setup in order to start iterating from the specified
* key by applying the operator 'op', which is just a string specifying the
* comparison operator to use in order to seek the first element. The
* operators avalable are:
*
* "^" -- Seek the first (lexicographically smaller) key.
* "$" -- Seek the last (lexicographically biffer) key.
* ">" -- Seek the first element greter than the specified key.
* ">=" -- Seek the first element greater or equal than the specified key.
* "<" -- Seek the first element smaller than the specified key.
* "<=" -- Seek the first element smaller or equal than the specified key.
* "==" -- Seek the first element matching exactly the specified key.
*
* Note that for "^" and "$" the passed key is not used, and the user may
* just pass NULL with a length of 0.
*
* If the element to start the iteration cannot be seeked based on the
* key and operator passed, RedisModule_DictNext() / Prev() will just return
* REDISMODULE_ERR at the first call, otherwise they'll produce elements.
*/
RedisModuleDictIter *RM_DictIteratorStartC(RedisModuleDict *d, const char *op, void *key, size_t keylen) {
RedisModuleDictIter *di = zmalloc(sizeof(*di));
di->dict = d;
raxStart(&di->ri,d->rax);
raxSeek(&di->ri,op,key,keylen);
return di;
}
/* Exactly like RedisModule_DictIteratorStartC, but the key is passed as a
* RedisModuleString. */
RedisModuleDictIter *RM_DictIteratorStart(RedisModuleDict *d, const char *op, RedisModuleString *key) {
return RM_DictIteratorStartC(d,op,key->ptr,sdslen(key->ptr));
}
/* Release the iterator created with RedisModule_DictIteratorStart(). This call
* is mandatory otherwise a memory leak is introduced in the module. */
void RM_DictIteratorStop(RedisModuleDictIter *di) {
raxStop(&di->ri);
zfree(di);
}
/* After its creation with RedisModule_DictIteratorStart(), it is possible to
* change the currently selected element of the iterator by using this
* API call. The result based on the operator and key is exactly like
* the function RedisModule_DictIteratorStart(), however in this case the
* return value is just REDISMODULE_OK in case the seeked element was found,
* or REDISMODULE_ERR in case it was not possible to seek the specified
* element. It is possible to reseek an iterator as many times as you want. */
int RM_DictIteratorReseekC(RedisModuleDictIter *di, const char *op, void *key, size_t keylen) {
return raxSeek(&di->ri,op,key,keylen);
}
/* Like RedisModule_DictIteratorReseekC() but takes the key as as a
* RedisModuleString. */
int RM_DictIteratorReseek(RedisModuleDictIter *di, const char *op, RedisModuleString *key) {
return RM_DictIteratorReseekC(di,op,key->ptr,sdslen(key->ptr));
}
/* Return the current item of the dictionary iterator 'di' and steps to the
* next element. If the iterator already yield the last element and there
* are no other elements to return, NULL is returned, otherwise a pointer
* to a string representing the key is provided, and the '*keylen' length
* is set by reference (if keylen is not NULL). The '*dataptr', if not NULL
* is set to the value of the pointer stored at the returned key as auxiliary
* data (as set by the RedisModule_DictSet API).
*
* Usage example:
*
* ... create the iterator here ...
* char *key;
* void *data;
* while((key = RedisModule_DictNextC(iter,&keylen,&data)) != NULL) {
* printf("%.*s %p\n", (int)keylen, key, data);
* }
*
* The returned pointer is of type void because sometimes it makes sense
* to cast it to a char* sometimes to an unsigned char* depending on the
* fact it contains or not binary data, so this API ends being more
* comfortable to use.
*
* The validity of the returned pointer is until the next call to the
* next/prev iterator step. Also the pointer is no longer valid once the
* iterator is released. */
void *RM_DictNextC(RedisModuleDictIter *di, size_t *keylen, void **dataptr) {
if (!raxNext(&di->ri)) return NULL;
if (keylen) *keylen = di->ri.key_len;
if (dataptr) *dataptr = di->ri.data;
return di->ri.key;
}
/* This function is exactly like RedisModule_DictNext() but after returning
* the currently selected element in the iterator, it selects the previous
* element (laxicographically smaller) instead of the next one. */
void *RM_DictPrevC(RedisModuleDictIter *di, size_t *keylen, void **dataptr) {
if (!raxPrev(&di->ri)) return NULL;
if (keylen) *keylen = di->ri.key_len;
if (dataptr) *dataptr = di->ri.data;
return di->ri.key;
}
/* Like RedisModuleNextC(), but instead of returning an internally allocated
* buffer and key length, it returns directly a module string object allocated
* in the specified context 'ctx' (that may be NULL exactly like for the main
* API RedisModule_CreateString).
*
* The returned string object should be deallocated after use, either manually
* or by using a context that has automatic memory management active. */
RedisModuleString *RM_DictNext(RedisModuleCtx *ctx, RedisModuleDictIter *di, void **dataptr) {
size_t keylen;
void *key = RM_DictNextC(di,&keylen,dataptr);
if (key == NULL) return NULL;
return RM_CreateString(ctx,key,keylen);
}
/* Like RedisModule_DictNext() but after returning the currently selected
* element in the iterator, it selects the previous element (laxicographically
* smaller) instead of the next one. */
RedisModuleString *RM_DictPrev(RedisModuleCtx *ctx, RedisModuleDictIter *di, void **dataptr) {
size_t keylen;
void *key = RM_DictPrevC(di,&keylen,dataptr);
if (key == NULL) return NULL;
return RM_CreateString(ctx,key,keylen);
}
/* Compare the element currently pointed by the iterator to the specified
* element given by key/keylen, according to the operator 'op' (the set of
* valid operators are the same valid for RedisModule_DictIteratorStart).
* If the comparision is successful the command returns REDISMODULE_OK
* otherwise REDISMODULE_ERR is returned.
*
* This is useful when we want to just emit a lexicographical range, so
* in the loop, as we iterate elements, we can also check if we are still
* on range.
*
* The function returne REDISMODULE_ERR if the iterator reached the
* end of elements condition as well. */
int RM_DictCompareC(RedisModuleDictIter *di, const char *op, void *key, size_t keylen) {
if (raxEOF(&di->ri)) return REDISMODULE_ERR;
int res = raxCompare(&di->ri,op,key,keylen);
return res ? REDISMODULE_OK : REDISMODULE_ERR;
}
/* Like RedisModule_DictCompareC but gets the key to compare with the current
* iterator key as a RedisModuleString. */
int RM_DictCompare(RedisModuleDictIter *di, const char *op, RedisModuleString *key) {
if (raxEOF(&di->ri)) return REDISMODULE_ERR;
int res = raxCompare(&di->ri,op,key->ptr,sdslen(key->ptr));
return res ? REDISMODULE_OK : REDISMODULE_ERR;
}
/* --------------------------------------------------------------------------
* Modules utility APIs
* -------------------------------------------------------------------------- */
@@ -4657,8 +4336,8 @@ void moduleInitModulesSystem(void) {
/* Set up the keyspace notification susbscriber list and static client */
moduleKeyspaceSubscribers = listCreate();
moduleFreeContextReusedClient = createClient(-1);
moduleFreeContextReusedClient->flags |= CLIENT_MODULE;
moduleKeyspaceSubscribersClient = createClient(-1);
moduleKeyspaceSubscribersClient->flags |= CLIENT_MODULE;
moduleRegisterCoreAPI();
if (pipe(server.module_blocked_pipe) == -1) {
@@ -4796,7 +4475,7 @@ int moduleUnload(sds name) {
moduleUnregisterCommands(module);
/* Remove any notification subscribers this module might have */
/* Remvoe any noification subscribers this module might have */
moduleUnsubscribeNotifications(module);
/* Unregister all the hooks. TODO: Yet no hooks support here. */
@@ -5021,27 +4700,4 @@ void moduleRegisterCoreAPI(void) {
REGISTER_API(BlockedClientDisconnected);
REGISTER_API(SetDisconnectCallback);
REGISTER_API(GetBlockedClientHandle);
REGISTER_API(SetClusterFlags);
REGISTER_API(CreateDict);
REGISTER_API(FreeDict);
REGISTER_API(DictSize);
REGISTER_API(DictSetC);
REGISTER_API(DictReplaceC);
REGISTER_API(DictSet);
REGISTER_API(DictReplace);
REGISTER_API(DictGetC);
REGISTER_API(DictGet);
REGISTER_API(DictDelC);
REGISTER_API(DictDel);
REGISTER_API(DictIteratorStartC);
REGISTER_API(DictIteratorStart);
REGISTER_API(DictIteratorStop);
REGISTER_API(DictIteratorReseekC);
REGISTER_API(DictIteratorReseek);
REGISTER_API(DictNextC);
REGISTER_API(DictPrevC);
REGISTER_API(DictNext);
REGISTER_API(DictPrev);
REGISTER_API(DictCompareC);
REGISTER_API(DictCompare);
}
+1 -6
View File
@@ -13,7 +13,7 @@ endif
.SUFFIXES: .c .so .xo .o
all: helloworld.so hellotype.so helloblock.so testmodule.so hellocluster.so hellotimer.so hellodict.so
all: helloworld.so hellotype.so helloblock.so testmodule.so hellocluster.so hellotimer.so
.c.xo:
$(CC) -I. $(CFLAGS) $(SHOBJ_CFLAGS) -fPIC -c $< -o $@
@@ -43,11 +43,6 @@ hellotimer.xo: ../redismodule.h
hellotimer.so: hellotimer.xo
$(LD) -o $@ $< $(SHOBJ_LDFLAGS) $(LIBS) -lc
hellodict.xo: ../redismodule.h
hellodict.so: hellodict.xo
$(LD) -o $@ $< $(SHOBJ_LDFLAGS) $(LIBS) -lc
testmodule.xo: ../redismodule.h
testmodule.so: testmodule.xo
+1 -1
View File
@@ -77,7 +77,7 @@ void *HelloBlock_ThreadMain(void *arg) {
/* An example blocked client disconnection callback.
*
* Note that in the case of the HELLO.BLOCK command, the blocked client is now
* owned by the thread calling sleep(). In this specific case, there is not
* owned by the thread calling sleep(). In this speciifc case, there is not
* much we can do, however normally we could instead implement a way to
* signal the thread that the client disconnected, and sleep the specified
* amount of seconds with a while loop calling sleep(1), so that once we
+1 -11
View File
@@ -69,7 +69,7 @@ int ListCommand_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int
RedisModule_ReplyWithLongLong(ctx,port);
}
RedisModule_FreeClusterNodesList(ids);
return REDISMODULE_OK;
return RedisModule_ReplyWithSimpleString(ctx, "OK");
}
/* Callback for message MSGTYPE_PING */
@@ -77,7 +77,6 @@ void PingReceiver(RedisModuleCtx *ctx, const char *sender_id, uint8_t type, cons
RedisModule_Log(ctx,"notice","PING (type %d) RECEIVED from %.*s: '%.*s'",
type,REDISMODULE_NODE_ID_LEN,sender_id,(int)len, payload);
RedisModule_SendClusterMessage(ctx,NULL,MSGTYPE_PONG,(unsigned char*)"Ohi!",4);
RedisModule_Call(ctx, "INCR", "c", "pings_received");
}
/* Callback for message MSGTYPE_PONG. */
@@ -103,15 +102,6 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
ListCommand_RedisCommand,"readonly",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
/* Disable Redis Cluster sharding and redirections. This way every node
* will be able to access every possible key, regardless of the hash slot.
* This way the PING message handler will be able to increment a specific
* variable. Normally you do that in order for the distributed system
* you create as a module to have total freedom in the keyspace
* manipulation. */
RedisModule_SetClusterFlags(ctx,REDISMODULE_CLUSTER_FLAG_NO_REDIRECTION);
/* Register our handlers for different message types. */
RedisModule_RegisterClusterMessageReceiver(ctx,MSGTYPE_PING,PingReceiver);
RedisModule_RegisterClusterMessageReceiver(ctx,MSGTYPE_PONG,PongReceiver);
return REDISMODULE_OK;
-132
View File
@@ -1,132 +0,0 @@
/* Hellodict -- An example of modules dictionary API
*
* This module implements a volatile key-value store on top of the
* dictionary exported by the Redis modules API.
*
* -----------------------------------------------------------------------------
*
* Copyright (c) 2018, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#define REDISMODULE_EXPERIMENTAL_API
#include "../redismodule.h"
#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
#include <string.h>
static RedisModuleDict *Keyspace;
/* HELLODICT.SET <key> <value>
*
* Set the specified key to the specified value. */
int cmd_SET(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc != 3) return RedisModule_WrongArity(ctx);
RedisModule_DictSet(Keyspace,argv[1],argv[2]);
/* We need to keep a reference to the value stored at the key, otherwise
* it would be freed when this callback returns. */
RedisModule_RetainString(NULL,argv[2]);
return RedisModule_ReplyWithSimpleString(ctx, "OK");
}
/* HELLODICT.GET <key>
*
* Return the value of the specified key, or a null reply if the key
* is not defined. */
int cmd_GET(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc != 2) return RedisModule_WrongArity(ctx);
RedisModuleString *val = RedisModule_DictGet(Keyspace,argv[1],NULL);
if (val == NULL) {
return RedisModule_ReplyWithNull(ctx);
} else {
return RedisModule_ReplyWithString(ctx, val);
}
}
/* HELLODICT.KEYRANGE <startkey> <endkey> <count>
*
* Return a list of matching keys, lexicographically between startkey
* and endkey. No more than 'count' items are emitted. */
int cmd_KEYRANGE(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc != 4) return RedisModule_WrongArity(ctx);
/* Parse the count argument. */
long long count;
if (RedisModule_StringToLongLong(argv[3],&count) != REDISMODULE_OK) {
return RedisModule_ReplyWithError(ctx,"ERR invalid count");
}
/* Seek the iterator. */
RedisModuleDictIter *iter = RedisModule_DictIteratorStart(
Keyspace, ">=", argv[1]);
/* Reply with the matching items. */
char *key;
size_t keylen;
long long replylen = 0; /* Keep track of the amitted array len. */
RedisModule_ReplyWithArray(ctx,REDISMODULE_POSTPONED_ARRAY_LEN);
while((key = RedisModule_DictNextC(iter,&keylen,NULL)) != NULL) {
if (replylen >= count) break;
if (RedisModule_DictCompare(iter,"<=",argv[2]) == REDISMODULE_ERR)
break;
RedisModule_ReplyWithStringBuffer(ctx,key,keylen);
replylen++;
}
RedisModule_ReplySetArrayLength(ctx,replylen);
/* Cleanup. */
RedisModule_DictIteratorStop(iter);
return REDISMODULE_OK;
}
/* This function must be present on each Redis module. It is used in order to
* register the commands into the Redis server. */
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
if (RedisModule_Init(ctx,"hellodict",1,REDISMODULE_APIVER_1)
== REDISMODULE_ERR) return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"hellodict.set",
cmd_SET,"write deny-oom",1,1,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"hellodict.get",
cmd_GET,"readonly",1,1,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"hellodict.keyrange",
cmd_KEYRANGE,"readonly",1,1,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
/* Create our global dictionray. Here we'll set our keys and values. */
Keyspace = RedisModule_CreateDict(NULL);
return REDISMODULE_OK;
}
+4 -1
View File
@@ -1,4 +1,4 @@
/* Timer API example -- Register and handle timer events
/* Helloworld cluster -- A ping/pong cluster API example.
*
* -----------------------------------------------------------------------------
*
@@ -37,6 +37,9 @@
#include <ctype.h>
#include <string.h>
#define MSGTYPE_PING 1
#define MSGTYPE_PONG 2
/* Timer callback. */
void timerHandler(RedisModuleCtx *ctx, void *data) {
REDISMODULE_NOT_USED(ctx);
+1 -1
View File
@@ -158,7 +158,7 @@ void execCommand(client *c) {
must_propagate = 1;
}
call(c,server.loading ? CMD_CALL_NONE : CMD_CALL_FULL);
call(c,CMD_CALL_FULL);
/* Commands may alter argc/argv, restore mstate. */
c->mstate.commands[j].argc = c->argc;
+50 -114
View File
@@ -160,32 +160,6 @@ client *createClient(int fd) {
return c;
}
/* This funciton puts the client in the queue of clients that should write
* their output buffers to the socket. Note that it does not *yet* install
* the write handler, to start clients are put in a queue of clients that need
* to write, so we try to do that before returning in the event loop (see the
* handleClientsWithPendingWrites() function).
* If we fail and there is more data to write, compared to what the socket
* buffers can hold, then we'll really install the handler. */
void clientInstallWriteHandler(client *c) {
/* Schedule the client to write the output buffers to the socket only
* if not already done and, for slaves, if the slave can actually receive
* writes at this stage. */
if (!(c->flags & CLIENT_PENDING_WRITE) &&
(c->replstate == REPL_STATE_NONE ||
(c->replstate == SLAVE_STATE_ONLINE && !c->repl_put_online_on_ack)))
{
/* Here instead of installing the write handler, we just flag the
* client and put it into a list of clients that have something
* to write to the socket. This way before re-entering the event
* loop, we can try to directly write to the client sockets avoiding
* a system call. We'll only really install the write handler if
* we'll not be able to write the whole reply at once. */
c->flags |= CLIENT_PENDING_WRITE;
listAddNodeHead(server.clients_pending_write,c);
}
}
/* This function is called every time we are going to transmit new data
* to the client. The behavior is the following:
*
@@ -223,9 +197,24 @@ int prepareClientToWrite(client *c) {
if (c->fd <= 0) return C_ERR; /* Fake client for AOF loading. */
/* Schedule the client to write the output buffers to the socket, unless
* it should already be setup to do so (it has already pending data). */
if (!clientHasPendingReplies(c)) clientInstallWriteHandler(c);
/* Schedule the client to write the output buffers to the socket only
* if not already done (there were no pending writes already and the client
* was yet not flagged), and, for slaves, if the slave can actually
* receive writes at this stage. */
if (!clientHasPendingReplies(c) &&
!(c->flags & CLIENT_PENDING_WRITE) &&
(c->replstate == REPL_STATE_NONE ||
(c->replstate == SLAVE_STATE_ONLINE && !c->repl_put_online_on_ack)))
{
/* Here instead of installing the write handler, we just flag the
* client and put it into a list of clients that have something
* to write to the socket. This way before re-entering the event
* loop, we can try to directly write to the client sockets avoiding
* a system call. We'll only really install the write handler if
* we'll not be able to write the whole reply at once. */
c->flags |= CLIENT_PENDING_WRITE;
listAddNodeHead(server.clients_pending_write,c);
}
/* Authorize the caller to queue in the output buffer of this client. */
return C_OK;
@@ -366,8 +355,8 @@ void addReplyErrorLength(client *c, const char *s, size_t len) {
* will produce an error. However it is useful to log such events since
* they are rare and may hint at errors in a script or a bug in Redis. */
if (c->flags & (CLIENT_MASTER|CLIENT_SLAVE)) {
char* to = c->flags & CLIENT_MASTER? "master": "replica";
char* from = c->flags & CLIENT_MASTER? "replica": "master";
char* to = c->flags & CLIENT_MASTER? "master": "slave";
char* from = c->flags & CLIENT_MASTER? "slave": "master";
char *cmdname = c->lastcmd ? c->lastcmd->name : "<unknown>";
serverLog(LL_WARNING,"== CRITICAL == This %s is sending an error "
"to its %s: '%s' after processing the command "
@@ -829,13 +818,6 @@ void unlinkClient(client *c) {
void freeClient(client *c) {
listNode *ln;
/* If a client is protected, yet we need to free it right now, make sure
* to at least use asynchronous freeing. */
if (c->flags & CLIENT_PROTECTED) {
freeClientAsync(c);
return;
}
/* If it is our master that's beging disconnected we should make sure
* to cache the state to try a partial resynchronization later.
*
@@ -854,7 +836,7 @@ void freeClient(client *c) {
/* Log link disconnection with slave */
if ((c->flags & CLIENT_SLAVE) && !(c->flags & CLIENT_MONITOR)) {
serverLog(LL_WARNING,"Connection with replica %s lost.",
serverLog(LL_WARNING,"Connection with slave %s lost.",
replicationGetSlaveName(c));
}
@@ -1072,10 +1054,6 @@ int handleClientsWithPendingWrites(void) {
c->flags &= ~CLIENT_PENDING_WRITE;
listDelNode(server.clients_pending_write,ln);
/* If a client is protected, don't do anything,
* that may trigger write error or recreate handler. */
if (c->flags & CLIENT_PROTECTED) continue;
/* Try to write buffers to the client socket. */
if (writeToClient(c->fd,c,0) == C_ERR) continue;
@@ -1127,34 +1105,6 @@ void resetClient(client *c) {
}
}
/* This funciton is used when we want to re-enter the event loop but there
* is the risk that the client we are dealing with will be freed in some
* way. This happens for instance in:
*
* * DEBUG RELOAD and similar.
* * When a Lua script is in -BUSY state.
*
* So the function will protect the client by doing two things:
*
* 1) It removes the file events. This way it is not possible that an
* error is signaled on the socket, freeing the client.
* 2) Moreover it makes sure that if the client is freed in a different code
* path, it is not really released, but only marked for later release. */
void protectClient(client *c) {
c->flags |= CLIENT_PROTECTED;
aeDeleteFileEvent(server.el,c->fd,AE_READABLE);
aeDeleteFileEvent(server.el,c->fd,AE_WRITABLE);
}
/* This will undo the client protection done by protectClient() */
void unprotectClient(client *c) {
if (c->flags & CLIENT_PROTECTED) {
c->flags &= ~CLIENT_PROTECTED;
aeCreateFileEvent(server.el,c->fd,AE_READABLE,readQueryFromClient,c);
if (clientHasPendingReplies(c)) clientInstallWriteHandler(c);
}
}
/* Like processMultibulkBuffer(), but for the inline protocol instead of RESP,
* this function consumes the client query buffer and creates a command ready
* to be executed inside the client structure. Returns C_OK if the command
@@ -1344,22 +1294,19 @@ int processMultibulkBuffer(client *c) {
c->qb_pos = newline-c->querybuf+2;
if (ll >= PROTO_MBULK_BIG_ARG) {
size_t qblen;
/* If we are going to read a large object from network
* try to make it likely that it will start at c->querybuf
* boundary so that we can optimize object creation
* avoiding a large copy of data.
*
* But only when the data we have not parsed is less than
* or equal to ll+2. If the data length is greater than
* ll+2, trimming querybuf is just a waste of time, because
* at this time the querybuf contains not only our bulk. */
if (sdslen(c->querybuf)-c->qb_pos <= (size_t)ll+2) {
sdsrange(c->querybuf,c->qb_pos,-1);
c->qb_pos = 0;
/* Hint the sds library about the amount of bytes this string is
* going to contain. */
c->querybuf = sdsMakeRoomFor(c->querybuf,ll+2);
}
* avoiding a large copy of data. */
sdsrange(c->querybuf,c->qb_pos,-1);
c->qb_pos = 0;
qblen = sdslen(c->querybuf);
/* Hint the sds library about the amount of bytes this string is
* going to contain. */
if (qblen < (size_t)ll+2)
c->querybuf = sdsMakeRoomFor(c->querybuf,ll+2-qblen);
}
c->bulklen = ll;
}
@@ -1478,25 +1425,6 @@ void processInputBuffer(client *c) {
server.current_client = NULL;
}
/* This is a wrapper for processInputBuffer that also cares about handling
* the replication forwarding to the sub-slaves, in case the client 'c'
* is flagged as master. Usually you want to call this instead of the
* raw processInputBuffer(). */
void processInputBufferAndReplicate(client *c) {
if (!(c->flags & CLIENT_MASTER)) {
processInputBuffer(c);
} else {
size_t prev_offset = c->reploff;
processInputBuffer(c);
size_t applied = c->reploff - prev_offset;
if (applied) {
replicationFeedSlavesFromMasterStream(server.slaves,
c->pending_querybuf, applied);
sdsrange(c->pending_querybuf,applied,-1);
}
}
}
void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
client *c = (client*) privdata;
int nread, readlen;
@@ -1516,9 +1444,7 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
{
ssize_t remaining = (size_t)(c->bulklen+2)-sdslen(c->querybuf);
/* Note that the 'remaining' variable may be zero in some edge case,
* for example once we resume a blocked client after CLIENT PAUSE. */
if (remaining > 0 && remaining < readlen) readlen = remaining;
if (remaining < readlen) readlen = remaining;
}
qblen = sdslen(c->querybuf);
@@ -1566,7 +1492,18 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
* was actually applied to the master state: this quantity, and its
* corresponding part of the replication stream, will be propagated to
* the sub-slaves and to the replication backlog. */
processInputBufferAndReplicate(c);
if (!(c->flags & CLIENT_MASTER)) {
processInputBuffer(c);
} else {
size_t prev_offset = c->reploff;
processInputBuffer(c);
size_t applied = c->reploff - prev_offset;
if (applied) {
replicationFeedSlavesFromMasterStream(server.slaves,
c->pending_querybuf, applied);
sdsrange(c->pending_querybuf,applied,-1);
}
}
}
void getClientsMaxBuffers(unsigned long *longest_output_list,
@@ -1704,10 +1641,10 @@ void clientCommand(client *c) {
"kill <ip:port> -- Kill connection made from <ip:port>.",
"kill <option> <value> [option value ...] -- Kill connections. Options are:",
" addr <ip:port> -- Kill connection made from <ip:port>",
" type (normal|master|replica|pubsub) -- Kill connections by type.",
" type (normal|master|slave|pubsub) -- Kill connections by type.",
" skipme (yes|no) -- Skip killing current connection (default: yes).",
"list [options ...] -- Return information about client connections. Options:",
" type (normal|master|replica|pubsub) -- Return clients of specified type.",
" type (normal|master|slave|pubsub) -- Return clients of specified type.",
"pause <timeout> -- Suspend all Redis clients for <timout> milliseconds.",
"reply (on|off|skip) -- Control the replies sent to the current connection.",
"setname <name> -- Assign the name <name> to the current connection.",
@@ -2041,7 +1978,6 @@ int getClientType(client *c) {
int getClientTypeByName(char *name) {
if (!strcasecmp(name,"normal")) return CLIENT_TYPE_NORMAL;
else if (!strcasecmp(name,"slave")) return CLIENT_TYPE_SLAVE;
else if (!strcasecmp(name,"replica")) return CLIENT_TYPE_SLAVE;
else if (!strcasecmp(name,"pubsub")) return CLIENT_TYPE_PUBSUB;
else if (!strcasecmp(name,"master")) return CLIENT_TYPE_MASTER;
else return -1;
@@ -2109,7 +2045,6 @@ int checkClientOutputBufferLimits(client *c) {
* called from contexts where the client can't be freed safely, i.e. from the
* lower level functions pushing data inside the client output buffers. */
void asyncCloseClientOnOutputBufferLimitReached(client *c) {
if (c->fd == -1) return; /* It is unsafe to free fake clients. */
serverAssert(c->reply_bytes < SIZE_MAX-(1024*64));
if (c->reply_bytes == 0 || c->flags & CLIENT_CLOSE_ASAP) return;
if (checkClientOutputBufferLimits(c)) {
@@ -2191,10 +2126,11 @@ int clientsArePaused(void) {
while ((ln = listNext(&li)) != NULL) {
c = listNodeValue(ln);
/* Don't touch slaves and blocked clients.
* The latter pending requests will be processed when unblocked. */
/* Don't touch slaves and blocked clients. The latter pending
* requests be processed when unblocked. */
if (c->flags & (CLIENT_SLAVE|CLIENT_BLOCKED)) continue;
queueClientForReprocessing(c);
c->flags |= CLIENT_UNBLOCKED;
listAddNodeTail(server.unblocked_clients,c);
}
}
return server.clients_paused;
+18 -16
View File
@@ -185,7 +185,7 @@ robj *createStringObjectFromLongDouble(long double value, int humanfriendly) {
/* Duplicate a string object, with the guarantee that the returned object
* has the same encoding as the original one.
*
* This function also guarantees that duplicating a small integer object
* This function also guarantees that duplicating a small integere object
* (or a string object that contains a representation of a small integer)
* will always result in a fresh object that is unshared (refcount == 1).
*
@@ -1011,13 +1011,13 @@ struct redisMemOverhead *getMemoryOverheadData(void) {
mem = 0;
if (server.aof_state != AOF_OFF) {
mem += sdsalloc(server.aof_buf);
mem += sdslen(server.aof_buf);
mem += aofRewriteBufferSize();
}
mh->aof_buffer = mem;
mem_total+=mem;
mem = server.lua_scripts_mem;
mem = 0;
mem += dictSize(server.lua_scripts) * sizeof(dictEntry) +
dictSlots(server.lua_scripts) * sizeof(dictEntry*);
mem += dictSize(server.repl_scriptcache_dict) * sizeof(dictEntry) +
@@ -1175,7 +1175,7 @@ sds getMemoryDoctorReport(void) {
s = sdscatprintf(s," * High process RSS overhead: This instance has non-allocator RSS memory overhead is greater than 1.1 (this means that the Resident Set Size of the Redis process is much larger than the RSS the allocator holds). This problem may be due to Lua scripts or Modules.\n\n");
}
if (big_slave_buf) {
s = sdscat(s," * Big replica buffers: The replica output buffers in this instance are greater than 10MB for each replica (on average). This likely means that there is some replica instance that is struggling receiving data, either because it is too slow or because of networking issues. As a result, data piles on the master output buffers. Please try to identify what replica is not receiving data correctly and why. You can use the INFO output in order to check the replicas delays and the CLIENT LIST command to check the output buffers of each replica.\n\n");
s = sdscat(s," * Big slave buffers: The slave output buffers in this instance are greater than 10MB for each slave (on average). This likely means that there is some slave instance that is struggling receiving data, either because it is too slow or because of networking issues. As a result, data piles on the master output buffers. Please try to identify what slave is not receiving data correctly and why. You can use the INFO output in order to check the slaves delays and the CLIENT LIST command to check the output buffers of each slave.\n\n");
}
if (big_client_buf) {
s = sdscat(s," * Big client buffers: The clients output buffers in this instance are greater than 200K per client (on average). This may result from different causes, like Pub/Sub clients subscribed to channels bot not receiving data fast enough, so that data piles on the Redis instance output buffer, or clients sending commands with large replies or very large sequences of commands in the same pipeline. Please use the CLIENT LIST command in order to investigate the issue if it causes problems in your instance, or to understand better why certain clients are using a big amount of memory.\n\n");
@@ -1287,17 +1287,7 @@ NULL
void memoryCommand(client *c) {
robj *o;
if (!strcasecmp(c->argv[1]->ptr,"help") && c->argc == 2) {
const char *help[] = {
"DOCTOR - Return memory problems reports.",
"MALLOC-STATS -- Return internal statistics report from the memory allocator.",
"PURGE -- Attempt to purge dirty pages for reclamation by the allocator.",
"STATS -- Return information about the memory usage of the server.",
"USAGE <key> [SAMPLES <count>] -- Return memory in bytes used by <key> and its value. Nested values are sampled up to <count> times (default: 5).",
NULL
};
addReplyHelp(c, help);
} else if (!strcasecmp(c->argv[1]->ptr,"usage") && c->argc >= 3) {
if (!strcasecmp(c->argv[1]->ptr,"usage") && c->argc >= 3) {
long long samples = OBJ_COMPUTE_SIZE_DEF_SAMPLES;
for (int j = 3; j < c->argc; j++) {
if (!strcasecmp(c->argv[j]->ptr,"samples") &&
@@ -1444,7 +1434,19 @@ NULL
addReply(c, shared.ok);
/* Nothing to do for other allocators. */
#endif
} else if (!strcasecmp(c->argv[1]->ptr,"help") && c->argc == 2) {
addReplyMultiBulkLen(c,5);
addReplyBulkCString(c,
"MEMORY DOCTOR - Outputs memory problems report");
addReplyBulkCString(c,
"MEMORY USAGE <key> [SAMPLES <count>] - Estimate memory usage of key");
addReplyBulkCString(c,
"MEMORY STATS - Show memory usage details");
addReplyBulkCString(c,
"MEMORY PURGE - Ask the allocator to release memory");
addReplyBulkCString(c,
"MEMORY MALLOC-STATS - Show allocator internal stats");
} else {
addReplyErrorFormat(c, "Unknown subcommand or wrong number of arguments for '%s'. Try MEMORY HELP", (char*)c->argv[1]->ptr);
addReplyError(c,"Syntax error. Try MEMORY HELP");
}
}
+1 -1
View File
@@ -40,7 +40,7 @@
* container: 2 bits, NONE=1, ZIPLIST=2.
* recompress: 1 bit, bool, true if node is temporarry decompressed for usage.
* attempted_compress: 1 bit, boolean, used for verifying during testing.
* extra: 10 bits, free for future use; pads out the remainder of 32 bits */
* extra: 12 bits, free for future use; pads out the remainder of 32 bits */
typedef struct quicklistNode {
struct quicklistNode *prev;
struct quicklistNode *next;
+65 -198
View File
@@ -1,6 +1,6 @@
/* Rax -- A radix tree implementation.
*
* Copyright (c) 2017-2018, Salvatore Sanfilippo <antirez at gmail dot com>
* Copyright (c) 2017, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -51,18 +51,14 @@ void *raxNotFound = (void*)"rax-not-found-pointer";
void raxDebugShowNode(const char *msg, raxNode *n);
/* Turn debugging messages on/off by compiling with RAX_DEBUG_MSG macro on.
* When RAX_DEBUG_MSG is defined by default Rax operations will emit a lot
* of debugging info to the standard output, however you can still turn
* debugging on/off in order to enable it only when you suspect there is an
* operation causing a bug using the function raxSetDebugMsg(). */
#ifdef RAX_DEBUG_MSG
/* Turn debugging messages on/off. */
#if 0
#define debugf(...) \
if (raxDebugMsg) { \
do { \
printf("%s:%s:%d:\t", __FILE__, __FUNCTION__, __LINE__); \
printf(__VA_ARGS__); \
fflush(stdout); \
}
} while (0);
#define debugnode(msg,n) raxDebugShowNode(msg,n)
#else
@@ -70,16 +66,6 @@ void raxDebugShowNode(const char *msg, raxNode *n);
#define debugnode(msg,n)
#endif
/* By default log debug info if RAX_DEBUG_MSG is defined. */
static int raxDebugMsg = 1;
/* When debug messages are enabled, turn them on/off dynamically. By
* default they are enabled. Set the state to 0 to disable, and 1 to
* re-enable. */
void raxSetDebugMsg(int onoff) {
raxDebugMsg = onoff;
}
/* ------------------------- raxStack functions --------------------------
* The raxStack is a simple stack of pointers that is capable of switching
* from using a stack-allocated array to dynamic heap once a given number of
@@ -148,43 +134,12 @@ static inline void raxStackFree(raxStack *ts) {
* Radix tree implementation
* --------------------------------------------------------------------------*/
/* Return the padding needed in the characters section of a node having size
* 'nodesize'. The padding is needed to store the child pointers to aligned
* addresses. Note that we add 4 to the node size because the node has a four
* bytes header. */
#define raxPadding(nodesize) ((sizeof(void*)-((nodesize+4) % sizeof(void*))) & (sizeof(void*)-1))
/* Return the pointer to the last child pointer in a node. For the compressed
* nodes this is the only child pointer. */
#define raxNodeLastChildPtr(n) ((raxNode**) ( \
((char*)(n)) + \
raxNodeCurrentLength(n) - \
sizeof(raxNode*) - \
(((n)->iskey && !(n)->isnull) ? sizeof(void*) : 0) \
))
/* Return the pointer to the first child pointer. */
#define raxNodeFirstChildPtr(n) ((raxNode**) ( \
(n)->data + \
(n)->size + \
raxPadding((n)->size)))
/* Return the current total size of the node. Note that the second line
* computes the padding after the string of characters, needed in order to
* save pointers to aligned addresses. */
#define raxNodeCurrentLength(n) ( \
sizeof(raxNode)+(n)->size+ \
raxPadding((n)->size)+ \
((n)->iscompr ? sizeof(raxNode*) : sizeof(raxNode*)*(n)->size)+ \
(((n)->iskey && !(n)->isnull)*sizeof(void*)) \
)
/* Allocate a new non compressed node with the specified number of children.
* If datafiled is true, the allocation is made large enough to hold the
* associated data pointer.
* Returns the new node pointer. On out of memory NULL is returned. */
raxNode *raxNewNode(size_t children, int datafield) {
size_t nodesize = sizeof(raxNode)+children+raxPadding(children)+
size_t nodesize = sizeof(raxNode)+children+
sizeof(raxNode*)*children;
if (datafield) nodesize += sizeof(void*);
raxNode *node = rax_malloc(nodesize);
@@ -212,6 +167,13 @@ rax *raxNew(void) {
}
}
/* Return the current total size of the node. */
#define raxNodeCurrentLength(n) ( \
sizeof(raxNode)+(n)->size+ \
((n)->iscompr ? sizeof(raxNode*) : sizeof(raxNode*)*(n)->size)+ \
(((n)->iskey && !(n)->isnull)*sizeof(void*)) \
)
/* realloc the node to make room for auxiliary data in order
* to store an item in that node. On out of memory NULL is returned. */
raxNode *raxReallocForData(raxNode *n, void *data) {
@@ -254,17 +216,18 @@ void *raxGetData(raxNode *n) {
raxNode *raxAddChild(raxNode *n, unsigned char c, raxNode **childptr, raxNode ***parentlink) {
assert(n->iscompr == 0);
size_t curlen = raxNodeCurrentLength(n);
n->size++;
size_t newlen = raxNodeCurrentLength(n);
n->size--; /* For now restore the orignal size. We'll update it only on
success at the end. */
size_t curlen = sizeof(raxNode)+
n->size+
sizeof(raxNode*)*n->size;
size_t newlen;
/* Alloc the new child we will link to 'n'. */
raxNode *child = raxNewNode(0,0);
if (child == NULL) return NULL;
/* Make space in the original node. */
if (n->iskey) curlen += sizeof(void*);
newlen = curlen+sizeof(raxNode*)+1; /* Add 1 char and 1 pointer. */
raxNode *newn = rax_realloc(n,newlen);
if (newn == NULL) {
rax_free(child);
@@ -272,34 +235,14 @@ raxNode *raxAddChild(raxNode *n, unsigned char c, raxNode **childptr, raxNode **
}
n = newn;
/* After the reallocation, we have up to 8/16 (depending on the system
* pointer size, and the required node padding) bytes at the end, that is,
* the additional char in the 'data' section, plus one pointer to the new
* child, plus the padding needed in order to store addresses into aligned
* locations.
/* After the reallocation, we have 5/9 (depending on the system
* pointer size) bytes at the end, that is, the additional char
* in the 'data' section, plus one pointer to the new child:
*
* So if we start with the following node, having "abde" edges.
*
* Note:
* - We assume 4 bytes pointer for simplicity.
* - Each space below corresponds to one byte
*
* [HDR*][abde][Aptr][Bptr][Dptr][Eptr]|AUXP|
*
* After the reallocation we need: 1 byte for the new edge character
* plus 4 bytes for a new child pointer (assuming 32 bit machine).
* However after adding 1 byte to the edge char, the header + the edge
* characters are no longer aligned, so we also need 3 bytes of padding.
* In total the reallocation will add 1+4+3 bytes = 8 bytes:
*
* (Blank bytes are represented by ".")
*
* [HDR*][abde][Aptr][Bptr][Dptr][Eptr]|AUXP|[....][....]
* [numc][abx][ap][bp][xp]|auxp|.....
*
* Let's find where to insert the new child in order to make sure
* it is inserted in-place lexicographically. Assuming we are adding
* a child "c" in our case pos will be = 2 after the end of the following
* loop. */
* it is inserted in-place lexicographically. */
int pos;
for (pos = 0; pos < n->size; pos++) {
if (n->data[pos] > c) break;
@@ -309,81 +252,55 @@ raxNode *raxAddChild(raxNode *n, unsigned char c, raxNode **childptr, raxNode **
* so that we can mess with the other data without overwriting it.
* We will obtain something like that:
*
* [HDR*][abde][Aptr][Bptr][Dptr][Eptr][....][....]|AUXP|
*/
unsigned char *src, *dst;
* [numc][abx][ap][bp][xp].....|auxp| */
unsigned char *src;
if (n->iskey && !n->isnull) {
src = ((unsigned char*)n+curlen-sizeof(void*));
dst = ((unsigned char*)n+newlen-sizeof(void*));
memmove(dst,src,sizeof(void*));
src = n->data+n->size+sizeof(raxNode*)*n->size;
memmove(src+1+sizeof(raxNode*),src,sizeof(void*));
}
/* Compute the "shift", that is, how many bytes we need to move the
* pointers section forward because of the addition of the new child
* byte in the string section. Note that if we had no padding, that
* would be always "1", since we are adding a single byte in the string
* section of the node (where now there is "abde" basically).
*
* However we have padding, so it could be zero, or up to 8.
*
* Another way to think at the shift is, how many bytes we need to
* move child pointers forward *other than* the obvious sizeof(void*)
* needed for the additional pointer itself. */
size_t shift = newlen - curlen - sizeof(void*);
/* We said we are adding a node with edge 'c'. The insertion
* point is between 'b' and 'd', so the 'pos' variable value is
* the index of the first child pointer that we need to move forward
* to make space for our new pointer.
*
/* Now imagine we are adding a node with edge 'c'. The insertion
* point is between 'b' and 'x', so the 'pos' variable value is
* To start, move all the child pointers after the insertion point
* of shift+sizeof(pointer) bytes on the right, to obtain:
* of 1+sizeof(pointer) bytes on the right, to obtain:
*
* [HDR*][abde][Aptr][Bptr][....][....][Dptr][Eptr]|AUXP|
*/
src = n->data+n->size+
raxPadding(n->size)+
sizeof(raxNode*)*pos;
memmove(src+shift+sizeof(raxNode*),src,sizeof(raxNode*)*(n->size-pos));
/* Move the pointers to the left of the insertion position as well. Often
* we don't need to do anything if there was already some padding to use. In
* that case the final destination of the pointers will be the same, however
* in our example there was no pre-existing padding, so we added one byte
* plus thre bytes of padding. After the next memmove() things will look
* like thata:
*
* [HDR*][abde][....][Aptr][Bptr][....][Dptr][Eptr]|AUXP|
*/
if (shift) {
src = (unsigned char*) raxNodeFirstChildPtr(n);
memmove(src+shift,src,sizeof(raxNode*)*pos);
}
* [numc][abx][ap][bp].....[xp]|auxp| */
src = n->data+n->size+sizeof(raxNode*)*pos;
memmove(src+1+sizeof(raxNode*),src,sizeof(raxNode*)*(n->size-pos));
/* Now make the space for the additional char in the data section,
* but also move the pointers before the insertion point to the right
* by shift bytes, in order to obtain the following:
* but also move the pointers before the insertion point in the right
* by 1 byte, in order to obtain the following:
*
* [HDR*][ab.d][e...][Aptr][Bptr][....][Dptr][Eptr]|AUXP|
*/
* [numc][ab.x][ap][bp]....[xp]|auxp| */
src = n->data+pos;
memmove(src+1,src,n->size-pos);
memmove(src+1,src,n->size-pos+sizeof(raxNode*)*pos);
/* We can now set the character and its child node pointer to get:
*
* [HDR*][abcd][e...][Aptr][Bptr][....][Dptr][Eptr]|AUXP|
* [HDR*][abcd][e...][Aptr][Bptr][Cptr][Dptr][Eptr]|AUXP|
*/
* [numc][abcx][ap][bp][cp]....|auxp|
* [numc][abcx][ap][bp][cp][xp]|auxp| */
n->data[pos] = c;
n->size++;
src = (unsigned char*) raxNodeFirstChildPtr(n);
raxNode **childfield = (raxNode**)(src+sizeof(raxNode*)*pos);
raxNode **childfield = (raxNode**)(n->data+n->size+sizeof(raxNode*)*pos);
memcpy(childfield,&child,sizeof(child));
*childptr = child;
*parentlink = childfield;
return n;
}
/* Return the pointer to the last child pointer in a node. For the compressed
* nodes this is the only child pointer. */
#define raxNodeLastChildPtr(n) ((raxNode**) ( \
((char*)(n)) + \
raxNodeCurrentLength(n) - \
sizeof(raxNode*) - \
(((n)->iskey && !(n)->isnull) ? sizeof(void*) : 0) \
))
/* Return the pointer to the first child pointer. */
#define raxNodeFirstChildPtr(n) ((raxNode**)((n)->data+(n)->size))
/* Turn the node 'n', that must be a node without any children, into a
* compressed node representing a set of nodes linked one after the other
* and having exactly one child each. The node can be a key or not: this
@@ -404,7 +321,7 @@ raxNode *raxCompressNode(raxNode *n, unsigned char *s, size_t len, raxNode **chi
if (*child == NULL) return NULL;
/* Make space in the parent node. */
newsize = sizeof(raxNode)+len+raxPadding(len)+sizeof(raxNode*);
newsize = sizeof(raxNode)+len+sizeof(raxNode*);
if (n->iskey) {
data = raxGetData(n); /* To restore it later. */
if (!n->isnull) newsize += sizeof(void*);
@@ -702,14 +619,13 @@ int raxGenericInsert(rax *rax, unsigned char *s, size_t len, void *data, void **
raxNode *postfix = NULL;
if (trimmedlen) {
nodesize = sizeof(raxNode)+trimmedlen+raxPadding(trimmedlen)+
sizeof(raxNode*);
nodesize = sizeof(raxNode)+trimmedlen+sizeof(raxNode*);
if (h->iskey && !h->isnull) nodesize += sizeof(void*);
trimmed = rax_malloc(nodesize);
}
if (postfixlen) {
nodesize = sizeof(raxNode)+postfixlen+raxPadding(postfixlen)+
nodesize = sizeof(raxNode)+postfixlen+
sizeof(raxNode*);
postfix = rax_malloc(nodesize);
}
@@ -785,12 +701,11 @@ int raxGenericInsert(rax *rax, unsigned char *s, size_t len, void *data, void **
/* Allocate postfix & trimmed nodes ASAP to fail for OOM gracefully. */
size_t postfixlen = h->size - j;
size_t nodesize = sizeof(raxNode)+postfixlen+raxPadding(postfixlen)+
sizeof(raxNode*);
size_t nodesize = sizeof(raxNode)+postfixlen+sizeof(raxNode*);
if (data != NULL) nodesize += sizeof(void*);
raxNode *postfix = rax_malloc(nodesize);
nodesize = sizeof(raxNode)+j+raxPadding(j)+sizeof(raxNode*);
nodesize = sizeof(raxNode)+j+sizeof(raxNode*);
if (h->iskey && !h->isnull) nodesize += sizeof(void*);
raxNode *trimmed = rax_malloc(nodesize);
@@ -960,7 +875,7 @@ raxNode *raxRemoveChild(raxNode *parent, raxNode *child) {
return parent;
}
/* Otherwise we need to scan for the child pointer and memmove()
/* Otherwise we need to scan for the children pointer and memmove()
* accordingly.
*
* 1. To start we seek the first element in both the children
@@ -985,21 +900,13 @@ raxNode *raxRemoveChild(raxNode *parent, raxNode *child) {
debugf("raxRemoveChild tail len: %d\n", taillen);
memmove(e,e+1,taillen);
/* Compute the shift, that is the amount of bytes we should move our
* child pointers to the left, since the removal of one edge character
* and the corresponding padding change, may change the layout.
* We just check if in the old version of the node there was at the
* end just a single byte and all padding: in that case removing one char
* will remove a whole sizeof(void*) word. */
size_t shift = ((parent->size+4) % sizeof(void*)) == 1 ? sizeof(void*) : 0;
/* Since we have one data byte less, also child pointers start one byte
* before now. */
memmove(((char*)cp)-1,cp,(parent->size-taillen-1)*sizeof(raxNode**));
/* Move the children pointers before the deletion point. */
if (shift)
memmove(((char*)cp)-shift,cp,(parent->size-taillen-1)*sizeof(raxNode**));
/* Move the remaining "tail" pointers at the right position as well. */
/* Move the remaining "tail" pointer at the right position as well. */
size_t valuelen = (parent->iskey && !parent->isnull) ? sizeof(void*) : 0;
memmove(((char*)c)-shift,c+1,taillen*sizeof(raxNode**)+valuelen);
memmove(((char*)c)-1,c+1,taillen*sizeof(raxNode**)+valuelen);
/* 4. Update size. */
parent->size--;
@@ -1165,7 +1072,7 @@ int raxRemove(rax *rax, unsigned char *s, size_t len, void **old) {
if (nodes > 1) {
/* If we can compress, create the new node and populate it. */
size_t nodesize =
sizeof(raxNode)+comprsize+raxPadding(comprsize)+sizeof(raxNode*);
sizeof(raxNode)+comprsize+sizeof(raxNode*);
raxNode *new = rax_malloc(nodesize);
/* An out of memory here just means we cannot optimize this
* node, but the tree is left in a consistent state. */
@@ -1886,7 +1793,6 @@ void raxShow(rax *rax) {
/* Used by debugnode() macro to show info about a given node. */
void raxDebugShowNode(const char *msg, raxNode *n) {
if (raxDebugMsg == 0) return;
printf("%s: %p [%.*s] key:%d size:%d children:",
msg, (void*)n, (int)n->size, (char*)n->data, n->iskey, n->size);
int numcld = n->iscompr ? 1 : n->size;
@@ -1901,43 +1807,4 @@ void raxDebugShowNode(const char *msg, raxNode *n) {
fflush(stdout);
}
/* Touch all the nodes of a tree returning a check sum. This is useful
* in order to make Valgrind detect if there is something wrong while
* reading the data structure.
*
* This function was used in order to identify Rax bugs after a big refactoring
* using this technique:
*
* 1. The rax-test is executed using Valgrind, adding a printf() so that for
* the fuzz tester we see what iteration in the loop we are in.
* 2. After every modification of the radix tree made by the fuzz tester
* in rax-test.c, we add a call to raxTouch().
* 3. Now as soon as an operation will corrupt the tree, raxTouch() will
* detect it (via Valgrind) immediately. We can add more calls to narrow
* the state.
* 4. At this point a good idea is to enable Rax debugging messages immediately
* before the moment the tree is corrupted, to see what happens.
*/
unsigned long raxTouch(raxNode *n) {
debugf("Touching %p\n", (void*)n);
unsigned long sum = 0;
if (n->iskey) {
sum += (unsigned long)raxGetData(n);
}
int numchildren = n->iscompr ? 1 : n->size;
raxNode **cp = raxNodeFirstChildPtr(n);
int count = 0;
for (int i = 0; i < numchildren; i++) {
if (numchildren > 1) {
sum += (long)n->data[i];
}
raxNode *child;
memcpy(&child,cp,sizeof(child));
if (child == (void*)0x65d1760) count++;
if (count > 1) exit(1);
sum += raxTouch(child);
cp++;
}
return sum;
}
+3 -35
View File
@@ -1,33 +1,3 @@
/* Rax -- A radix tree implementation.
*
* Copyright (c) 2017-2018, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef RAX_H
#define RAX_H
@@ -107,16 +77,16 @@ typedef struct raxNode {
* Note how the character is not stored in the children but in the
* edge of the parents:
*
* [header iscompr=0][abc][a-ptr][b-ptr][c-ptr](value-ptr?)
* [header strlen=0][abc][a-ptr][b-ptr][c-ptr](value-ptr?)
*
* if node is compressed (iscompr bit is 1) the node has 1 children.
* if node is compressed (strlen != 0) the node has 1 children.
* In that case the 'size' bytes of the string stored immediately at
* the start of the data section, represent a sequence of successive
* nodes linked one after the other, for which only the last one in
* the sequence is actually represented as a node, and pointed to by
* the current compressed node.
*
* [header iscompr=1][xyz][z-ptr](value-ptr?)
* [header strlen=3][xyz][z-ptr](value-ptr?)
*
* Both compressed and not compressed nodes can represent a key
* with associated data in the radix tree at any level (not just terminal
@@ -206,8 +176,6 @@ void raxStop(raxIterator *it);
int raxEOF(raxIterator *it);
void raxShow(rax *rax);
uint64_t raxSize(rax *rax);
unsigned long raxTouch(raxNode *n);
void raxSetDebugMsg(int onoff);
/* Internal API. May be used by the node callback in order to access rax nodes
* in a low level way, so this function is exported as well. */
+54 -167
View File
@@ -545,7 +545,7 @@ static void cliIntegrateHelp(void) {
ch->params = sdscat(ch->params,"key ");
args--;
}
while(args-- > 0) ch->params = sdscat(ch->params,"arg ");
while(args--) ch->params = sdscat(ch->params,"arg ");
if (entry->element[1]->integer < 0)
ch->params = sdscat(ch->params,"...options...");
ch->summary = "Help not available";
@@ -1088,7 +1088,6 @@ static int cliSendCommand(int argc, char **argv, long repeat) {
output_raw = 0;
if (!strcasecmp(command,"info") ||
!strcasecmp(command,"lolwut") ||
(argc >= 2 && !strcasecmp(command,"debug") &&
!strcasecmp(argv[1],"htstats")) ||
(argc >= 2 && !strcasecmp(command,"debug") &&
@@ -1155,7 +1154,7 @@ static int cliSendCommand(int argc, char **argv, long repeat) {
}
if (config.slave_mode) {
printf("Entering replica output mode... (press Ctrl-C to quit)\n");
printf("Entering slave output mode... (press Ctrl-C to quit)\n");
slaveMode();
config.slave_mode = 0;
zfree(argvlen);
@@ -1173,6 +1172,16 @@ static int cliSendCommand(int argc, char **argv, long repeat) {
} else if (!strcasecmp(command,"auth") && argc == 2) {
cliSelect();
}
/* Issue the command again if we got redirected in cluster mode */
if (config.cluster_mode && config.cluster_reissue_command) {
cliConnect(CC_FORCE);
config.cluster_reissue_command = 0;
/* for a '-MOVED' or '-ASK' response, we need to issue the command again, so
* add repeat by 1. */
repeat++;
}
}
if (config.interval) usleep(config.interval);
fflush(stdout); /* Make it grep friendly */
@@ -1273,8 +1282,6 @@ static int parseOptions(int argc, char **argv) {
config.lru_test_sample_size = strtoll(argv[++i],NULL,10);
} else if (!strcmp(argv[i],"--slave")) {
config.slave_mode = 1;
} else if (!strcmp(argv[i],"--replica")) {
config.slave_mode = 1;
} else if (!strcmp(argv[i],"--stat")) {
config.stat_mode = 1;
} else if (!strcmp(argv[i],"--scan")) {
@@ -1471,7 +1478,7 @@ static void usage(void) {
" --latency-dist Shows latency as a spectrum, requires xterm 256 colors.\n"
" Default time interval is 1 sec. Change it using -i.\n"
" --lru-test <keys> Simulate a cache workload with an 80-20 distribution.\n"
" --replica Simulate a replica showing commands received from the master.\n"
" --slave Simulate a slave showing commands received from the master.\n"
" --rdb <filename> Transfer an RDB dump from remote server to local file.\n"
" --pipe Transfer raw Redis protocol from stdin to server.\n"
" --pipe-timeout <n> In --pipe mode, abort with error if after sending all data.\n"
@@ -1488,7 +1495,7 @@ static void usage(void) {
" --ldb Used with --eval enable the Redis Lua debugger.\n"
" --ldb-sync-mode Like --ldb but uses the synchronous Lua debugger, in\n"
" this mode the server is blocked and script changes are\n"
" not rolled back from the server memory.\n"
" are not rolled back from the server memory.\n"
" --cluster <command> [args...] [opts...]\n"
" Cluster Manager command and arguments (see below).\n"
" --verbose Verbose mode.\n"
@@ -1553,13 +1560,8 @@ static int issueCommandRepeat(int argc, char **argv, long repeat) {
cliPrintContextError();
return REDIS_ERR;
}
}
/* Issue the command again if we got redirected in cluster mode */
if (config.cluster_mode && config.cluster_reissue_command) {
cliConnect(CC_FORCE);
} else {
break;
}
} else
break;
}
return REDIS_OK;
}
@@ -2726,7 +2728,8 @@ static int clusterManagerSetSlot(clusterManagerNode *node1,
if (err != NULL) {
*err = zmalloc((reply->len + 1) * sizeof(char));
strcpy(*err, reply->str);
} else CLUSTER_MANAGER_PRINT_REPLY_ERROR(node1, reply->str);
CLUSTER_MANAGER_PRINT_REPLY_ERROR(node1, err);
}
goto cleanup;
}
cleanup:
@@ -2847,21 +2850,14 @@ static int clusterManagerMigrateKeysInSlot(clusterManagerNode *source,
if (migrate_reply == NULL) goto next;
if (migrate_reply->type == REDIS_REPLY_ERROR) {
if (do_fix && strstr(migrate_reply->str, "BUSYKEY")) {
/* If the key already exists, try to migrate keys
* adding REPLACE option.
* If the key's slot is not served, try to assign slot
* to the target node. */
int is_busy = (strstr(migrate_reply->str, "BUSYKEY") != NULL);
if (strstr(migrate_reply->str, "slot not served") != NULL)
clusterManagerSetSlot(source, target, slot, "node", NULL);
clusterManagerLogWarn("*** Target key exists. "
"Replacing it for FIX.\n");
freeReplyObject(migrate_reply);
/* Try to migrate keys adding REPLACE option. */
migrate_reply = clusterManagerMigrateKeysInReply(source,
target,
reply,
is_busy,
timeout,
1, timeout,
NULL);
success = (migrate_reply != NULL &&
migrate_reply->type != REDIS_REPLY_ERROR);
@@ -3307,8 +3303,8 @@ static sds clusterManagerGetConfigSignature(clusterManagerNode *node) {
nodename = token;
tot_size = (p - token);
name_len = tot_size++; // Make room for ':' in tot_size
}
if (++i == 8) break;
} else if (i == 8) break;
i++;
}
if (i != 8) continue;
if (nodename == NULL) continue;
@@ -3347,7 +3343,7 @@ static sds clusterManagerGetConfigSignature(clusterManagerNode *node) {
char *sp = cfg + name_len;
*(sp++) = ':';
for (i = 0; i < c; i++) {
if (i > 0) *(sp++) = ',';
if (i > 0) *(sp++) = '|';
int slen = strlen(slots[i]);
memcpy(sp, slots[i], slen);
sp += slen;
@@ -3589,17 +3585,10 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
clusterManagerNode *n = node_n->value;
clusterManagerLogInfo(">>> Covering slot %s with %s:%d\n",
slot, n->ip, n->port);
/* Ensure the slot is not already assigned. */
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER DELSLOTS %s", slot);
if (r) freeReplyObject(r);
r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER ADDSLOTS %s", slot);
if (!clusterManagerCheckRedisReply(n, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
r = CLUSTER_MANAGER_COMMAND(n, "CLUSTER BUMPEPOCH");
if (!clusterManagerCheckRedisReply(n, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
/* Since CLUSTER ADDSLOTS succeeded, we also update the slot
* info into the node struct, in order to keep it synced */
@@ -3627,17 +3616,10 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
clusterManagerNode *n = fn->value;
clusterManagerLogInfo(">>> Covering slot %s with %s:%d\n",
slot, n->ip, n->port);
/* Ensure the slot is not already assigned. */
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER DELSLOTS %s", slot);
if (r) freeReplyObject(r);
r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER ADDSLOTS %s", slot);
if (!clusterManagerCheckRedisReply(n, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
r = CLUSTER_MANAGER_COMMAND(n, "CLUSTER BUMPEPOCH");
if (!clusterManagerCheckRedisReply(n, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
/* Since CLUSTER ADDSLOTS succeeded, we also update the slot
* info into the node struct, in order to keep it synced */
@@ -3671,11 +3653,7 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
clusterManagerLogInfo(">>> Covering slot %s moving keys "
"to %s:%d\n", slot,
target->ip, target->port);
/* Ensure the slot is not already assigned. */
redisReply *r = CLUSTER_MANAGER_COMMAND(target,
"CLUSTER DELSLOTS %s", slot);
if (r) freeReplyObject(r);
r = CLUSTER_MANAGER_COMMAND(target,
"CLUSTER ADDSLOTS %s", slot);
if (!clusterManagerCheckRedisReply(target, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
@@ -3684,9 +3662,6 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
"CLUSTER SETSLOT %s %s", slot, "STABLE");
if (!clusterManagerCheckRedisReply(target, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
r = CLUSTER_MANAGER_COMMAND(target, "CLUSTER BUMPEPOCH");
if (!clusterManagerCheckRedisReply(target, r, NULL)) fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
/* Since CLUSTER ADDSLOTS succeeded, we also update the slot
* info into the node struct, in order to keep it synced */
@@ -3697,22 +3672,14 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
while ((nln = listNext(&nli)) != NULL) {
clusterManagerNode *src = nln->value;
if (src == target) continue;
/* Assign the slot to target node in the source node. */
redisReply *r = CLUSTER_MANAGER_COMMAND(src,
"CLUSTER SETSLOT %s %s %s", slot,
"NODE", target->name);
if (!clusterManagerCheckRedisReply(src, r, NULL))
fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
/* Set the source node in 'importing' state
* (even if we will actually migrate keys away)
* in order to avoid receiving redirections
* for MIGRATE. */
r = CLUSTER_MANAGER_COMMAND(src,
redisReply *r = CLUSTER_MANAGER_COMMAND(src,
"CLUSTER SETSLOT %s %s %s", slot,
"IMPORTING", target->name);
if (!clusterManagerCheckRedisReply(src, r, NULL))
if (!clusterManagerCheckRedisReply(target, r, NULL))
fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
@@ -3722,13 +3689,6 @@ static int clusterManagerFixSlotsCoverage(char *all_slots) {
fixed = -1;
goto cleanup;
}
r = CLUSTER_MANAGER_COMMAND(src,
"CLUSTER SETSLOT %s %s", slot,
"STABLE");
if (!clusterManagerCheckRedisReply(src, r, NULL))
fixed = -1;
if (r) freeReplyObject(r);
if (fixed < 0) goto cleanup;
}
fixed++;
}
@@ -3762,78 +3722,43 @@ static int clusterManagerFixOpenSlot(int slot) {
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n->flags & CLUSTER_MANAGER_FLAG_SLAVE) continue;
if (n->slots[slot]) listAddNodeTail(owners, n);
else {
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER COUNTKEYSINSLOT %d", slot);
success = clusterManagerCheckRedisReply(n, r, NULL);
if (success && r->integer > 0) {
clusterManagerLogWarn("*** Found keys about slot %d "
"in non-owner node %s:%d!\n", slot,
n->ip, n->port);
listAddNodeTail(owners, n);
}
if (r) freeReplyObject(r);
if (!success) goto cleanup;
if (n->slots[slot]) {
if (owner == NULL) owner = n;
listAddNodeTail(owners, n);
}
}
if (listLength(owners) == 1) owner = listFirst(owners)->value;
listRewind(cluster_manager.nodes, &li);
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n->flags & CLUSTER_MANAGER_FLAG_SLAVE) continue;
int is_migrating = 0, is_importing = 0;
if (n->migrating) {
for (int i = 0; i < n->migrating_count; i += 2) {
sds migrating_slot = n->migrating[i];
if (atoi(migrating_slot) == slot) {
char *sep = (listLength(migrating) == 0 ? "" : ",");
migrating_str = sdscatfmt(migrating_str, "%s%s:%u",
migrating_str = sdscatfmt(migrating_str, "%s%S:%u",
sep, n->ip, n->port);
listAddNodeTail(migrating, n);
is_migrating = 1;
break;
}
}
}
if (!is_migrating && n->importing) {
if (n->importing) {
for (int i = 0; i < n->importing_count; i += 2) {
sds importing_slot = n->importing[i];
if (atoi(importing_slot) == slot) {
char *sep = (listLength(importing) == 0 ? "" : ",");
importing_str = sdscatfmt(importing_str, "%s%s:%u",
importing_str = sdscatfmt(importing_str, "%s%S:%u",
sep, n->ip, n->port);
listAddNodeTail(importing, n);
is_importing = 1;
break;
}
}
}
/* If the node is neither migrating nor importing and it's not
* the owner, then is added to the importing list in case
* it has keys in the slot. */
if (!is_migrating && !is_importing && n != owner) {
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER COUNTKEYSINSLOT %d", slot);
success = clusterManagerCheckRedisReply(n, r, NULL);
if (success && r->integer > 0) {
clusterManagerLogWarn("*** Found keys about slot %d "
"in node %s:%d!\n", slot, n->ip,
n->port);
char *sep = (listLength(importing) == 0 ? "" : ",");
importing_str = sdscatfmt(importing_str, "%s%S:%u",
sep, n->ip, n->port);
listAddNodeTail(importing, n);
}
if (r) freeReplyObject(r);
if (!success) goto cleanup;
}
}
if (sdslen(migrating_str) > 0)
printf("Set as migrating in: %s\n", migrating_str);
if (sdslen(importing_str) > 0)
printf("Set as importing in: %s\n", importing_str);
/* If there is no slot owner, set as owner the node with the biggest
printf("Set as migrating in: %s\n", migrating_str);
printf("Set as importing in: %s\n", importing_str);
/* If there is no slot owner, set as owner the slot with the biggest
* number of keys, among the set of migrating / importing nodes. */
if (owner == NULL) {
clusterManagerLogInfo(">>> Nobody claims ownership, "
@@ -3857,15 +3782,6 @@ static int clusterManagerFixOpenSlot(int slot) {
success = clusterManagerCheckRedisReply(owner, reply, NULL);
if (reply) freeReplyObject(reply);
if (!success) goto cleanup;
/* Ensure that the slot is unassigned before assigning it to the
* owner. */
reply = CLUSTER_MANAGER_COMMAND(owner, "CLUSTER DELSLOTS %d", slot);
success = clusterManagerCheckRedisReply(owner, reply, NULL);
/* Ignore "already unassigned" error. */
if (!success && reply && reply->type == REDIS_REPLY_ERROR &&
strstr(reply->str, "already unassigned") != NULL) success = 1;
if (reply) freeReplyObject(reply);
if (!success) goto cleanup;
reply = CLUSTER_MANAGER_COMMAND(owner, "CLUSTER ADDSLOTS %d", slot);
success = clusterManagerCheckRedisReply(owner, reply, NULL);
if (reply) freeReplyObject(reply);
@@ -3894,43 +3810,32 @@ static int clusterManagerFixOpenSlot(int slot) {
* in migrating state, since migrating is a valid state only for
* slot owners. */
if (listLength(owners) > 1) {
/* Owner cannot be NULL at this point, since if there are more owners,
* the owner has been set in the previous condition (owner == NULL). */
assert(owner != NULL);
owner = clusterManagerGetNodeWithMostKeysInSlot(owners, slot, NULL);
listRewind(owners, &li);
redisReply *reply = NULL;
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n == owner) continue;
reply = CLUSTER_MANAGER_COMMAND(n, "CLUSTER DELSLOTS %d", slot);
reply = CLUSTER_MANAGER_COMMAND(n, "CLUSTER DELSLOT %d", slot);
success = clusterManagerCheckRedisReply(n, reply, NULL);
/* Ignore "already unassigned" error. */
if (!success && reply && reply->type == REDIS_REPLY_ERROR &&
strstr(reply->str, "already unassigned") != NULL) success = 1;
if (reply) freeReplyObject(reply);
if (!success) goto cleanup;
n->slots[slot] = 0;
/* Assign the slot to the owner in the node 'n' configuration.' */
success = clusterManagerSetSlot(n, owner, slot, "node", NULL);
if (!success) goto cleanup;
success = clusterManagerSetSlot(n, owner, slot, "importing", NULL);
if (!success) goto cleanup;
/* Avoid duplicates. */
clusterManagerRemoveNodeFromList(importing, n);
clusterManagerRemoveNodeFromList(importing, n); //Avoid duplicates
listAddNodeTail(importing, n);
/* Ensure that the node is not in the migrating list. */
clusterManagerRemoveNodeFromList(migrating, n);
}
reply = CLUSTER_MANAGER_COMMAND(owner, "CLUSTER BUMPEPOCH");
success = clusterManagerCheckRedisReply(owner, reply, NULL);
if (reply) freeReplyObject(reply);
if (!success) goto cleanup;
}
int move_opts = CLUSTER_MANAGER_OPT_VERBOSE;
/* Case 1: The slot is in migrating state in one node, and in
* importing state in 1 node. That's trivial to address. */
/* Case 1: The slot is in migrating state in one slot, and in
* importing state in 1 slot. That's trivial to address. */
if (listLength(migrating) == 1 && listLength(importing) == 1) {
clusterManagerNode *src = listFirst(migrating)->value;
clusterManagerNode *dst = listFirst(importing)->value;
clusterManagerLogInfo(">>> Case 1: Moving slot %d from "
"%s:%d to %s:%d\n", slot,
src->ip, src->port, dst->ip, dst->port);
success = clusterManagerMoveSlot(src, dst, slot, move_opts, NULL);
}
/* Case 2: There are multiple nodes that claim the slot as importing,
@@ -3938,7 +3843,7 @@ static int clusterManagerFixOpenSlot(int slot) {
* the slot. In this case we just move all the keys to the owner
* according to the configuration. */
else if (listLength(migrating) == 0 && listLength(importing) > 0) {
clusterManagerLogInfo(">>> Case 2: Moving all the %d slot keys to its "
clusterManagerLogInfo(">>> Moving all the %d slot keys to its "
"owner %s:%d\n", slot, owner->ip, owner->port);
move_opts |= CLUSTER_MANAGER_OPT_COLD;
listRewind(importing, &li);
@@ -3956,42 +3861,25 @@ static int clusterManagerFixOpenSlot(int slot) {
if (r) freeReplyObject(r);
if (!success) goto cleanup;
}
/* Since the slot has been moved in "cold" mode, ensure that all the
* other nodes update their own configuration about the slot itself. */
listRewind(cluster_manager.nodes, &li);
while ((ln = listNext(&li)) != NULL) {
clusterManagerNode *n = ln->value;
if (n == owner) continue;
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER SETSLOT %d %s %s", slot, "NODE", owner->name);
success = clusterManagerCheckRedisReply(n, r, NULL);
if (r) freeReplyObject(r);
if (!success) goto cleanup;
}
} else {
int try_to_close_slot = (listLength(importing) == 0 &&
listLength(migrating) == 1);
if (try_to_close_slot) {
clusterManagerNode *n = listFirst(migrating)->value;
if (!owner || owner != n) {
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER GETKEYSINSLOT %d %d", slot, 10);
success = clusterManagerCheckRedisReply(n, r, NULL);
if (r) {
if (success) try_to_close_slot = (r->elements == 0);
freeReplyObject(r);
}
if (!success) goto cleanup;
redisReply *r = CLUSTER_MANAGER_COMMAND(n,
"CLUSTER GETKEYSINSLOT %d %d", slot, 10);
success = clusterManagerCheckRedisReply(n, r, NULL);
if (r) {
if (success) try_to_close_slot = (r->elements == 0);
freeReplyObject(r);
}
if (!success) goto cleanup;
}
/* Case 3: There are no slots claiming to be in importing state, but
* there is a migrating node that actually don't have any key or is the
* slot owner. We can just close the slot, probably a reshard interrupted
* in the middle. */
* there is a migrating node that actually don't have any key. We
* can just close the slot, probably a reshard interrupted in the middle. */
if (try_to_close_slot) {
clusterManagerNode *n = listFirst(migrating)->value;
clusterManagerLogInfo(">>> Case 3: Closing slot %d on %s:%d\n",
slot, n->ip, n->port);
redisReply *r = CLUSTER_MANAGER_COMMAND(n, "CLUSTER SETSLOT %d %s",
slot, "STABLE");
success = clusterManagerCheckRedisReply(n, r, NULL);
@@ -4120,7 +4008,6 @@ static int clusterManagerCheckCluster(int quiet) {
result = 0;
if (do_fix/* && result*/) {
dictType dtype = clusterManagerDictType;
dtype.keyDestructor = dictSdsDestructor;
dtype.valDestructor = dictListDestructor;
clusterManagerUncoveredSlots = dictCreate(&dtype, NULL);
int fixed = clusterManagerFixSlotsCoverage(slots);
@@ -4157,7 +4044,7 @@ static clusterManagerNode *clusterNodeForResharding(char *id,
static list *clusterManagerComputeReshardTable(list *sources, int numslots) {
list *moved = listCreate();
int src_count = listLength(sources), i = 0, tot_slots = 0, j;
clusterManagerNode **sorted = zmalloc(src_count * sizeof(*sorted));
clusterManagerNode **sorted = zmalloc(src_count * sizeof(**sorted));
listIter li;
listNode *ln;
listRewind(sources, &li);
-52
View File
@@ -117,10 +117,6 @@
#define REDISMODULE_NODE_FAIL (1<<4)
#define REDISMODULE_NODE_NOFAILOVER (1<<5)
#define REDISMODULE_CLUSTER_FLAG_NONE 0
#define REDISMODULE_CLUSTER_FLAG_NO_FAILOVER (1<<1)
#define REDISMODULE_CLUSTER_FLAG_NO_REDIRECTION (1<<2)
#define REDISMODULE_NOT_USED(V) ((void) V)
/* This type represents a timer handle, and is returned when a timer is
@@ -145,8 +141,6 @@ typedef struct RedisModuleType RedisModuleType;
typedef struct RedisModuleDigest RedisModuleDigest;
typedef struct RedisModuleBlockedClient RedisModuleBlockedClient;
typedef struct RedisModuleClusterInfo RedisModuleClusterInfo;
typedef struct RedisModuleDict RedisModuleDict;
typedef struct RedisModuleDictIter RedisModuleDictIter;
typedef int (*RedisModuleCmdFunc)(RedisModuleCtx *ctx, RedisModuleString **argv, int argc);
typedef void (*RedisModuleDisconnectFunc)(RedisModuleCtx *ctx, RedisModuleBlockedClient *bc);
@@ -279,28 +273,6 @@ long long REDISMODULE_API_FUNC(RedisModule_Milliseconds)(void);
void REDISMODULE_API_FUNC(RedisModule_DigestAddStringBuffer)(RedisModuleDigest *md, unsigned char *ele, size_t len);
void REDISMODULE_API_FUNC(RedisModule_DigestAddLongLong)(RedisModuleDigest *md, long long ele);
void REDISMODULE_API_FUNC(RedisModule_DigestEndSequence)(RedisModuleDigest *md);
RedisModuleDict *REDISMODULE_API_FUNC(RedisModule_CreateDict)(RedisModuleCtx *ctx);
void REDISMODULE_API_FUNC(RedisModule_FreeDict)(RedisModuleCtx *ctx, RedisModuleDict *d);
uint64_t REDISMODULE_API_FUNC(RedisModule_DictSize)(RedisModuleDict *d);
int REDISMODULE_API_FUNC(RedisModule_DictSetC)(RedisModuleDict *d, void *key, size_t keylen, void *ptr);
int REDISMODULE_API_FUNC(RedisModule_DictReplaceC)(RedisModuleDict *d, void *key, size_t keylen, void *ptr);
int REDISMODULE_API_FUNC(RedisModule_DictSet)(RedisModuleDict *d, RedisModuleString *key, void *ptr);
int REDISMODULE_API_FUNC(RedisModule_DictReplace)(RedisModuleDict *d, RedisModuleString *key, void *ptr);
void *REDISMODULE_API_FUNC(RedisModule_DictGetC)(RedisModuleDict *d, void *key, size_t keylen, int *nokey);
void *REDISMODULE_API_FUNC(RedisModule_DictGet)(RedisModuleDict *d, RedisModuleString *key, int *nokey);
int REDISMODULE_API_FUNC(RedisModule_DictDelC)(RedisModuleDict *d, void *key, size_t keylen, void *oldval);
int REDISMODULE_API_FUNC(RedisModule_DictDel)(RedisModuleDict *d, RedisModuleString *key, void *oldval);
RedisModuleDictIter *REDISMODULE_API_FUNC(RedisModule_DictIteratorStartC)(RedisModuleDict *d, const char *op, void *key, size_t keylen);
RedisModuleDictIter *REDISMODULE_API_FUNC(RedisModule_DictIteratorStart)(RedisModuleDict *d, const char *op, RedisModuleString *key);
void REDISMODULE_API_FUNC(RedisModule_DictIteratorStop)(RedisModuleDictIter *di);
int REDISMODULE_API_FUNC(RedisModule_DictIteratorReseekC)(RedisModuleDictIter *di, const char *op, void *key, size_t keylen);
int REDISMODULE_API_FUNC(RedisModule_DictIteratorReseek)(RedisModuleDictIter *di, const char *op, RedisModuleString *key);
void *REDISMODULE_API_FUNC(RedisModule_DictNextC)(RedisModuleDictIter *di, size_t *keylen, void **dataptr);
void *REDISMODULE_API_FUNC(RedisModule_DictPrevC)(RedisModuleDictIter *di, size_t *keylen, void **dataptr);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_DictNext)(RedisModuleCtx *ctx, RedisModuleDictIter *di, void **dataptr);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_DictPrev)(RedisModuleCtx *ctx, RedisModuleDictIter *di, void **dataptr);
int REDISMODULE_API_FUNC(RedisModule_DictCompareC)(RedisModuleDictIter *di, const char *op, void *key, size_t keylen);
int REDISMODULE_API_FUNC(RedisModule_DictCompare)(RedisModuleDictIter *di, const char *op, RedisModuleString *key);
/* Experimental APIs */
#ifdef REDISMODULE_EXPERIMENTAL_API
@@ -331,7 +303,6 @@ size_t REDISMODULE_API_FUNC(RedisModule_GetClusterSize)(void);
void REDISMODULE_API_FUNC(RedisModule_GetRandomBytes)(unsigned char *dst, size_t len);
void REDISMODULE_API_FUNC(RedisModule_GetRandomHexChars)(char *dst, size_t len);
void REDISMODULE_API_FUNC(RedisModule_SetDisconnectCallback)(RedisModuleBlockedClient *bc, RedisModuleDisconnectFunc callback);
void REDISMODULE_API_FUNC(RedisModule_SetClusterFlags)(RedisModuleCtx *ctx, uint64_t flags);
#endif
/* This is included inline inside each Redis module. */
@@ -441,28 +412,6 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(DigestAddStringBuffer);
REDISMODULE_GET_API(DigestAddLongLong);
REDISMODULE_GET_API(DigestEndSequence);
REDISMODULE_GET_API(CreateDict);
REDISMODULE_GET_API(FreeDict);
REDISMODULE_GET_API(DictSize);
REDISMODULE_GET_API(DictSetC);
REDISMODULE_GET_API(DictReplaceC);
REDISMODULE_GET_API(DictSet);
REDISMODULE_GET_API(DictReplace);
REDISMODULE_GET_API(DictGetC);
REDISMODULE_GET_API(DictGet);
REDISMODULE_GET_API(DictDelC);
REDISMODULE_GET_API(DictDel);
REDISMODULE_GET_API(DictIteratorStartC);
REDISMODULE_GET_API(DictIteratorStart);
REDISMODULE_GET_API(DictIteratorStop);
REDISMODULE_GET_API(DictIteratorReseekC);
REDISMODULE_GET_API(DictIteratorReseek);
REDISMODULE_GET_API(DictNextC);
REDISMODULE_GET_API(DictPrevC);
REDISMODULE_GET_API(DictNext);
REDISMODULE_GET_API(DictPrev);
REDISMODULE_GET_API(DictCompare);
REDISMODULE_GET_API(DictCompareC);
#ifdef REDISMODULE_EXPERIMENTAL_API
REDISMODULE_GET_API(GetThreadSafeContext);
@@ -491,7 +440,6 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(GetClusterSize);
REDISMODULE_GET_API(GetRandomBytes);
REDISMODULE_GET_API(GetRandomHexChars);
REDISMODULE_GET_API(SetClusterFlags);
#endif
if (RedisModule_IsModuleNameBusy && RedisModule_IsModuleNameBusy(name)) return REDISMODULE_ERR;
+37 -49
View File
@@ -48,7 +48,7 @@ int cancelReplicationHandshake(void);
/* Return the pointer to a string representing the slave ip:listening_port
* pair. Mostly useful for logging, since we want to log a slave using its
* IP address and its listening port which is more clear for the user, for
* example: "Closing connection with replica 10.1.2.3:6380". */
* example: "Closing connection with slave 10.1.2.3:6380". */
char *replicationGetSlaveName(client *c) {
static char buf[NET_PEER_ID_LEN];
char ip[NET_IP_STR_LEN];
@@ -64,7 +64,7 @@ char *replicationGetSlaveName(client *c) {
if (c->slave_listening_port)
anetFormatAddr(buf,sizeof(buf),ip,c->slave_listening_port);
else
snprintf(buf,sizeof(buf),"%s:<unknown-replica-port>",ip);
snprintf(buf,sizeof(buf),"%s:<unknown-slave-port>",ip);
} else {
snprintf(buf,sizeof(buf),"client id #%llu",
(unsigned long long) c->id);
@@ -344,7 +344,7 @@ void replicationFeedMonitors(client *c, list *monitors, int dictid, robj **argv,
long long addReplyReplicationBacklog(client *c, long long offset) {
long long j, skip, len;
serverLog(LL_DEBUG, "[PSYNC] Replica request offset: %lld", offset);
serverLog(LL_DEBUG, "[PSYNC] Slave request offset: %lld", offset);
if (server.repl_backlog_histlen == 0) {
serverLog(LL_DEBUG, "[PSYNC] Backlog history len is zero");
@@ -472,7 +472,7 @@ int masterTryPartialResynchronization(client *c) {
strcasecmp(master_replid, server.replid2))
{
serverLog(LL_NOTICE,"Partial resynchronization not accepted: "
"Replication ID mismatch (Replica asked for '%s', my "
"Replication ID mismatch (Slave asked for '%s', my "
"replication IDs are '%s' and '%s')",
master_replid, server.replid, server.replid2);
} else {
@@ -481,7 +481,7 @@ int masterTryPartialResynchronization(client *c) {
"up to %lld", psync_offset, server.second_replid_offset);
}
} else {
serverLog(LL_NOTICE,"Full resync requested by replica %s",
serverLog(LL_NOTICE,"Full resync requested by slave %s",
replicationGetSlaveName(c));
}
goto need_full_resync;
@@ -493,10 +493,10 @@ int masterTryPartialResynchronization(client *c) {
psync_offset > (server.repl_backlog_off + server.repl_backlog_histlen))
{
serverLog(LL_NOTICE,
"Unable to partial resync with replica %s for lack of backlog (Replica request was: %lld).", replicationGetSlaveName(c), psync_offset);
"Unable to partial resync with slave %s for lack of backlog (Slave request was: %lld).", replicationGetSlaveName(c), psync_offset);
if (psync_offset > server.master_repl_offset) {
serverLog(LL_WARNING,
"Warning: replica %s tried to PSYNC with an offset that is greater than the master replication offset.", replicationGetSlaveName(c));
"Warning: slave %s tried to PSYNC with an offset that is greater than the master replication offset.", replicationGetSlaveName(c));
}
goto need_full_resync;
}
@@ -567,7 +567,7 @@ int startBgsaveForReplication(int mincapa) {
listNode *ln;
serverLog(LL_NOTICE,"Starting BGSAVE for SYNC with target: %s",
socket_target ? "replicas sockets" : "disk");
socket_target ? "slaves sockets" : "disk");
rdbSaveInfo rsi, *rsiptr;
rsiptr = rdbPopulateSaveInfo(&rsi);
@@ -644,7 +644,7 @@ void syncCommand(client *c) {
return;
}
serverLog(LL_NOTICE,"Replica %s asks for synchronization",
serverLog(LL_NOTICE,"Slave %s asks for synchronization",
replicationGetSlaveName(c));
/* Try a partial resynchronization if this is a PSYNC command.
@@ -725,7 +725,7 @@ void syncCommand(client *c) {
} else {
/* No way, we need to wait for the next BGSAVE in order to
* register differences. */
serverLog(LL_NOTICE,"Can't attach the replica to the current BGSAVE. Waiting for next BGSAVE for SYNC");
serverLog(LL_NOTICE,"Can't attach the slave to the current BGSAVE. Waiting for next BGSAVE for SYNC");
}
/* CASE 2: BGSAVE is in progress, with socket target. */
@@ -798,7 +798,7 @@ void replconfCommand(client *c) {
memcpy(c->slave_ip,ip,sdslen(ip)+1);
} else {
addReplyErrorFormat(c,"REPLCONF ip-address provided by "
"replica instance is too long: %zd bytes", sdslen(ip));
"slave instance is too long: %zd bytes", sdslen(ip));
return;
}
} else if (!strcasecmp(c->argv[j]->ptr,"capa")) {
@@ -858,12 +858,12 @@ void putSlaveOnline(client *slave) {
slave->repl_ack_time = server.unixtime; /* Prevent false timeout. */
if (aeCreateFileEvent(server.el, slave->fd, AE_WRITABLE,
sendReplyToClient, slave) == AE_ERR) {
serverLog(LL_WARNING,"Unable to register writable event for replica bulk transfer: %s", strerror(errno));
serverLog(LL_WARNING,"Unable to register writable event for slave bulk transfer: %s", strerror(errno));
freeClient(slave);
return;
}
refreshGoodSlavesCount();
serverLog(LL_NOTICE,"Synchronization with replica %s succeeded",
serverLog(LL_NOTICE,"Synchronization with slave %s succeeded",
replicationGetSlaveName(slave));
}
@@ -880,7 +880,7 @@ void sendBulkToSlave(aeEventLoop *el, int fd, void *privdata, int mask) {
if (slave->replpreamble) {
nwritten = write(fd,slave->replpreamble,sdslen(slave->replpreamble));
if (nwritten == -1) {
serverLog(LL_VERBOSE,"Write error sending RDB preamble to replica: %s",
serverLog(LL_VERBOSE,"Write error sending RDB preamble to slave: %s",
strerror(errno));
freeClient(slave);
return;
@@ -900,14 +900,14 @@ void sendBulkToSlave(aeEventLoop *el, int fd, void *privdata, int mask) {
lseek(slave->repldbfd,slave->repldboff,SEEK_SET);
buflen = read(slave->repldbfd,buf,PROTO_IOBUF_LEN);
if (buflen <= 0) {
serverLog(LL_WARNING,"Read error sending DB to replica: %s",
serverLog(LL_WARNING,"Read error sending DB to slave: %s",
(buflen == 0) ? "premature EOF" : strerror(errno));
freeClient(slave);
return;
}
if ((nwritten = write(fd,buf,buflen)) == -1) {
if (errno != EAGAIN) {
serverLog(LL_WARNING,"Write error sending DB to replica: %s",
serverLog(LL_WARNING,"Write error sending DB to slave: %s",
strerror(errno));
freeClient(slave);
}
@@ -961,7 +961,7 @@ void updateSlavesWaitingBgsave(int bgsaveerr, int type) {
* the slave online. */
if (type == RDB_CHILD_TYPE_SOCKET) {
serverLog(LL_NOTICE,
"Streamed RDB transfer with replica %s succeeded (socket). Waiting for REPLCONF ACK from slave to enable streaming",
"Streamed RDB transfer with slave %s succeeded (socket). Waiting for REPLCONF ACK from slave to enable streaming",
replicationGetSlaveName(slave));
/* Note: we wait for a REPLCONF ACK message from slave in
* order to really put it online (install the write handler
@@ -1096,7 +1096,7 @@ void restartAOF() {
sleep(1);
}
if (!retry) {
serverLog(LL_WARNING,"FATAL: this replica instance finished the synchronization with its master, but the AOF can't be turned on. Exiting now.");
serverLog(LL_WARNING,"FATAL: this slave instance finished the synchronization with its master, but the AOF can't be turned on. Exiting now.");
exit(1);
}
}
@@ -1161,12 +1161,12 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
* at the next call. */
server.repl_transfer_size = 0;
serverLog(LL_NOTICE,
"MASTER <-> REPLICA sync: receiving streamed RDB from master");
"MASTER <-> SLAVE sync: receiving streamed RDB from master");
} else {
usemark = 0;
server.repl_transfer_size = strtol(buf+1,NULL,10);
serverLog(LL_NOTICE,
"MASTER <-> REPLICA sync: receiving %lld bytes from master",
"MASTER <-> SLAVE sync: receiving %lld bytes from master",
(long long) server.repl_transfer_size);
}
return;
@@ -1207,7 +1207,7 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
server.repl_transfer_lastio = server.unixtime;
if ((nwritten = write(server.repl_transfer_fd,buf,nread)) != nread) {
serverLog(LL_WARNING,"Write error or short write writing to the DB dump file needed for MASTER <-> REPLICA synchronization: %s",
serverLog(LL_WARNING,"Write error or short write writing to the DB dump file needed for MASTER <-> SLAVE synchronization: %s",
(nwritten == -1) ? strerror(errno) : "short write");
goto error;
}
@@ -1245,24 +1245,12 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
if (eof_reached) {
int aof_is_enabled = server.aof_state != AOF_OFF;
/* Ensure background save doesn't overwrite synced data */
if (server.rdb_child_pid != -1) {
serverLog(LL_NOTICE,
"Replica is about to load the RDB file received from the "
"master, but there is a pending RDB child running. "
"Killing process %ld and removing its temp file to avoid "
"any race",
(long) server.rdb_child_pid);
kill(server.rdb_child_pid,SIGUSR1);
rdbRemoveTempFile(server.rdb_child_pid);
}
if (rename(server.repl_transfer_tmpfile,server.rdb_filename) == -1) {
serverLog(LL_WARNING,"Failed trying to rename the temp DB into dump.rdb in MASTER <-> REPLICA synchronization: %s", strerror(errno));
serverLog(LL_WARNING,"Failed trying to rename the temp DB into dump.rdb in MASTER <-> SLAVE synchronization: %s", strerror(errno));
cancelReplicationHandshake();
return;
}
serverLog(LL_NOTICE, "MASTER <-> REPLICA sync: Flushing old data");
serverLog(LL_NOTICE, "MASTER <-> SLAVE sync: Flushing old data");
/* We need to stop any AOFRW fork before flusing and parsing
* RDB, otherwise we'll create a copy-on-write disaster. */
if(aof_is_enabled) stopAppendOnly();
@@ -1276,7 +1264,7 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
* rdbLoad() will call the event loop to process events from time to
* time for non blocking loading. */
aeDeleteFileEvent(server.el,server.repl_transfer_s,AE_READABLE);
serverLog(LL_NOTICE, "MASTER <-> REPLICA sync: Loading DB in memory");
serverLog(LL_NOTICE, "MASTER <-> SLAVE sync: Loading DB in memory");
rdbSaveInfo rsi = RDB_SAVE_INFO_INIT;
if (rdbLoad(server.rdb_filename,&rsi) != C_OK) {
serverLog(LL_WARNING,"Failed trying to load the MASTER synchronization DB from disk");
@@ -1304,7 +1292,7 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
* masters after a failover. */
if (server.repl_backlog == NULL) createReplicationBacklog();
serverLog(LL_NOTICE, "MASTER <-> REPLICA sync: Finished with success");
serverLog(LL_NOTICE, "MASTER <-> SLAVE sync: Finished with success");
/* Restart the AOF subsystem now that we finished the sync. This
* will trigger an AOF rewrite, and when done will start appending
* to the new file. */
@@ -1803,7 +1791,7 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
* uninstalling the read handler from the file descriptor. */
if (psync_result == PSYNC_CONTINUE) {
serverLog(LL_NOTICE, "MASTER <-> REPLICA sync: Master accepted a Partial Resynchronization.");
serverLog(LL_NOTICE, "MASTER <-> SLAVE sync: Master accepted a Partial Resynchronization.");
return;
}
@@ -1835,7 +1823,7 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
sleep(1);
}
if (dfd == -1) {
serverLog(LL_WARNING,"Opening the temp file needed for MASTER <-> REPLICA synchronization: %s",strerror(errno));
serverLog(LL_WARNING,"Opening the temp file needed for MASTER <-> SLAVE synchronization: %s",strerror(errno));
goto error;
}
@@ -2009,11 +1997,11 @@ void replicationHandleMasterDisconnection(void) {
* the slaves only if we'll have to do a full resync with our master. */
}
void replicaofCommand(client *c) {
void slaveofCommand(client *c) {
/* SLAVEOF is not allowed in cluster mode as replication is automatically
* configured using the current address of the master node. */
if (server.cluster_enabled) {
addReplyError(c,"REPLICAOF not allowed in cluster mode.");
addReplyError(c,"SLAVEOF not allowed in cluster mode.");
return;
}
@@ -2037,7 +2025,7 @@ void replicaofCommand(client *c) {
/* Check if we are already attached to the specified slave */
if (server.masterhost && !strcasecmp(server.masterhost,c->argv[1]->ptr)
&& server.masterport == port) {
serverLog(LL_NOTICE,"REPLICAOF would result into synchronization with the master we are already connected with. No operation performed.");
serverLog(LL_NOTICE,"SLAVE OF would result into synchronization with the master we are already connected with. No operation performed.");
addReplySds(c,sdsnew("+OK Already connected to specified master\r\n"));
return;
}
@@ -2045,7 +2033,7 @@ void replicaofCommand(client *c) {
* we can continue. */
replicationSetMaster(c->argv[1]->ptr, port);
sds client = catClientInfoString(sdsempty(),c);
serverLog(LL_NOTICE,"REPLICAOF %s:%d enabled (user request from '%s')",
serverLog(LL_NOTICE,"SLAVE OF %s:%d enabled (user request from '%s')",
server.masterhost, server.masterport, client);
sdsfree(client);
}
@@ -2203,7 +2191,7 @@ void replicationCacheMasterUsingMyself(void) {
unlinkClient(server.master);
server.cached_master = server.master;
server.master = NULL;
serverLog(LL_NOTICE,"Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.");
serverLog(LL_NOTICE,"Before turning into a slave, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.");
}
/* Free a cached master, called when there are no longer the conditions for
@@ -2419,7 +2407,7 @@ void waitCommand(client *c) {
long long offset = c->woff;
if (server.masterhost) {
addReplyError(c,"WAIT cannot be used with replica instances. Please also note that since Redis 4.0 if a replica is configured to be writable (which is not the default) writes to replicas are just local and are not propagated.");
addReplyError(c,"WAIT cannot be used with slave instances. Please also note that since Redis 4.0 if a slave is configured to be writable (which is not the default) writes to slaves are just local and are not propagated.");
return;
}
@@ -2551,7 +2539,7 @@ void replicationCron(void) {
serverLog(LL_NOTICE,"Connecting to MASTER %s:%d",
server.masterhost, server.masterport);
if (connectWithMaster() == C_OK) {
serverLog(LL_NOTICE,"MASTER <-> REPLICA sync started");
serverLog(LL_NOTICE,"MASTER <-> SLAVE sync started");
}
}
@@ -2623,7 +2611,7 @@ void replicationCron(void) {
if (slave->flags & CLIENT_PRE_PSYNC) continue;
if ((server.unixtime - slave->repl_ack_time) > server.repl_timeout)
{
serverLog(LL_WARNING, "Disconnecting timedout replica: %s",
serverLog(LL_WARNING, "Disconnecting timedout slave: %s",
replicationGetSlaveName(slave));
freeClient(slave);
}
@@ -2653,7 +2641,7 @@ void replicationCron(void) {
* be the same as our repl-id.
* 3. We, yet as master, receive some updates, that will not
* increment the master_repl_offset.
* 4. Later we are turned into a slave, connect to the new
* 4. Later we are turned into a slave, connecto to the new
* master that will accept our PSYNC request by second
* replication ID, but there will be data inconsistency
* because we received writes. */
@@ -2662,7 +2650,7 @@ void replicationCron(void) {
freeReplicationBacklog();
serverLog(LL_NOTICE,
"Replication backlog freed after %d seconds "
"without connected replicas.",
"without connected slaves.",
(int) server.repl_backlog_time_limit);
}
}
+18 -34
View File
@@ -512,13 +512,10 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
* could enlarge the memory usage are not allowed, but only if this is the
* first write in the context of this script, otherwise we can't stop
* in the middle. */
if (server.maxmemory && /* Maxmemory is actually enabled. */
!server.loading && /* Don't care about mem if loading. */
!server.masterhost && /* Slave must execute the script. */
server.lua_write_dirty == 0 && /* Script had no side effects so far. */
if (server.maxmemory && server.lua_write_dirty == 0 &&
(cmd->flags & CMD_DENYOOM))
{
if (getMaxmemoryState(NULL,NULL,NULL,NULL) != C_OK) {
if (freeMemoryIfNeeded() == C_ERR) {
luaPushError(lua, shared.oomerr->ptr);
goto cleanup;
}
@@ -777,7 +774,7 @@ int luaRedisSetReplCommand(lua_State *lua) {
flags = lua_tonumber(lua,-1);
if ((flags & ~(PROPAGATE_AOF|PROPAGATE_REPL)) != 0) {
lua_pushstring(lua, "Invalid replication flags. Use REPL_AOF, REPL_REPLICA, REPL_ALL or REPL_NONE.");
lua_pushstring(lua, "Invalid replication flags. Use REPL_AOF, REPL_SLAVE, REPL_ALL or REPL_NONE.");
return lua_error(lua);
}
server.lua_repl = flags;
@@ -917,6 +914,7 @@ void scriptingInit(int setup) {
server.lua_client = NULL;
server.lua_caller = NULL;
server.lua_timedout = 0;
server.lua_always_replicate_commands = 0; /* Only DEBUG can change it.*/
ldbInit();
}
@@ -998,10 +996,6 @@ void scriptingInit(int setup) {
lua_pushnumber(lua,PROPAGATE_REPL);
lua_settable(lua,-3);
lua_pushstring(lua,"REPL_REPLICA");
lua_pushnumber(lua,PROPAGATE_REPL);
lua_settable(lua,-3);
lua_pushstring(lua,"REPL_ALL");
lua_pushnumber(lua,PROPAGATE_AOF|PROPAGATE_REPL);
lua_settable(lua,-3);
@@ -1221,7 +1215,7 @@ sds luaCreateFunction(client *c, lua_State *lua, robj *body) {
* EVALSHA commands as EVAL using the original script. */
int retval = dictAdd(server.lua_scripts,sha,body);
serverAssertWithInfo(c ? c : server.lua_client,NULL,retval == DICT_OK);
server.lua_scripts_mem += sdsZmallocSize(sha) + getStringObjectSdsUsedMemory(body);
server.lua_scripts_mem += sdsZmallocSize(sha) + sdsZmallocSize(body->ptr);
incrRefCount(body);
return sha;
}
@@ -1242,7 +1236,7 @@ void luaMaskCountHook(lua_State *lua, lua_Debug *ar) {
* we need to mask the client executing the script from the event loop.
* If we don't do that the client may disconnect and could no longer be
* here when the EVAL command will return. */
protectClient(server.lua_caller);
aeDeleteFileEvent(server.el, server.lua_caller->fd, AE_READABLE);
}
if (server.lua_timedout) processEventsWhileBlocked();
if (server.lua_kill) {
@@ -1256,7 +1250,6 @@ void evalGenericCommand(client *c, int evalsha) {
lua_State *lua = server.lua;
char funcname[43];
long long numkeys;
long long initial_server_dirty = server.dirty;
int delhook = 0, err;
/* When we replicate whole scripts, we want the same PRNG sequence at
@@ -1370,11 +1363,14 @@ void evalGenericCommand(client *c, int evalsha) {
if (delhook) lua_sethook(lua,NULL,0,0); /* Disable hook */
if (server.lua_timedout) {
server.lua_timedout = 0;
/* Restore the client that was protected when the script timeout
* was detected. */
unprotectClient(c);
if (server.masterhost && server.master)
queueClientForReprocessing(server.master);
/* Restore the readable handler that was unregistered when the
* script timeout was detected. */
aeCreateFileEvent(server.el,c->fd,AE_READABLE,
readQueryFromClient,c);
if (server.masterhost && server.master) {
server.master->flags |= CLIENT_UNBLOCKED;
listAddNodeTail(server.unblocked_clients,server.master);
}
}
server.lua_caller = NULL;
@@ -1438,21 +1434,9 @@ void evalGenericCommand(client *c, int evalsha) {
replicationScriptCacheAdd(c->argv[1]->ptr);
serverAssertWithInfo(c,NULL,script != NULL);
/* If the script did not produce any changes in the dataset we want
* just to replicate it as SCRIPT LOAD, otherwise we risk running
* an aborted script on slaves (that may then produce results there)
* or just running a CPU costly read-only script on the slaves. */
if (server.dirty == initial_server_dirty) {
rewriteClientCommandVector(c,3,
resetRefCount(createStringObject("SCRIPT",6)),
resetRefCount(createStringObject("LOAD",4)),
script);
} else {
rewriteClientCommandArgument(c,0,
resetRefCount(createStringObject("EVAL",4)));
rewriteClientCommandArgument(c,1,script);
}
rewriteClientCommandArgument(c,0,
resetRefCount(createStringObject("EVAL",4)));
rewriteClientCommandArgument(c,1,script);
forceCommandPropagation(c,PROPAGATE_REPL|PROPAGATE_AOF);
}
}
@@ -1487,7 +1471,7 @@ void scriptCommand(client *c) {
const char *help[] = {
"DEBUG (yes|sync|no) -- Set the debug mode for subsequent scripts executed.",
"EXISTS <sha1> [<sha1> ...] -- Return information about the existence of the scripts in the script cache.",
"FLUSH -- Flush the Lua scripts cache. Very dangerous on replicas.",
"FLUSH -- Flush the Lua scripts cache. Very dangerous on slaves.",
"KILL -- Kill the currently executing Lua script.",
"LOAD <script> -- Load a script into the scripts cache, without executing it.",
NULL
+11 -30
View File
@@ -452,15 +452,13 @@ struct redisCommand sentinelcmds[] = {
{"info",sentinelInfoCommand,-1,"",0,NULL,0,0,0,0,0},
{"role",sentinelRoleCommand,1,"l",0,NULL,0,0,0,0,0},
{"client",clientCommand,-2,"rs",0,NULL,0,0,0,0,0},
{"shutdown",shutdownCommand,-1,"",0,NULL,0,0,0,0,0},
{"auth",authCommand,2,"sltF",0,NULL,0,0,0,0,0}
{"shutdown",shutdownCommand,-1,"",0,NULL,0,0,0,0,0}
};
/* This function overwrites a few normal Redis config default with Sentinel
* specific defaults. */
void initSentinelConfig(void) {
server.port = REDIS_SENTINEL_PORT;
server.protected_mode = 0; /* Sentinel must be exposed. */
}
/* Perform the Sentinel mode initialization. */
@@ -1689,18 +1687,16 @@ char *sentinelHandleConfiguration(char **argv, int argc) {
ri = sentinelGetMasterByName(argv[1]);
if (!ri) return "No such master with specified name.";
ri->leader_epoch = strtoull(argv[2],NULL,10);
} else if ((!strcasecmp(argv[0],"known-slave") ||
!strcasecmp(argv[0],"known-replica")) && argc == 4)
{
} else if (!strcasecmp(argv[0],"known-slave") && argc == 4) {
sentinelRedisInstance *slave;
/* known-replica <name> <ip> <port> */
/* known-slave <name> <ip> <port> */
ri = sentinelGetMasterByName(argv[1]);
if (!ri) return "No such master with specified name.";
if ((slave = createSentinelRedisInstance(NULL,SRI_SLAVE,argv[2],
atoi(argv[3]), ri->quorum, ri)) == NULL)
{
return "Wrong hostname or port for replica.";
return "Wrong hostname or port for slave.";
}
} else if (!strcasecmp(argv[0],"known-sentinel") &&
(argc == 4 || argc == 5)) {
@@ -1858,7 +1854,7 @@ void rewriteConfigSentinelOption(struct rewriteConfigState *state) {
if (sentinelAddrIsEqual(slave_addr,master_addr))
slave_addr = master->addr;
line = sdscatprintf(sdsempty(),
"sentinel known-replica %s %s %d",
"sentinel known-slave %s %s %d",
master->name, slave_addr->ip, slave_addr->port);
rewriteConfigRewriteLine(state,"sentinel",line,1);
}
@@ -1943,25 +1939,12 @@ werr:
/* Send the AUTH command with the specified master password if needed.
* Note that for slaves the password set for the master is used.
*
* In case this Sentinel requires a password as well, via the "requirepass"
* configuration directive, we assume we should use the local password in
* order to authenticate when connecting with the other Sentinels as well.
* So basically all the Sentinels share the same password and use it to
* authenticate reciprocally.
*
* We don't check at all if the command was successfully transmitted
* to the instance as if it fails Sentinel will detect the instance down,
* will disconnect and reconnect the link and so forth. */
void sentinelSendAuthIfNeeded(sentinelRedisInstance *ri, redisAsyncContext *c) {
char *auth_pass = NULL;
if (ri->flags & SRI_MASTER) {
auth_pass = ri->auth_pass;
} else if (ri->flags & SRI_SLAVE) {
auth_pass = ri->master->auth_pass;
} else if (ri->flags & SRI_SENTINEL) {
if (server.requirepass) auth_pass = server.requirepass;
}
char *auth_pass = (ri->flags & SRI_MASTER) ? ri->auth_pass :
ri->master->auth_pass;
if (auth_pass) {
if (redisAsyncCommand(c, sentinelDiscardReplyCallback, ri, "%s %s",
@@ -2995,10 +2978,8 @@ void sentinelCommand(client *c) {
if ((ri = sentinelGetMasterByNameOrReplyError(c,c->argv[2]))
== NULL) return;
addReplySentinelRedisInstance(c,ri);
} else if (!strcasecmp(c->argv[1]->ptr,"slaves") ||
!strcasecmp(c->argv[1]->ptr,"replicas"))
{
/* SENTINEL REPLICAS <master-name> */
} else if (!strcasecmp(c->argv[1]->ptr,"slaves")) {
/* SENTINEL SLAVES <master-name> */
sentinelRedisInstance *ri;
if (c->argc != 3) goto numargserr;
@@ -3098,7 +3079,7 @@ void sentinelCommand(client *c) {
return;
}
if (sentinelSelectSlave(ri) == NULL) {
addReplySds(c,sdsnew("-NOGOODSLAVE No suitable replica to promote\r\n"));
addReplySds(c,sdsnew("-NOGOODSLAVE No suitable slave to promote\r\n"));
return;
}
serverLog(LL_WARNING,"Executing user requested FAILOVER of '%s'",
@@ -3280,7 +3261,7 @@ void sentinelCommand(client *c) {
sentinel.simfailure_flags |=
SENTINEL_SIMFAILURE_CRASH_AFTER_PROMOTION;
serverLog(LL_WARNING,"Failure simulation: this Sentinel "
"will crash after promoting the selected replica to master");
"will crash after promoting the selected slave to master");
} else if (!strcasecmp(c->argv[j]->ptr,"help")) {
addReplyMultiBulkLen(c,2);
addReplyBulkCString(c,"crash-after-election");
+25 -48
View File
@@ -260,8 +260,7 @@ struct redisCommand redisCommandTable[] = {
{"touch",touchCommand,-2,"rF",0,NULL,1,1,1,0,0},
{"pttl",pttlCommand,2,"rFR",0,NULL,1,1,1,0,0},
{"persist",persistCommand,2,"wF",0,NULL,1,1,1,0,0},
{"slaveof",replicaofCommand,3,"ast",0,NULL,0,0,0,0,0},
{"replicaof",replicaofCommand,3,"ast",0,NULL,0,0,0,0,0},
{"slaveof",slaveofCommand,3,"ast",0,NULL,0,0,0,0,0},
{"role",roleCommand,1,"lst",0,NULL,0,0,0,0,0},
{"debug",debugCommand,-2,"as",0,NULL,0,0,0,0,0},
{"config",configCommand,-2,"last",0,NULL,0,0,0,0,0},
@@ -307,24 +306,22 @@ struct redisCommand redisCommandTable[] = {
{"pfcount",pfcountCommand,-2,"r",0,NULL,1,-1,1,0,0},
{"pfmerge",pfmergeCommand,-2,"wm",0,NULL,1,-1,1,0,0},
{"pfdebug",pfdebugCommand,-3,"w",0,NULL,0,0,0,0,0},
{"xadd",xaddCommand,-5,"wmFR",0,NULL,1,1,1,0,0},
{"xadd",xaddCommand,-5,"wmF",0,NULL,1,1,1,0,0},
{"xrange",xrangeCommand,-4,"r",0,NULL,1,1,1,0,0},
{"xrevrange",xrevrangeCommand,-4,"r",0,NULL,1,1,1,0,0},
{"xlen",xlenCommand,2,"rF",0,NULL,1,1,1,0,0},
{"xread",xreadCommand,-4,"rs",0,xreadGetKeys,1,1,1,0,0},
{"xreadgroup",xreadCommand,-7,"ws",0,xreadGetKeys,1,1,1,0,0},
{"xgroup",xgroupCommand,-2,"wm",0,NULL,2,2,1,0,0},
{"xsetid",xsetidCommand,3,"wmF",0,NULL,1,1,1,0,0},
{"xack",xackCommand,-4,"wF",0,NULL,1,1,1,0,0},
{"xpending",xpendingCommand,-3,"rR",0,NULL,1,1,1,0,0},
{"xclaim",xclaimCommand,-6,"wRF",0,NULL,1,1,1,0,0},
{"xinfo",xinfoCommand,-2,"rR",0,NULL,2,2,1,0,0},
{"xpending",xpendingCommand,-3,"r",0,NULL,1,1,1,0,0},
{"xclaim",xclaimCommand,-6,"wF",0,NULL,1,1,1,0,0},
{"xinfo",xinfoCommand,-2,"r",0,NULL,2,2,1,0,0},
{"xdel",xdelCommand,-3,"wF",0,NULL,1,1,1,0,0},
{"xtrim",xtrimCommand,-2,"wFR",0,NULL,1,1,1,0,0},
{"xtrim",xtrimCommand,-2,"wF",0,NULL,1,1,1,0,0},
{"post",securityWarningCommand,-1,"lt",0,NULL,0,0,0,0,0},
{"host:",securityWarningCommand,-1,"lt",0,NULL,0,0,0,0,0},
{"latency",latencyCommand,-2,"aslt",0,NULL,0,0,0,0,0},
{"lolwut",lolwutCommand,-1,"r",0,NULL,0,0,0,0,0}
{"latency",latencyCommand,-2,"aslt",0,NULL,0,0,0,0,0}
};
/*============================ Utility functions ============================ */
@@ -1195,7 +1192,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
if (!server.sentinel_mode) {
run_with_period(5000) {
serverLog(LL_VERBOSE,
"%lu clients connected (%lu replicas), %zu bytes in use",
"%lu clients connected (%lu slaves), %zu bytes in use",
listLength(server.clients)-listLength(server.slaves),
listLength(server.slaves),
zmalloc_used_memory());
@@ -1323,7 +1320,9 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
}
/* Run the Sentinel timer if we are in sentinel mode. */
if (server.sentinel_mode) sentinelTimer();
run_with_period(100) {
if (server.sentinel_mode) sentinelTimer();
}
/* Cleanup expired MIGRATE cached sockets. */
run_with_period(1000) {
@@ -1452,11 +1451,11 @@ void createSharedObjects(void) {
shared.slowscripterr = createObject(OBJ_STRING,sdsnew(
"-BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.\r\n"));
shared.masterdownerr = createObject(OBJ_STRING,sdsnew(
"-MASTERDOWN Link with MASTER is down and replica-serve-stale-data is set to 'no'.\r\n"));
"-MASTERDOWN Link with MASTER is down and slave-serve-stale-data is set to 'no'.\r\n"));
shared.bgsaveerr = createObject(OBJ_STRING,sdsnew(
"-MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.\r\n"));
shared.roslaveerr = createObject(OBJ_STRING,sdsnew(
"-READONLY You can't write against a read only replica.\r\n"));
"-READONLY You can't write against a read only slave.\r\n"));
shared.noautherr = createObject(OBJ_STRING,sdsnew(
"-NOAUTH Authentication required.\r\n"));
shared.oomerr = createObject(OBJ_STRING,sdsnew(
@@ -1464,7 +1463,7 @@ void createSharedObjects(void) {
shared.execaborterr = createObject(OBJ_STRING,sdsnew(
"-EXECABORT Transaction discarded because of previous errors.\r\n"));
shared.noreplicaserr = createObject(OBJ_STRING,sdsnew(
"-NOREPLICAS Not enough good replicas to write.\r\n"));
"-NOREPLICAS Not enough good slaves to write.\r\n"));
shared.busykeyerr = createObject(OBJ_STRING,sdsnew(
"-BUSYKEY Target key name already exists.\r\n"));
shared.space = createObject(OBJ_STRING,sdsnew(" "));
@@ -1526,10 +1525,10 @@ void initServerConfig(void) {
server.runid[CONFIG_RUN_ID_SIZE] = '\0';
changeReplicationId();
clearReplicationId2();
server.timezone = getTimeZone(); /* Initialized by tzset(). */
server.timezone = timezone; /* Initialized by tzset(). */
server.configfile = NULL;
server.executable = NULL;
server.hz = server.config_hz = CONFIG_DEFAULT_HZ;
server.config_hz = CONFIG_DEFAULT_HZ;
server.dynamic_hz = CONFIG_DEFAULT_DYNAMIC_HZ;
server.arch_bits = (sizeof(long) == 8) ? 64 : 32;
server.port = CONFIG_DEFAULT_SERVER_PORT;
@@ -1622,7 +1621,6 @@ void initServerConfig(void) {
server.cluster_announce_ip = CONFIG_DEFAULT_CLUSTER_ANNOUNCE_IP;
server.cluster_announce_port = CONFIG_DEFAULT_CLUSTER_ANNOUNCE_PORT;
server.cluster_announce_bus_port = CONFIG_DEFAULT_CLUSTER_ANNOUNCE_BUS_PORT;
server.cluster_module_flags = CLUSTER_MODULE_FLAG_NONE;
server.migrate_cached_sockets = dictCreate(&migrateCacheDictType,NULL);
server.next_client_id = 1; /* Client IDs, start from 1 .*/
server.loading_process_events_interval_bytes = (1024*1024*2);
@@ -1703,7 +1701,6 @@ void initServerConfig(void) {
server.expireCommand = lookupCommandByCString("expire");
server.pexpireCommand = lookupCommandByCString("pexpire");
server.xclaimCommand = lookupCommandByCString("xclaim");
server.xgroupCommand = lookupCommandByCString("xgroup");
/* Slow log */
server.slowlog_log_slower_than = CONFIG_DEFAULT_SLOWLOG_LOG_SLOWER_THAN;
@@ -1718,12 +1715,6 @@ void initServerConfig(void) {
server.assert_line = 0;
server.bug_report_start = 0;
server.watchdog_period = 0;
/* By default we want scripts to be always replicated by effects
* (single commands executed by the script), and not by sending the
* script to the slave / AOF. This is the new way starting from
* Redis 5. However it is possible to revert it via redis.conf. */
server.lua_always_replicate_commands = 1;
}
extern char **environ;
@@ -2410,7 +2401,6 @@ void preventCommandReplication(client *c) {
void call(client *c, int flags) {
long long dirty, start, duration;
int client_old_flags = c->flags;
struct redisCommand *real_cmd = c->cmd;
/* Sent the command to clients in MONITOR mode, only if the commands are
* not generated from reading an AOF. */
@@ -2459,11 +2449,8 @@ void call(client *c, int flags) {
slowlogPushEntryIfNeeded(c,c->argv,c->argc,duration);
}
if (flags & CMD_CALL_STATS) {
/* use the real command that was executed (cmd and lastamc) may be
* different, in case of MULTI-EXEC or re-written commands such as
* EXPIRE, GEOADD, etc. */
real_cmd->microseconds += duration;
real_cmd->calls++;
c->lastcmd->microseconds += duration;
c->lastcmd->calls++;
}
/* Propagate the command into the AOF and replication link */
@@ -2606,24 +2593,16 @@ int processCommand(client *c) {
*
* First we try to free some memory if possible (if there are volatile
* keys in the dataset). If there are not the only thing we can do
* is returning an error.
*
* Note that we do not want to reclaim memory if we are here re-entering
* the event loop since there is a busy Lua script running in timeout
* condition, to avoid mixing the propagation of scripts with the propagation
* of DELs due to eviction. */
if (server.maxmemory && !server.lua_timedout) {
* is returning an error. */
if (server.maxmemory) {
int out_of_memory = freeMemoryIfNeeded() == C_ERR;
/* freeMemoryIfNeeded may flush slave output buffers. This may result
* into a slave, that may be the active client, to be freed. */
if (server.current_client == NULL) return C_ERR;
/* It was impossible to free enough memory, and the command the client
* is trying to execute is denied during OOM conditions or the client
* is in MULTI/EXEC context? Error. */
if (out_of_memory &&
(c->cmd->flags & CMD_DENYOOM ||
(c->flags & CLIENT_MULTI && c->cmd->proc != execCommand))) {
* is trying to execute is denied during OOM conditions? Error. */
if ((c->cmd->flags & CMD_DENYOOM) && out_of_memory) {
flagTransaction(c);
addReply(c, shared.oomerr);
return C_OK;
@@ -3209,7 +3188,7 @@ sds genRedisInfoString(char *section) {
bytesToHuman(peak_hmem,server.stat_peak_memory);
bytesToHuman(total_system_hmem,total_system_mem);
bytesToHuman(used_memory_lua_hmem,memory_lua);
bytesToHuman(used_memory_scripts_hmem,mh->lua_caches);
bytesToHuman(used_memory_scripts_hmem,server.lua_scripts_mem);
bytesToHuman(used_memory_rss_hmem,server.cron_malloc_stats.process_rss);
bytesToHuman(maxmemory_hmem,server.maxmemory);
@@ -3274,7 +3253,7 @@ sds genRedisInfoString(char *section) {
total_system_hmem,
memory_lua,
used_memory_lua_hmem,
(long long) mh->lua_caches,
server.lua_scripts_mem,
used_memory_scripts_hmem,
dictSize(server.lua_scripts),
server.maxmemory,
@@ -3737,7 +3716,7 @@ void usage(void) {
fprintf(stderr," ./redis-server (run the server with default conf)\n");
fprintf(stderr," ./redis-server /etc/redis/6379.conf\n");
fprintf(stderr," ./redis-server --port 7777\n");
fprintf(stderr," ./redis-server --port 7777 --replicaof 127.0.0.1 8888\n");
fprintf(stderr," ./redis-server --port 7777 --slaveof 127.0.0.1 8888\n");
fprintf(stderr," ./redis-server /etc/myredis.conf --loglevel verbose\n\n");
fprintf(stderr,"Sentinel mode:\n");
fprintf(stderr," ./redis-server /etc/sentinel.conf --sentinel\n");
@@ -4024,8 +4003,6 @@ int main(int argc, char **argv) {
return endianconvTest(argc, argv);
} else if (!strcasecmp(argv[2], "crc64")) {
return crc64Test(argc, argv);
} else if (!strcasecmp(argv[2], "zmalloc")) {
return zmalloc_test(argc, argv);
}
return -1; /* test not found */
+2 -14
View File
@@ -256,7 +256,6 @@ typedef long long mstime_t; /* millisecond time type. */
#define CLIENT_LUA_DEBUG (1<<25) /* Run EVAL in debug mode. */
#define CLIENT_LUA_DEBUG_SYNC (1<<26) /* EVAL debugging without fork() */
#define CLIENT_MODULE (1<<27) /* Non connected client used by some module. */
#define CLIENT_PROTECTED (1<<28) /* Client should not be freed for now. */
/* Client block type (btype field in client structure)
* if CLIENT_BLOCKED flag is set. */
@@ -990,8 +989,7 @@ struct redisServer {
struct redisCommand *delCommand, *multiCommand, *lpushCommand,
*lpopCommand, *rpopCommand, *zpopminCommand,
*zpopmaxCommand, *sremCommand, *execCommand,
*expireCommand, *pexpireCommand, *xclaimCommand,
*xgroupCommand;
*expireCommand, *pexpireCommand, *xclaimCommand;
/* Fields used only for stats */
time_t stat_starttime; /* Server start time */
long long stat_numcommands; /* Number of processed commands */
@@ -1234,10 +1232,6 @@ struct redisServer {
char *cluster_announce_ip; /* IP address to announce on cluster bus. */
int cluster_announce_port; /* base port to announce on cluster bus. */
int cluster_announce_bus_port; /* bus port to announce on cluster bus. */
int cluster_module_flags; /* Set of flags that Redis modules are able
to set in order to suppress certain
native Redis Cluster features. Check the
REDISMODULE_CLUSTER_FLAG_*. */
/* Scripting */
lua_State *lua; /* The Lua interpreter. We use just one for all clients */
client *lua_client; /* The "fake client" to query Redis from Lua */
@@ -1424,7 +1418,6 @@ void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask);
void *addDeferredMultiBulkLength(client *c);
void setDeferredMultiBulkLength(client *c, void *node, long length);
void processInputBuffer(client *c);
void processInputBufferAndReplicate(client *c);
void acceptHandler(aeEventLoop *el, int fd, void *privdata, int mask);
void acceptTcpHandler(aeEventLoop *el, int fd, void *privdata, int mask);
void acceptUnixHandler(aeEventLoop *el, int fd, void *privdata, int mask);
@@ -1475,8 +1468,6 @@ int clientHasPendingReplies(client *c);
void unlinkClient(client *c);
int writeToClient(int fd, client *c, int handler_installed);
void linkClient(client *c);
void protectClient(client *c);
void unprotectClient(client *c);
#ifdef __GNUC__
void addReplyErrorFormat(client *c, const char *fmt, ...)
@@ -1892,7 +1883,6 @@ sds luaCreateFunction(client *c, lua_State *lua, robj *body);
void processUnblockedClients(void);
void blockClient(client *c, int btype);
void unblockClient(client *c);
void queueClientForReprocessing(client *c);
void replyToBlockedClientTimedOut(client *c);
int getTimeoutFromObjectOrReply(client *c, robj *object, mstime_t *timeout, int unit);
void disconnectAllBlockedClients(void);
@@ -2006,7 +1996,7 @@ void ttlCommand(client *c);
void touchCommand(client *c);
void pttlCommand(client *c);
void persistCommand(client *c);
void replicaofCommand(client *c);
void slaveofCommand(client *c);
void roleCommand(client *c);
void debugCommand(client *c);
void msetCommand(client *c);
@@ -2111,14 +2101,12 @@ void xrevrangeCommand(client *c);
void xlenCommand(client *c);
void xreadCommand(client *c);
void xgroupCommand(client *c);
void xsetidCommand(client *c);
void xackCommand(client *c);
void xpendingCommand(client *c);
void xclaimCommand(client *c);
void xinfoCommand(client *c);
void xdelCommand(client *c);
void xtrimCommand(client *c);
void lolwutCommand(client *c);
#if defined(__GNUC__)
void *calloc(size_t count, size_t size) __attribute__ ((deprecated));
+1 -1
View File
@@ -596,7 +596,7 @@ void rpoplpushCommand(client *c) {
signalModifiedKey(c->db,touchedkey);
decrRefCount(touchedkey);
server.dirty++;
if (c->cmd->proc == brpoplpushCommand) {
if (c->lastcmd->proc == brpoplpushCommand) {
rewriteClientCommandVector(c,3,shared.rpoplpush,c->argv[1],c->argv[2]);
}
}
+65 -233
View File
@@ -349,8 +349,7 @@ int streamAppendItem(stream *s, robj **argv, int64_t numfields, streamID *added_
lp = lpAppendInteger(lp,lp_count);
/* Insert back into the tree in order to update the listpack pointer. */
if (ri.data != lp)
raxInsert(s->rax,(unsigned char*)&rax_key,sizeof(rax_key),lp,NULL);
raxInsert(s->rax,(unsigned char*)&rax_key,sizeof(rax_key),lp,NULL);
s->length++;
s->last_id = id;
if (added_id) *added_id = id;
@@ -725,10 +724,6 @@ void streamIteratorRemoveEntry(streamIterator *si, streamID *current) {
p = lpNext(lp,p); /* Seek deleted field. */
aux = lpGetInteger(p);
lp = lpReplaceInteger(lp,&p,aux+1);
/* Update the listpack with the new pointer. */
if (si->lp != lp)
raxInsert(si->stream->rax,si->ri.key,si->ri.key_len,lp,NULL);
}
/* Update the number of entries counter. */
@@ -777,8 +772,8 @@ int streamDeleteItem(stream *s, streamID *id) {
* in the standard <ms>-<seq> format, using the simple string protocol
* of REPL. */
void addReplyStreamID(client *c, streamID *id) {
sds replyid = sdscatfmt(sdsempty(),"%U-%U",id->ms,id->seq);
addReplyBulkSds(c,replyid);
sds replyid = sdscatfmt(sdsempty(),"+%U-%U\r\n",id->ms,id->seq);
addReplySds(c,replyid);
}
/* Similar to the above function, but just creates an object, usually useful
@@ -791,18 +786,18 @@ robj *createObjectFromStreamID(streamID *id) {
/* As a result of an explicit XCLAIM or XREADGROUP command, new entries
* are created in the pending list of the stream and consumers. We need
* to propagate this changes in the form of XCLAIM commands. */
void streamPropagateXCLAIM(client *c, robj *key, streamCG *group, robj *groupname, robj *id, streamNACK *nack) {
void streamPropagateXCLAIM(client *c, robj *key, robj *group, robj *id, streamNACK *nack) {
/* We need to generate an XCLAIM that will work in a idempotent fashion:
*
* XCLAIM <key> <group> <consumer> 0 <id> TIME <milliseconds-unix-time>
* RETRYCOUNT <count> FORCE JUSTID LASTID <id>.
* RETRYCOUNT <count> FORCE JUSTID.
*
* Note that JUSTID is useful in order to avoid that XCLAIM will do
* useless work in the slave side, trying to fetch the stream item. */
robj *argv[14];
robj *argv[12];
argv[0] = createStringObject("XCLAIM",6);
argv[1] = key;
argv[2] = groupname;
argv[2] = group;
argv[3] = createStringObject(nack->consumer->name,sdslen(nack->consumer->name));
argv[4] = createStringObjectFromLongLong(0);
argv[5] = id;
@@ -812,9 +807,7 @@ void streamPropagateXCLAIM(client *c, robj *key, streamCG *group, robj *groupnam
argv[9] = createStringObjectFromLongLong(nack->delivery_count);
argv[10] = createStringObject("FORCE",5);
argv[11] = createStringObject("JUSTID",6);
argv[12] = createStringObject("LASTID",6);
argv[13] = createObjectFromStreamID(&group->last_id);
propagate(server.xclaimCommand,c->db->id,argv,14,PROPAGATE_AOF|PROPAGATE_REPL);
propagate(server.xclaimCommand,c->db->id,argv,12,PROPAGATE_AOF|PROPAGATE_REPL);
decrRefCount(argv[0]);
decrRefCount(argv[3]);
decrRefCount(argv[4]);
@@ -824,38 +817,13 @@ void streamPropagateXCLAIM(client *c, robj *key, streamCG *group, robj *groupnam
decrRefCount(argv[9]);
decrRefCount(argv[10]);
decrRefCount(argv[11]);
decrRefCount(argv[12]);
decrRefCount(argv[13]);
}
/* We need this when we want to propoagate the new last-id of a consumer group
* that was consumed by XREADGROUP with the NOACK option: in that case we can't
* propagate the last ID just using the XCLAIM LASTID option, so we emit
*
* XGROUP SETID <key> <groupname> <id>
*/
void streamPropagateGroupID(client *c, robj *key, streamCG *group, robj *groupname) {
robj *argv[5];
argv[0] = createStringObject("XGROUP",6);
argv[1] = createStringObject("SETID",5);
argv[2] = key;
argv[3] = groupname;
argv[4] = createObjectFromStreamID(&group->last_id);
propagate(server.xgroupCommand,c->db->id,argv,5,PROPAGATE_AOF|PROPAGATE_REPL);
decrRefCount(argv[0]);
decrRefCount(argv[1]);
decrRefCount(argv[4]);
}
/* Send the stream items in the specified range to the client 'c'. The range
* the client will receive is between start and end inclusive, if 'count' is
* non zero, no more than 'count' elements are sent.
*
* The 'end' pointer can be NULL to mean that we want all the elements from
* 'start' till the end of the stream. If 'rev' is non zero, elements are
* produced in reversed order from end to start.
*
* The function returns the number of entries emitted.
/* Send the specified range to the client 'c'. The range the client will
* receive is between start and end inclusive, if 'count' is non zero, no more
* than 'count' elemnets are sent. The 'end' pointer can be NULL to mean that
* we want all the elements from 'start' till the end of the stream. If 'rev'
* is non zero, elements are produced in reversed order from end to start.
*
* If group and consumer are not NULL, the function performs additional work:
* 1. It updates the last delivered ID in the group in case we are
@@ -900,7 +868,6 @@ size_t streamReplyWithRange(client *c, stream *s, streamID *start, streamID *end
streamIterator si;
int64_t numfields;
streamID id;
int propagate_last_id = 0;
/* If a group was passed, we check if the request is about messages
* never delivered so far (normally this happens when ">" ID is passed).
@@ -920,10 +887,8 @@ size_t streamReplyWithRange(client *c, stream *s, streamID *start, streamID *end
streamIteratorStart(&si,s,start,end,rev);
while(streamIteratorGetID(&si,&id,&numfields)) {
/* Update the group last_id if needed. */
if (group && streamCompareID(&id,&group->last_id) > 0) {
if (group && streamCompareID(&id,&group->last_id) > 0)
group->last_id = id;
propagate_last_id = 1;
}
/* Emit a two elements array for each item. The first is
* the ID, the second is an array of field-value pairs. */
@@ -983,12 +948,9 @@ size_t streamReplyWithRange(client *c, stream *s, streamID *start, streamID *end
/* Propagate as XCLAIM. */
if (spi) {
robj *idarg = createObjectFromStreamID(&id);
streamPropagateXCLAIM(c,spi->keyname,group,spi->groupname,idarg,nack);
streamPropagateXCLAIM(c,spi->keyname,spi->groupname,idarg,nack);
decrRefCount(idarg);
}
} else {
if (propagate_last_id)
streamPropagateGroupID(c,spi->keyname,group,spi->groupname);
}
arraylen++;
@@ -1152,20 +1114,8 @@ int streamParseStrictIDOrReply(client *c, robj *o, streamID *id, uint64_t missin
return streamGenericParseIDOrReply(c,o,id,missing_seq,1);
}
/* We propagate MAXLEN ~ <count> as MAXLEN = <resulting-len-of-stream>
* otherwise trimming is no longer determinsitic on replicas / AOF. */
void streamRewriteApproxMaxlen(client *c, stream *s, int maxlen_arg_idx) {
robj *maxlen_obj = createStringObjectFromLongLong(s->length);
robj *equal_obj = createStringObject("=",1);
rewriteClientCommandArgument(c,maxlen_arg_idx,maxlen_obj);
rewriteClientCommandArgument(c,maxlen_arg_idx-1,equal_obj);
decrRefCount(equal_obj);
decrRefCount(maxlen_obj);
}
/* XADD key [MAXLEN [~|=] <count>] <ID or *> [field value] [field value] ... */
/* XADD key [MAXLEN <count>] <ID or *> [field value] [field value] ... */
void xaddCommand(client *c) {
streamID id;
int id_given = 0; /* Was an ID different than "*" specified? */
@@ -1185,14 +1135,11 @@ void xaddCommand(client *c) {
* creation. */
break;
} else if (!strcasecmp(opt,"maxlen") && moreargs) {
approx_maxlen = 0;
char *next = c->argv[i+1]->ptr;
/* Check for the form MAXLEN ~ <count>. */
if (moreargs >= 2 && next[0] == '~' && next[1] == '\0') {
approx_maxlen = 1;
i++;
} else if (moreargs >= 2 && next[0] == '=' && next[1] == '\0') {
i++;
}
if (getLongLongFromObjectOrReply(c,c->argv[i+1],&maxlen,NULL)
!= C_OK) return;
@@ -1239,12 +1186,18 @@ void xaddCommand(client *c) {
notifyKeyspaceEvent(NOTIFY_STREAM,"xadd",c->argv[1],c->db->id);
server.dirty++;
/* Remove older elements if MAXLEN was specified. */
if (maxlen >= 0) {
/* Notify xtrim event if needed. */
if (streamTrimByLength(s,maxlen,approx_maxlen)) {
if (!streamTrimByLength(s,maxlen,approx_maxlen)) {
/* If no trimming was performed, for instance because approximated
* trimming length was specified, rewrite the MAXLEN argument
* as zero, so that the command is propagated without trimming. */
robj *zeroobj = createStringObjectFromLongLong(0);
rewriteClientCommandArgument(c,maxlen_arg_idx,zeroobj);
decrRefCount(zeroobj);
} else {
notifyKeyspaceEvent(NOTIFY_STREAM,"xtrim",c->argv[1],c->db->id);
}
if (approx_maxlen) streamRewriteApproxMaxlen(c,s,maxlen_arg_idx);
}
/* Let's rewrite the ID argument with the one actually generated for
@@ -1264,7 +1217,7 @@ void xrangeGenericCommand(client *c, int rev) {
robj *o;
stream *s;
streamID startid, endid;
long long count = -1;
long long count = 0;
robj *startarg = rev ? c->argv[3] : c->argv[2];
robj *endarg = rev ? c->argv[2] : c->argv[3];
@@ -1291,13 +1244,7 @@ void xrangeGenericCommand(client *c, int rev) {
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.emptymultibulk)) == NULL
|| checkType(c,o,OBJ_STREAM)) return;
s = o->ptr;
if (count == 0) {
addReply(c,shared.nullmultibulk);
} else {
if (count == -1) count = 0;
streamReplyWithRange(c,s,&startid,&endid,count,rev,NULL,NULL,0,NULL);
}
streamReplyWithRange(c,s,&startid,&endid,count,rev,NULL,NULL,0,NULL);
}
/* XRANGE key start end [COUNT <n>] */
@@ -1489,7 +1436,7 @@ void xreadCommand(client *c) {
* synchronously in case the group top item delivered is smaller
* than what the stream has inside. */
streamID *last = &groups[i]->last_id;
if (s->length && (streamCompareID(&s->last_id, last) > 0)) {
if (streamCompareID(&s->last_id, last) > 0) {
serve_synchronously = 1;
*gt = *last;
}
@@ -1497,7 +1444,7 @@ void xreadCommand(client *c) {
} else {
/* For consumers without a group, we serve synchronously if we can
* actually provide at least one item from the stream. */
if (s->length && (streamCompareID(&s->last_id, gt) > 0)) {
if (streamCompareID(&s->last_id, gt) > 0) {
serve_synchronously = 1;
}
}
@@ -1693,14 +1640,13 @@ uint64_t streamDelConsumer(streamCG *cg, sds name) {
* Consumer groups commands
* ----------------------------------------------------------------------- */
/* XGROUP CREATE <key> <groupname> <id or $> [MKSTREAM]
* XGROUP SETID <key> <groupname> <id or $>
/* XGROUP CREATE <key> <groupname> <id or $>
* XGROUP SETID <key> <id or $>
* XGROUP DESTROY <key> <groupname>
* XGROUP DELCONSUMER <key> <groupname> <consumername> */
void xgroupCommand(client *c) {
const char *help[] = {
"CREATE <key> <groupname> <id or $> [opt] -- Create a new consumer group.",
" option MKSTREAM: create the empty stream if it does not exist.",
"CREATE <key> <groupname> <id or $> -- Create a new consumer group.",
"SETID <key> <groupname> <id or $> -- Set the current group ID.",
"DESTROY <key> <groupname> -- Remove the specified group.",
"DELCONSUMER <key> <groupname> <consumer> -- Remove the specified consumer.",
@@ -1711,39 +1657,13 @@ NULL
sds grpname = NULL;
streamCG *cg = NULL;
char *opt = c->argv[1]->ptr; /* Subcommand name. */
int mkstream = 0;
robj *o;
/* CREATE has an MKSTREAM option that creates the stream if it
* does not exist. */
if (c->argc == 6 && !strcasecmp(opt,"CREATE")) {
if (strcasecmp(c->argv[5]->ptr,"MKSTREAM")) {
addReplySubcommandSyntaxError(c);
return;
}
mkstream = 1;
grpname = c->argv[3]->ptr;
}
/* Everything but the "HELP" option requires a key and group name. */
/* Lookup the key now, this is common for all the subcommands but HELP. */
if (c->argc >= 4) {
o = lookupKeyWrite(c->db,c->argv[2]);
if (o) s = o->ptr;
robj *o = lookupKeyWriteOrReply(c,c->argv[2],shared.nokeyerr);
if (o == NULL || checkType(c,o,OBJ_STREAM)) return;
s = o->ptr;
grpname = c->argv[3]->ptr;
}
/* Check for missing key/group. */
if (c->argc >= 4 && !mkstream) {
/* At this point key must exist, or there is an error. */
if (o == NULL) {
addReplyError(c,
"The XGROUP subcommand requires the key to exist. "
"Note that for CREATE you may want to use the MKSTREAM "
"option to create an empty stream automatically.");
return;
}
if (checkType(c,o,OBJ_STREAM)) return;
/* Certain subcommands require the group to exist. */
if ((cg = streamLookupCG(s,grpname)) == NULL &&
@@ -1758,26 +1678,13 @@ NULL
}
/* Dispatch the different subcommands. */
if (!strcasecmp(opt,"CREATE") && (c->argc == 5 || c->argc == 6)) {
if (!strcasecmp(opt,"CREATE") && c->argc == 5) {
streamID id;
if (!strcmp(c->argv[4]->ptr,"$")) {
if (s) {
id = s->last_id;
} else {
id.ms = 0;
id.seq = 0;
}
id = s->last_id;
} else if (streamParseStrictIDOrReply(c,c->argv[4],&id,0) != C_OK) {
return;
}
/* Handle the MKSTREAM option now that the command can no longer fail. */
if (s == NULL && mkstream) {
o = createStreamObject();
dbAdd(c->db,c->argv[2],o);
s = o->ptr;
}
streamCG *cg = streamCreateCG(s,grpname,sdslen(grpname),&id);
if (cg) {
addReply(c,shared.ok);
@@ -1825,40 +1732,6 @@ NULL
}
}
/* XSETID <stream> <groupname> <id>
*
* Set the internal "last ID" of a stream. */
void xsetidCommand(client *c) {
robj *o = lookupKeyWriteOrReply(c,c->argv[1],shared.nokeyerr);
if (o == NULL || checkType(c,o,OBJ_STREAM)) return;
stream *s = o->ptr;
streamID id;
if (streamParseStrictIDOrReply(c,c->argv[2],&id,0) != C_OK) return;
/* If the stream has at least one item, we want to check that the user
* is setting a last ID that is equal or greater than the current top
* item, otherwise the fundamental ID monotonicity assumption is violated. */
if (s->length > 0) {
streamID maxid;
streamIterator si;
streamIteratorStart(&si,s,NULL,NULL,1);
int64_t numfields;
streamIteratorGetID(&si,&maxid,&numfields);
streamIteratorStop(&si);
if (streamCompareID(&id,&maxid) < 0) {
addReplyError(c,"The ID specified in XSETID is smaller than the "
"target stream top item");
return;
}
}
s->last_id = id;
addReply(c,shared.ok);
server.dirty++;
notifyKeyspaceEvent(NOTIFY_STREAM,"xsetid",c->argv[1],c->db->id);
}
/* XACK <key> <group> <id> <id> ... <id>
*
* Acknowledge a message as processed. In practical terms we just check the
@@ -1904,7 +1777,7 @@ void xackCommand(client *c) {
addReplyLongLong(c,acknowledged);
}
/* XPENDING <key> <group> [<start> <stop> <count> [<consumer>]]
/* XPENDING <key> <group> [<start> <stop> <count>] [<consumer>]
*
* If start and stop are omitted, the command just outputs information about
* the amount of pending messages for the key/group pair, together with
@@ -1933,7 +1806,6 @@ void xpendingCommand(client *c) {
if (c->argc >= 6) {
if (getLongLongFromObjectOrReply(c,c->argv[5],&count,NULL) == C_ERR)
return;
if (count < 0) count = 0;
if (streamParseIDOrReply(c,c->argv[3],&startid,0) == C_ERR)
return;
if (streamParseIDOrReply(c,c->argv[4],&endid,UINT64_MAX) == C_ERR)
@@ -2107,14 +1979,6 @@ void xpendingCommand(client *c) {
* Return just an array of IDs of messages successfully claimed,
* without returning the actual message.
*
* 6. LASTID <id>:
* Update the consumer group last ID with the specified ID if the
* current last ID is smaller than the provided one.
* This is used for replication / AOF, so that when we read from a
* consumer group, the XCLAIM that gets propagated to give ownership
* to the consumer, is also used in order to update the group current
* ID.
*
* The command returns an array of messages that the user
* successfully claimed, so that the caller is able to understand
* what messages it is now in charge of. */
@@ -2159,9 +2023,7 @@ void xclaimCommand(client *c) {
/* If we stopped because some IDs cannot be parsed, perhaps they
* are trailing options. */
mstime_t now = mstime();
streamID last_id = {0,0};
int propagate_last_id = 0;
time_t now = mstime();
for (; j < c->argc; j++) {
int moreargs = (c->argc-1) - j; /* Number of additional arguments. */
char *opt = c->argv[j]->ptr;
@@ -2185,28 +2047,20 @@ void xclaimCommand(client *c) {
if (getLongLongFromObjectOrReply(c,c->argv[j],&retrycount,
"Invalid RETRYCOUNT option argument for XCLAIM")
!= C_OK) return;
} else if (!strcasecmp(opt,"LASTID") && moreargs) {
j++;
if (streamParseStrictIDOrReply(c,c->argv[j],&last_id,0) != C_OK) return;
} else {
addReplyErrorFormat(c,"Unrecognized XCLAIM option '%s'",opt);
return;
}
}
if (streamCompareID(&last_id,&group->last_id) > 0) {
group->last_id = last_id;
propagate_last_id = 1;
}
if (deliverytime != -1) {
/* If a delivery time was passed, either with IDLE or TIME, we
* do some sanity check on it, and set the deliverytime to now
* (which is a sane choice usually) if the value is bogus.
* To raise an error here is not wise because clients may compute
* the idle time doing some math starting from their local time,
* the idle time doing some math startin from their local time,
* and this is not a good excuse to fail in case, for instance,
* the computer time is a bit in the future from our POV. */
* the computed time is a bit in the future from our POV. */
if (deliverytime < 0 || deliverytime > now) deliverytime = now;
} else {
/* If no IDLE/TIME option was passed, we want the last delivery
@@ -2222,8 +2076,7 @@ void xclaimCommand(client *c) {
for (int j = 5; j <= last_id_arg; j++) {
streamID id;
unsigned char buf[sizeof(streamID)];
if (streamParseStrictIDOrReply(c,c->argv[j],&id,0) != C_OK)
serverPanic("StreamID invalid after check. Should not be possible.");
if (streamParseStrictIDOrReply(c,c->argv[j],&id,0) != C_OK) return;
streamEncodeID(buf,&id);
/* Lookup the ID in the group PEL. */
@@ -2253,12 +2106,8 @@ void xclaimCommand(client *c) {
if (nack != raxNotFound) {
/* We need to check if the minimum idle time requested
* by the caller is satisfied by this entry.
*
* Note that the nack could be created by FORCE, in this
* case there was no pre-existing entry and minidle should
* be ignored, but in that case nick->consumer is NULL. */
if (nack->consumer && minidle) {
* by the caller is satisfied by this entry. */
if (minidle) {
mstime_t this_idle = now - nack->delivery_time;
if (this_idle < minidle) continue;
}
@@ -2278,22 +2127,16 @@ void xclaimCommand(client *c) {
if (justid) {
addReplyStreamID(c,&id);
} else {
size_t emitted = streamReplyWithRange(c,o->ptr,&id,&id,1,0,
NULL,NULL,STREAM_RWR_RAWENTRIES,NULL);
if (!emitted) addReply(c,shared.nullbulk);
streamReplyWithRange(c,o->ptr,&id,NULL,1,0,NULL,NULL,
STREAM_RWR_RAWENTRIES,NULL);
}
arraylen++;
/* Propagate this change. */
streamPropagateXCLAIM(c,c->argv[1],group,c->argv[2],c->argv[j],nack);
propagate_last_id = 0; /* Will be propagated by XCLAIM itself. */
streamPropagateXCLAIM(c,c->argv[1],c->argv[3],c->argv[j],nack);
server.dirty++;
}
}
if (propagate_last_id) {
streamPropagateGroupID(c,c->argv[1],group,c->argv[2]);
server.dirty++;
}
setDeferredMultiBulkLength(c,arraylenptr,arraylen);
preventCommandPropagation(c);
}
@@ -2339,7 +2182,7 @@ void xdelCommand(client *c) {
*
* List of options:
*
* MAXLEN [~|=] <count> -- Trim so that the stream will be capped at
* MAXLEN [~] <count> -- Trim so that the stream will be capped at
* the specified length. Use ~ before the
* count in order to demand approximated trimming
* (like XADD MAXLEN option).
@@ -2358,10 +2201,9 @@ void xtrimCommand(client *c) {
/* Argument parsing. */
int trim_strategy = TRIM_STRATEGY_NONE;
long long maxlen = -1; /* If left to -1 no trimming is performed. */
long long maxlen = 0; /* 0 means no maximum length. */
int approx_maxlen = 0; /* If 1 only delete whole radix tree nodes, so
the maxium length is not applied verbatim. */
int maxlen_arg_idx = 0; /* Index of the count in MAXLEN, for rewriting. */
/* Parse options. */
int i = 2; /* Start of options. */
@@ -2369,25 +2211,16 @@ void xtrimCommand(client *c) {
int moreargs = (c->argc-1) - i; /* Number of additional arguments. */
char *opt = c->argv[i]->ptr;
if (!strcasecmp(opt,"maxlen") && moreargs) {
approx_maxlen = 0;
trim_strategy = TRIM_STRATEGY_MAXLEN;
char *next = c->argv[i+1]->ptr;
/* Check for the form MAXLEN ~ <count>. */
if (moreargs >= 2 && next[0] == '~' && next[1] == '\0') {
approx_maxlen = 1;
i++;
} else if (moreargs >= 2 && next[0] == '=' && next[1] == '\0') {
i++;
}
if (getLongLongFromObjectOrReply(c,c->argv[i+1],&maxlen,NULL)
!= C_OK) return;
if (maxlen < 0) {
addReplyError(c,"The MAXLEN argument must be >= 0.");
return;
}
i++;
maxlen_arg_idx = i;
} else {
addReply(c,shared.syntaxerr);
return;
@@ -2408,12 +2241,11 @@ void xtrimCommand(client *c) {
signalModifiedKey(c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_STREAM,"xtrim",c->argv[1],c->db->id);
server.dirty += deleted;
if (approx_maxlen) streamRewriteApproxMaxlen(c,s,maxlen_arg_idx);
}
addReplyLongLong(c,deleted);
}
/* XINFO CONSUMERS <key> <group>
/* XINFO CONSUMERS key group
* XINFO GROUPS <key>
* XINFO STREAM <key>
* XINFO HELP. */
@@ -2470,11 +2302,11 @@ NULL
if (idle < 0) idle = 0;
addReplyMultiBulkLen(c,6);
addReplyBulkCString(c,"name");
addReplyStatus(c,"name");
addReplyBulkCBuffer(c,consumer->name,sdslen(consumer->name));
addReplyBulkCString(c,"pending");
addReplyStatus(c,"pending");
addReplyLongLong(c,raxSize(consumer->pel));
addReplyBulkCString(c,"idle");
addReplyStatus(c,"idle");
addReplyLongLong(c,idle);
}
raxStop(&ri);
@@ -2492,28 +2324,28 @@ NULL
while(raxNext(&ri)) {
streamCG *cg = ri.data;
addReplyMultiBulkLen(c,8);
addReplyBulkCString(c,"name");
addReplyStatus(c,"name");
addReplyBulkCBuffer(c,ri.key,ri.key_len);
addReplyBulkCString(c,"consumers");
addReplyStatus(c,"consumers");
addReplyLongLong(c,raxSize(cg->consumers));
addReplyBulkCString(c,"pending");
addReplyStatus(c,"pending");
addReplyLongLong(c,raxSize(cg->pel));
addReplyBulkCString(c,"last-delivered-id");
addReplyStatus(c,"last-delivered-id");
addReplyStreamID(c,&cg->last_id);
}
raxStop(&ri);
} else if (!strcasecmp(opt,"STREAM") && c->argc == 3) {
/* XINFO STREAM <key> (or the alias XINFO <key>). */
addReplyMultiBulkLen(c,14);
addReplyBulkCString(c,"length");
addReplyStatus(c,"length");
addReplyLongLong(c,s->length);
addReplyBulkCString(c,"radix-tree-keys");
addReplyStatus(c,"radix-tree-keys");
addReplyLongLong(c,raxSize(s->rax));
addReplyBulkCString(c,"radix-tree-nodes");
addReplyStatus(c,"radix-tree-nodes");
addReplyLongLong(c,s->rax->numnodes);
addReplyBulkCString(c,"groups");
addReplyStatus(c,"groups");
addReplyLongLong(c,s->cgroups ? raxSize(s->cgroups) : 0);
addReplyBulkCString(c,"last-generated-id");
addReplyStatus(c,"last-generated-id");
addReplyStreamID(c,&s->last_id);
/* To emit the first/last entry we us the streamReplyWithRange()
@@ -2522,11 +2354,11 @@ NULL
streamID start, end;
start.ms = start.seq = 0;
end.ms = end.seq = UINT64_MAX;
addReplyBulkCString(c,"first-entry");
addReplyStatus(c,"first-entry");
count = streamReplyWithRange(c,s,&start,&end,1,0,NULL,NULL,
STREAM_RWR_RAWENTRIES,NULL);
if (!count) addReply(c,shared.nullbulk);
addReplyBulkCString(c,"last-entry");
addReplyStatus(c,"last-entry");
count = streamReplyWithRange(c,s,&start,&end,1,1,NULL,NULL,
STREAM_RWR_RAWENTRIES,NULL);
if (!count) addReply(c,shared.nullbulk);
+7 -5
View File
@@ -301,22 +301,24 @@ void mgetCommand(client *c) {
}
void msetGenericCommand(client *c, int nx) {
int j;
int j, busykeys = 0;
if ((c->argc % 2) == 0) {
addReplyError(c,"wrong number of arguments for MSET");
return;
}
/* Handle the NX flag. The MSETNX semantic is to return zero and don't
* set anything if at least one key alerady exists. */
* set nothing at all if at least one already key exists. */
if (nx) {
for (j = 1; j < c->argc; j += 2) {
if (lookupKeyWrite(c->db,c->argv[j]) != NULL) {
addReply(c, shared.czero);
return;
busykeys++;
}
}
if (busykeys) {
addReply(c, shared.czero);
return;
}
}
for (j = 1; j < c->argc; j += 2) {
-19
View File
@@ -39,7 +39,6 @@
#include <float.h>
#include <stdint.h>
#include <errno.h>
#include <time.h>
#include "util.h"
#include "sha1.h"
@@ -653,24 +652,6 @@ sds getAbsolutePath(char *filename) {
return abspath;
}
/*
* Gets the proper timezone in a more portable fashion
* i.e timezone variables are linux specific.
*/
unsigned long getTimeZone(void) {
#ifdef __linux__
return timezone;
#else
struct timeval tv;
struct timezone tz;
gettimeofday(&tv, &tz);
return tz.tz_minuteswest * 60UL;
#endif
}
/* Return true if the specified path is just a file basename without any
* relative or absolute path. This function just checks that no / or \
* character exists inside the specified path, that's enough in the
-1
View File
@@ -50,7 +50,6 @@ int string2ld(const char *s, size_t slen, long double *dp);
int d2string(char *buf, size_t len, double value);
int ld2string(char *buf, size_t len, long double value, int humanfriendly);
sds getAbsolutePath(char *filename);
unsigned long getTimeZone(void);
int pathIsBaseName(char *path);
#ifdef REDIS_TEST
+1 -1
View File
@@ -1 +1 @@
#define REDIS_VERSION "5.0.1"
#define REDIS_VERSION "999.999.999"
+1 -17
View File
@@ -164,7 +164,7 @@ void *zrealloc(void *ptr, size_t size) {
if (!newptr) zmalloc_oom_handler(size);
*((size_t*)newptr) = size;
update_zmalloc_stat_free(oldsize+PREFIX_SIZE);
update_zmalloc_stat_free(oldsize);
update_zmalloc_stat_alloc(size+PREFIX_SIZE);
return (char*)newptr+PREFIX_SIZE;
#endif
@@ -438,20 +438,4 @@ size_t zmalloc_get_memory_size(void) {
#endif
}
#ifdef REDIS_TEST
#define UNUSED(x) ((void)(x))
int zmalloc_test(int argc, char **argv) {
void *ptr;
UNUSED(argc);
UNUSED(argv);
printf("Initial used memory: %zu\n", zmalloc_used_memory());
ptr = zmalloc(123);
printf("Allocated 123 bytes; used: %zu\n", zmalloc_used_memory());
ptr = zrealloc(ptr, 456);
printf("Reallocated to 456 bytes; used: %zu\n", zmalloc_used_memory());
zfree(ptr);
printf("Freed pointer; used: %zu\n", zmalloc_used_memory());
return 0;
}
#endif
-4
View File
@@ -103,8 +103,4 @@ size_t zmalloc_usable(void *ptr);
#define zmalloc_usable(p) zmalloc_size(p)
#endif
#ifdef REDIS_TEST
int zmalloc_test(int argc, char **argv);
#endif
#endif /* __ZMALLOC_H */
+1 -1
View File
@@ -49,7 +49,7 @@ start_server {tags {"repl"}} {
set fd [open /tmp/repldump2.txt w]
puts -nonewline $fd $csv2
close $fd
puts "Master - Replica inconsistency"
puts "Master - Slave inconsistency"
puts "Run diff -u against /tmp/repldump*.txt for more info"
}
assert_equal [r debug digest] [r -1 debug digest]
+3 -3
View File
@@ -29,7 +29,7 @@ start_server {} {
wait_for_condition 50 1000 {
[$R(1) dbsize] == 1 && [$R(2) dbsize] == 1
} else {
fail "Replicas not replicating from master"
fail "Slaves not replicating from master"
}
$R(0) config set repl-backlog-size 10mb
$R(1) config set repl-backlog-size 10mb
@@ -41,12 +41,12 @@ start_server {} {
set elapsed [expr {[clock milliseconds]-$cycle_start_time}]
if {$elapsed > $duration*1000} break
if {rand() < .05} {
test "PSYNC2 #3899 regression: kill first replica" {
test "PSYNC2 #3899 regression: kill first slave" {
$R(1) client kill type master
}
}
if {rand() < .05} {
test "PSYNC2 #3899 regression: kill chained replica" {
test "PSYNC2 #3899 regression: kill chained slave" {
$R(2) client kill type master
}
}
+7 -7
View File
@@ -95,7 +95,7 @@ start_server {} {
if {$disconnect} {
$R($slave_id) client kill type master
if {$debug_msg} {
puts "+++ Breaking link for replica #$slave_id"
puts "+++ Breaking link for slave #$slave_id"
}
}
}
@@ -158,7 +158,7 @@ start_server {} {
wait_for_condition 50 1000 {
[status $R($master_id) connected_slaves] == 4
} else {
fail "Replica not reconnecting"
fail "Slave not reconnecting"
}
}
@@ -173,13 +173,13 @@ start_server {} {
wait_for_condition 50 1000 {
[status $R($master_id) connected_slaves] == 4
} else {
fail "Replica not reconnecting"
fail "Slave not reconnecting"
}
set new_sync_count [status $R($master_id) sync_full]
assert {$sync_count == $new_sync_count}
}
test "PSYNC2: Replica RDB restart with EVALSHA in backlog issue #4483" {
test "PSYNC2: Slave RDB restart with EVALSHA in backlog issue #4483" {
# Pick a random slave
set slave_id [expr {($master_id+1)%5}]
set sync_count [status $R($master_id) sync_full]
@@ -194,7 +194,7 @@ start_server {} {
wait_for_condition 50 1000 {
[$R($master_id) debug digest] == [$R($slave_id) debug digest]
} else {
fail "Replica not reconnecting"
fail "Slave not reconnecting"
}
# Prevent the slave from receiving master updates, and at
@@ -228,7 +228,7 @@ start_server {} {
wait_for_condition 50 1000 {
[status $R($master_id) connected_slaves] == 4
} else {
fail "Replica not reconnecting"
fail "Slave not reconnecting"
}
set new_sync_count [status $R($master_id) sync_full]
assert {$sync_count == $new_sync_count}
@@ -238,7 +238,7 @@ start_server {} {
wait_for_condition 50 1000 {
[$R($master_id) debug digest] == [$R($slave_id) debug digest]
} else {
fail "Debug digest mismatch between master and replica in post-restart handshake"
fail "Debug digest mismatch between master and slave in post-restart handshake"
}
}
+4 -4
View File
@@ -16,7 +16,7 @@ start_server {tags {"repl"}} {
wait_for_condition 50 100 {
[r -1 get foo] eq {12345}
} else {
fail "Write did not reached replica"
fail "Write did not reached slave"
}
}
@@ -34,7 +34,7 @@ start_server {tags {"repl"}} {
wait_for_condition 50 100 {
[r -1 get foo] eq {12345}
} else {
fail "Write did not reached replica"
fail "Write did not reached slave"
}
}
@@ -60,7 +60,7 @@ start_server {tags {"repl"}} {
wait_for_condition 50 100 {
[r -1 get foo] eq {aaabbb}
} else {
fail "Write did not reached replica"
fail "Write did not reached slave"
}
}
@@ -81,7 +81,7 @@ start_server {tags {"repl"}} {
set fd [open /tmp/repldump2.txt w]
puts -nonewline $fd $csv2
close $fd
puts "Master - Replica inconsistency"
puts "Master - Slave inconsistency"
puts "Run diff -u against /tmp/repldump*.txt for more info"
}
assert_equal [r debug digest] [r -1 debug digest]
+2 -2
View File
@@ -25,7 +25,7 @@ start_server {tags {"repl"}} {
set fd [open /tmp/repldump2.txt w]
puts -nonewline $fd $csv2
close $fd
puts "Master - Replica inconsistency"
puts "Master - Slave inconsistency"
puts "Run diff -u against /tmp/repldump*.txt for more info"
}
assert_equal [r debug digest] [r -1 debug digest]
@@ -98,7 +98,7 @@ start_server {tags {"repl"}} {
set fd [open /tmp/repldump2.txt w]
puts -nonewline $fd $csv2
close $fd
puts "Master - Replica inconsistency"
puts "Master - Slave inconsistency"
puts "Run diff -u against /tmp/repldump*.txt for more info"
}
+1 -1
View File
@@ -47,7 +47,7 @@ start_server {tags {"repl"}} {
set fd [open /tmp/repldump2.txt w]
puts -nonewline $fd $csv2
close $fd
puts "Master - Replica inconsistency"
puts "Master - Slave inconsistency"
puts "Run diff -u against /tmp/repldump*.txt for more info"
}
assert_equal [r debug digest] [r -1 debug digest]
+2 -2
View File
@@ -60,7 +60,7 @@ proc test_psync {descr duration backlog_size backlog_ttl delay cond diskless rec
if ($reconnect) {
for {set j 0} {$j < $duration*10} {incr j} {
after 100
# catch {puts "MASTER [$master dbsize] keys, REPLICA [$slave dbsize] keys"}
# catch {puts "MASTER [$master dbsize] keys, SLAVE [$slave dbsize] keys"}
if {($j % 20) == 0} {
catch {
@@ -96,7 +96,7 @@ proc test_psync {descr duration backlog_size backlog_ttl delay cond diskless rec
set fd [open /tmp/repldump2.txt w]
puts -nonewline $fd $csv2
close $fd
puts "Master - Replica inconsistency"
puts "Master - Slave inconsistency"
puts "Run diff -u against /tmp/repldump*.txt for more info"
}
assert_equal [r debug digest] [r -1 debug digest]
+16 -16
View File
@@ -32,7 +32,7 @@ start_server {tags {"repl"}} {
wait_for_condition 50 1000 {
[string match *handshake* [$slave role]]
} else {
fail "Replica does not enter handshake state"
fail "Slave does not enter handshake state"
}
}
@@ -45,7 +45,7 @@ start_server {tags {"repl"}} {
wait_for_condition 50 1000 {
[log_file_matches $slave_log "*Timeout connecting to the MASTER*"]
} else {
fail "Replica is not able to detect timeout"
fail "Slave is not able to detect timeout"
}
}
}
@@ -66,7 +66,7 @@ start_server {tags {"repl"}} {
[lindex [$A role] 0] eq {slave} &&
[string match {*master_link_status:up*} [$A info replication]]
} else {
fail "Can't turn the instance into a replica"
fail "Can't turn the instance into a slave"
}
}
@@ -77,7 +77,7 @@ start_server {tags {"repl"}} {
wait_for_condition 50 100 {
[$A debug digest] eq [$B debug digest]
} else {
fail "Master and replica have different digest: [$A debug digest] VS [$B debug digest]"
fail "Master and slave have different digest: [$A debug digest] VS [$B debug digest]"
}
}
@@ -102,10 +102,10 @@ start_server {tags {"repl"}} {
[lindex [$B role] 0] eq {slave} &&
[string match {*master_link_status:up*} [$B info replication]]
} else {
fail "Can't turn the instance into a replica"
fail "Can't turn the instance into a slave"
}
# Push elements into the "foo" list of the new replica.
# Push elements into the "foo" list of the new slave.
# If the client is still attached to the instance, we'll get
# a desync between the two instances.
$A rpush foo a b c
@@ -116,7 +116,7 @@ start_server {tags {"repl"}} {
[$A lrange foo 0 -1] eq {a b c} &&
[$B lrange foo 0 -1] eq {a b c}
} else {
fail "Master and replica have different digest: [$A debug digest] VS [$B debug digest]"
fail "Master and slave have different digest: [$A debug digest] VS [$B debug digest]"
}
}
}
@@ -135,7 +135,7 @@ start_server {tags {"repl"}} {
s master_link_status
} {down}
test {The role should immediately be changed to "replica"} {
test {The role should immediately be changed to "slave"} {
s role
} {slave}
@@ -154,7 +154,7 @@ start_server {tags {"repl"}} {
wait_for_condition 500 100 {
[r 0 get mykey] eq {bar}
} else {
fail "SET on master did not propagated on replica"
fail "SET on master did not propagated on slave"
}
}
@@ -201,7 +201,7 @@ foreach dl {no yes} {
lappend slaves [srv 0 client]
start_server {} {
lappend slaves [srv 0 client]
test "Connect multiple replicas at the same time (issue #141), diskless=$dl" {
test "Connect multiple slaves at the same time (issue #141), diskless=$dl" {
# Send SLAVEOF commands to slaves
[lindex $slaves 0] slaveof $master_host $master_port
[lindex $slaves 1] slaveof $master_host $master_port
@@ -220,7 +220,7 @@ foreach dl {no yes} {
}
}
if {$retry == 0} {
error "assertion:Replicas not correctly synchronized"
error "assertion:Slaves not correctly synchronized"
}
# Wait that slaves acknowledge they are online so
@@ -231,7 +231,7 @@ foreach dl {no yes} {
[lindex [[lindex $slaves 1] role] 3] eq {connected} &&
[lindex [[lindex $slaves 2] role] 3] eq {connected}
} else {
fail "Replicas still not connected after some time"
fail "Slaves still not connected after some time"
}
# Stop the write load
@@ -248,7 +248,7 @@ foreach dl {no yes} {
[$master dbsize] == [[lindex $slaves 1] dbsize] &&
[$master dbsize] == [[lindex $slaves 2] dbsize]
} else {
fail "Different number of keys between masted and replica after too long time."
fail "Different number of keys between masted and slave after too long time."
}
# Check digests
@@ -273,7 +273,7 @@ start_server {tags {"repl"}} {
set master_port [srv 0 port]
set load_handle0 [start_write_load $master_host $master_port 3]
start_server {} {
test "Master stream is correctly processed while the replica has a script in -BUSY state" {
test "Master stream is correctly processed while the slave has a script in -BUSY state" {
set slave [srv 0 client]
puts [srv 0 port]
$slave config set lua-time-limit 500
@@ -283,7 +283,7 @@ start_server {tags {"repl"}} {
wait_for_condition 500 100 {
[lindex [$slave role] 3] eq {connected}
} else {
fail "Replica still not connected after some time"
fail "Slave still not connected after some time"
}
# Wait some time to make sure the master is sending data
@@ -305,7 +305,7 @@ start_server {tags {"repl"}} {
wait_for_condition 500 100 {
[$master debug digest] eq [$slave debug digest]
} else {
fail "Different datasets between replica and master"
fail "Different datasets between slave and master"
}
}
}
+1 -1
View File
@@ -33,7 +33,7 @@ start_server {tags {"dump"}} {
set now [clock milliseconds]
r restore foo [expr $now+3000] $encoded absttl
set ttl [r pttl foo]
assert {$ttl >= 2900 && $ttl <= 3100}
assert {$ttl >= 2990 && $ttl <= 3000}
r get foo
} {bar}
-57
View File
@@ -1,9 +1,3 @@
proc cmdstat {cmd} {
if {[regexp "\r\ncmdstat_$cmd:(.*?)\r\n" [r info commandstats] _ value]} {
set _ $value
}
}
start_server {tags {"introspection"}} {
test {TTL and TYPYE do not alter the last access time of a key} {
r set foo bar
@@ -26,55 +20,4 @@ start_server {tags {"introspection"}} {
r set key2 2
r touch key0 key1 key2 key3
} 2
test {command stats for GEOADD} {
r config resetstat
r GEOADD foo 0 0 bar
assert_match {*calls=1,*} [cmdstat geoadd]
assert_match {} [cmdstat zadd]
}
test {command stats for EXPIRE} {
r config resetstat
r SET foo bar
r EXPIRE foo 0
assert_match {*calls=1,*} [cmdstat expire]
assert_match {} [cmdstat del]
}
test {command stats for BRPOP} {
r config resetstat
r LPUSH list foo
r BRPOP list 0
assert_match {*calls=1,*} [cmdstat brpop]
assert_match {} [cmdstat rpop]
}
test {command stats for MULTI} {
r config resetstat
r MULTI
r set foo bar
r GEOADD foo2 0 0 bar
r EXPIRE foo2 0
r EXEC
assert_match {*calls=1,*} [cmdstat multi]
assert_match {*calls=1,*} [cmdstat exec]
assert_match {*calls=1,*} [cmdstat set]
assert_match {*calls=1,*} [cmdstat expire]
assert_match {*calls=1,*} [cmdstat geoadd]
}
test {command stats for scripts} {
r config resetstat
r set mykey myval
r eval {
redis.call('set', KEYS[1], 0)
redis.call('expire', KEYS[1], 0)
redis.call('geoadd', KEYS[1], 0, 0, "bar")
} 1 mykey
assert_match {*calls=1,*} [cmdstat eval]
assert_match {*calls=2,*} [cmdstat set]
assert_match {*calls=1,*} [cmdstat expire]
assert_match {*calls=1,*} [cmdstat geoadd]
}
}
+2 -2
View File
@@ -163,7 +163,7 @@ proc test_slave_buffers {test_name cmd_count payload_len limit_memory pipeline}
# make sure master doesn't disconnect slave because of timeout
$master config set repl-timeout 300 ;# 5 minutes
$master config set maxmemory-policy allkeys-random
$master config set client-output-buffer-limit "replica 100000000 100000000 300"
$master config set client-output-buffer-limit "slave 100000000 100000000 300"
$master config set repl-backlog-size [expr {10*1024}]
$slave slaveof $master_host $master_port
@@ -238,5 +238,5 @@ test_slave_buffers {slave buffer are counted correctly} 1000000 10 0 1
# test that slave buffer don't induce eviction
# test again with fewer (and bigger) commands without pipeline, but with eviction
test_slave_buffers "replica buffer don't induce eviction" 100000 100 1 0
test_slave_buffers "slave buffer don't induce eviction" 100000 100 1 0
+1 -1
View File
@@ -29,7 +29,7 @@ start_server {} {
set m_usedmemory [info_memory $master used_memory]
set s_usedmemory [info_memory $slave used_memory]
if { $s_usedmemory > $m_usedmemory + 10*1024*1024 } {
fail "the used_memory of replica is much larger than master. Master:$m_usedmemory Replica:$s_usedmemory"
fail "the used_memory of slave is too larger than master.Master:$m_usedmemory Slave:$s_usedmemory"
}
}
}}
+10 -17
View File
@@ -148,11 +148,9 @@ start_server {tags {"scripting"}} {
test {EVAL - Scripts can't run certain commands} {
set e {}
r debug lua-always-replicate-commands 0
catch {
r eval "redis.pcall('randomkey'); return redis.pcall('set','x','ciao')" 0
} e
r debug lua-always-replicate-commands 1
set e
} {*not allowed after*}
@@ -301,12 +299,9 @@ start_server {tags {"scripting"}} {
} {b534286061d4b9e4026607613b95c06c06015ae8 loaded}
test "In the context of Lua the output of random commands gets ordered" {
r debug lua-always-replicate-commands 0
r del myset
r sadd myset a b c d e f g h i l m n o p q r s t u v z aa aaa azz
set res [r eval {return redis.call('smembers',KEYS[1])} 1 myset]
r debug lua-always-replicate-commands 1
set res
r eval {return redis.call('smembers',KEYS[1])} 1 myset
} {a aa aaa azz b c d e f g h i l m n o p q r s t u v z}
test "SORT is normally not alpha re-ordered for the scripting engine" {
@@ -542,7 +537,7 @@ foreach cmdrepl {0 1} {
r debug lua-always-replicate-commands 1
}
test "Before the replica connects we issue two EVAL commands $rt" {
test "Before the slave connects we issue two EVAL commands $rt" {
# One with an error, but still executing a command.
# SHA is: 67164fc43fa971f76fd1aaeeaf60c1c178d25876
catch {
@@ -553,13 +548,13 @@ foreach cmdrepl {0 1} {
r eval {return redis.call('incr',KEYS[1])} 1 x
} {2}
test "Connect a replica to the master instance $rt" {
test "Connect a slave to the master instance $rt" {
r -1 slaveof [srv 0 host] [srv 0 port]
wait_for_condition 50 100 {
[s -1 role] eq {slave} &&
[string match {*master_link_status:up*} [r -1 info replication]]
} else {
fail "Can't turn the instance into a replica"
fail "Can't turn the instance into a slave"
}
}
@@ -592,7 +587,7 @@ foreach cmdrepl {0 1} {
wait_for_condition 50 100 {
[r -1 lrange a 0 -1] eq [r lrange a 0 -1]
} else {
fail "Expected list 'a' in replica and master to be the same, but they are respectively '[r -1 lrange a 0 -1]' and '[r lrange a 0 -1]'"
fail "Expected list 'a' in slave and master to be the same, but they are respectively '[r -1 lrange a 0 -1]' and '[r lrange a 0 -1]'"
}
set res
} {a 1}
@@ -627,7 +622,7 @@ foreach cmdrepl {0 1} {
wait_for_condition 50 100 {
[r -1 debug digest] eq [r debug digest]
} else {
fail "Master-Replica desync after Lua script using SELECT."
fail "Master-Slave desync after Lua script using SELECT."
}
}
}
@@ -636,13 +631,13 @@ foreach cmdrepl {0 1} {
start_server {tags {"scripting repl"}} {
start_server {overrides {appendonly yes aof-use-rdb-preamble no}} {
test "Connect a replica to the master instance" {
test "Connect a slave to the master instance" {
r -1 slaveof [srv 0 host] [srv 0 port]
wait_for_condition 50 100 {
[s -1 role] eq {slave} &&
[string match {*master_link_status:up*} [r -1 info replication]]
} else {
fail "Can't turn the instance into a replica"
fail "Can't turn the instance into a slave"
}
}
@@ -660,13 +655,11 @@ start_server {tags {"scripting repl"}} {
} {1}
test "Redis.set_repl() must be issued after replicate_commands()" {
r debug lua-always-replicate-commands 0
catch {
r eval {
redis.set_repl(redis.REPL_ALL);
} 0
} e
r debug lua-always-replicate-commands 1
set e
} {*only after turning on*}
@@ -696,7 +689,7 @@ start_server {tags {"scripting repl"}} {
wait_for_condition 50 100 {
[r -1 mget a b c d] eq {1 {} {} 4}
} else {
fail "Only a and c should be replicated to replica"
fail "Only a and c should be replicated to slave"
}
# Master should have everything right now
@@ -735,7 +728,7 @@ start_server {tags {"scripting repl"}} {
wait_for_condition 50 100 {
[r get time] eq [r -1 get time]
} else {
fail "Time key does not match between master and replica"
fail "Time key does not match between master and slave"
}
}
}
-60
View File
@@ -9,17 +9,6 @@ start_server {
set err
} {BUSYGROUP*}
test {XGROUP CREATE: automatic stream creation fails without MKSTREAM} {
r DEL mystream
catch {r XGROUP CREATE mystream mygroup $} err
set err
} {ERR*}
test {XGROUP CREATE: automatic stream creation works with MKSTREAM} {
r DEL mystream
r XGROUP CREATE mystream mygroup $ MKSTREAM
} {OK}
test {XREADGROUP will return only new elements} {
r XADD mystream * a 1
r XADD mystream * b 2
@@ -107,53 +96,4 @@ start_server {
set c [llength [lindex [r xreadgroup group g1 c2 streams events >] 0 1]]
assert {$c == 5}
}
start_server {} {
set master [srv -1 client]
set master_host [srv -1 host]
set master_port [srv -1 port]
set slave [srv 0 client]
foreach noack {0 1} {
test "Consumer group last ID propagation to slave (NOACK=$noack)" {
$slave slaveof $master_host $master_port
wait_for_condition 50 100 {
[s 0 master_link_status] eq {up}
} else {
fail "Replication not started."
}
$master del stream
$master xadd stream * a 1
$master xadd stream * a 2
$master xadd stream * a 3
$master xgroup create stream mygroup 0
# Consume the first two items on the master
for {set j 0} {$j < 2} {incr j} {
if {$noack} {
set item [$master xreadgroup group mygroup \
myconsumer COUNT 1 NOACK STREAMS stream >]
} else {
set item [$master xreadgroup group mygroup \
myconsumer COUNT 1 STREAMS stream >]
}
set id [lindex $item 0 1 0 0]
if {$noack == 0} {
assert {[$master xack stream mygroup $id] eq "1"}
}
}
# Turn slave into master
$slave slaveof no one
set item [$slave xreadgroup group mygroup myconsumer \
COUNT 1 STREAMS stream >]
# The consumed enty should be the third
set myentry [lindex $item 0 1 0 1]
assert {$myentry eq {a 3}}
}
}
}
}
-94
View File
@@ -317,97 +317,3 @@ start_server {
assert_equal [r xrevrange teststream2 1234567891245 -] {{1234567891240-0 {key1 value2}} {1234567891230-0 {key1 value1}}}
}
}
start_server {tags {"stream"} overrides {appendonly yes}} {
test {XADD with MAXLEN > xlen can propagate correctly} {
for {set j 0} {$j < 100} {incr j} {
r XADD mystream * xitem v
}
r XADD mystream MAXLEN 200 * xitem v
incr j
assert {[r xlen mystream] == $j}
r debug loadaof
r XADD mystream * xitem v
incr j
assert {[r xlen mystream] == $j}
}
}
start_server {tags {"stream"} overrides {appendonly yes}} {
test {XADD with ~ MAXLEN can propagate correctly} {
for {set j 0} {$j < 100} {incr j} {
r XADD mystream * xitem v
}
r XADD mystream MAXLEN ~ $j * xitem v
incr j
assert {[r xlen mystream] == $j}
r config set stream-node-max-entries 1
r debug loadaof
r XADD mystream * xitem v
incr j
assert {[r xlen mystream] == $j}
}
}
start_server {tags {"stream"} overrides {appendonly yes stream-node-max-entries 10}} {
test {XTRIM with ~ MAXLEN can propagate correctly} {
for {set j 0} {$j < 100} {incr j} {
r XADD mystream * xitem v
}
r XTRIM mystream MAXLEN ~ 85
assert {[r xlen mystream] == 89}
r config set stream-node-max-entries 1
r debug loadaof
r XADD mystream * xitem v
incr j
assert {[r xlen mystream] == 90}
}
}
start_server {tags {"xsetid"}} {
test {XADD can CREATE an empty stream} {
r XADD mystream MAXLEN 0 * a b
assert {[dict get [r xinfo stream mystream] length] == 0}
}
test {XSETID can set a specific ID} {
r XSETID mystream "200-0"
assert {[dict get [r xinfo stream mystream] last-generated-id] == "200-0"}
}
test {XSETID cannot SETID with smaller ID} {
r XADD mystream * a b
catch {r XSETID mystream "1-1"} err
r XADD mystream MAXLEN 0 * a b
set err
} {ERR*smaller*}
test {XSETID cannot SETID on non-existent key} {
catch {r XSETID stream 1-1} err
set _ $err
} {ERR no such key}
}
start_server {tags {"stream"} overrides {appendonly yes aof-use-rdb-preamble no}} {
test {Empty stream can be rewrite into AOF correctly} {
r XADD mystream MAXLEN 0 * a b
assert {[dict get [r xinfo stream mystream] length] == 0}
r bgrewriteaof
waitForBgrewriteaof r
r debug loadaof
assert {[dict get [r xinfo stream mystream] length] == 0}
}
test {Stream can be rewrite into AOF correctly after XDEL lastid} {
r XSETID mystream 0-0
r XADD mystream 1-1 a b
r XADD mystream 2-2 a b
assert {[dict get [r xinfo stream mystream] length] == 2}
r XDEL mystream 2-2
r bgrewriteaof
waitForBgrewriteaof r
r debug loadaof
assert {[dict get [r xinfo stream mystream] length] == 1}
assert {[dict get [r xinfo stream mystream] last-generated-id] == "2-2"}
}
}
+3 -8
View File
@@ -1,17 +1,12 @@
#!/usr/bin/env tclsh
if {[llength $::argv] != 2 && [llength $::argv] != 3} {
puts "Usage: $::argv0 <branch> <version> \[<num-commits>\]"
if {[llength $::argv] != 2} {
puts "Usage: $::argv0 <branch> <version>"
exit 1
}
set branch [lindex $::argv 0]
set ver [lindex $::argv 1]
if {[llength $::argv] == 3} {
set count [lindex ::$argv 2]
} else {
set count 100
}
set template {
================================================================================
@@ -26,7 +21,7 @@ append template "\n\n"
set date [clock format [clock seconds]]
set template [string map [list %ver% $ver %date% $date] $template]
append template [exec git log $branch~$count..$branch "--format=format:%an in commit %h:%n %s" --shortstat]
append template [exec git log $branch~100..$branch "--format=format:%an in commit %h:%n %s" --shortstat]
#Older, more verbose version.
#