Compare commits

..
859 Commits
Author SHA1 Message Date
antirez 01671edcca RSS aware maxmemory: algorith #1 implemented. 2014-11-17 15:10:03 +01:00
antirez 5f185413df RSS aware maxmemory: enforced/adjusted == maxmemory on config changes. 2014-11-17 10:46:42 +01:00
antirez 2a11e94fdd RSS aware maxmemory: config settings. 2014-11-17 10:38:51 +01:00
antirez 7ea331d601 THP detection for LATENCY DOCTOR. 2014-11-12 11:17:12 +01:00
antirez 110f0464e0 Check THP support at startup and warn about it. 2014-11-12 10:55:47 +01:00
antirez 3ef0876b95 THP detection / reporting functions added. 2014-11-12 10:43:32 +01:00
antirez bb7fea0d5c Diskless SYNC: fix RDB EOF detection.
RDB EOF detection was relying on the final part of the RDB transfer to
be a magic 40 bytes EOF marker. However as the slave is put online
immediately, and because of sockets timeouts, the replication stream is
actually contiguous with the RDB file.

This means that to detect the EOF correctly we should either:

1) Scan all the stream searching for the mark. Sucks CPU-wise.
2) Start to send the replication stream only after an acknowledge.
3) Implement a proper chunked encoding.

For now solution "2" was picked, so the master does not start to send
ASAP the stream of commands in the case of diskless replication. We wait
for the first REPLCONF ACK command from the slave, that certifies us
that the slave correctly loaded the RDB file and is ready to get more
data.
2014-11-11 17:12:12 +01:00
antirez f5c6ebbfe3 Disconnect timedout slave: regression introduced with diskless repl. 2014-11-11 15:10:58 +01:00
Salvatore Sanfilippo 5a526c22cc Merge pull request #2096 from mattsta/cluster-ipv6
Enable Cluster IPv6 Support
2014-10-31 10:38:22 +01:00
Salvatore Sanfilippo a076743079 Merge pull request #2110 from mattsta/more-outbound-bind-fixes
Networking: add more outbound IP binding fixes
2014-10-31 10:01:59 +01:00
Salvatore Sanfilippo 2c42b645bc Merge pull request #2078 from mattsta/hiredis-sigpipe
Fix redis-cli from exiting after idle connection breaks
2014-10-30 12:01:51 +01:00
Matt Stancliff 0014966c1e Networking: add more outbound IP binding fixes
Same as the original bind fixes (we just missed these the
first time around).

This helps Redis not automatically send
connections from the first IP on an interface if we are bound
to a specific IP address (e.g. with multiple IP aliases on one
interface, you want to send from _your_ IP, not from the first IP
on the interface).
2014-10-29 15:09:09 -04:00
Matt Stancliff daca1edb6e Parse cluster state file in IPv6 compatible way
We need to pick the port based on the _last_ colon, not the first one.
2014-10-29 15:08:35 -04:00
Matt Stancliff bbf1af2da3 Fix redis-trib.rb IP:Port disassembly for IPv6
IP format is now any of:
  - 127.0.0.1:6379
  - ::1:6379
2014-10-29 15:08:35 -04:00
Matt Stancliff e10c5444e7 redis-cli: ignore SIGPIPE network errors
Closes #2066
2014-10-29 14:55:08 -04:00
antirez 6fbaeddf3f Merge branch 'memsync' into unstable 2014-10-29 14:25:18 +01:00
antirez 9ec22d9223 Diskless replication: missing listRewind() added.
This caused BGSAVE to be triggered a second time without any need when
we switch from socket to disk target via the command

    CONFIG SET repl-diskless-sync no

and there is already a slave waiting for the BGSAVE to start.
Also comments clarified about what is happening.
2014-10-29 12:48:22 +01:00
antirez 4b8f4b90b9 Log slave ip:port in more log messages. 2014-10-27 12:30:07 +01:00
antirez 775cc30a98 Use new slave name function for diskless repl reporting. 2014-10-27 12:23:03 +01:00
antirez 8a416ca46e Added a function to get slave name for logs. 2014-10-27 11:58:20 +01:00
antirez a27befc495 Diskless replication: log BGSAVE delay only when it is non-zero. 2014-10-27 10:48:39 +01:00
antirez 3b9a97984a Document repl-diskless-sync-delay in redis.conf. 2014-10-27 10:42:49 +01:00
antirez 707352439c Diskless sync delay is now configurable. 2014-10-27 10:36:30 +01:00
antirez c4dbc7cdec Remove duplicated log message about starting BGSAVE. 2014-10-24 10:38:42 +02:00
antirez 18de5395b2 Diskless replication documented inside example redis.conf. 2014-10-24 10:12:43 +02:00
antirez d6797d34c0 Diskless replication tested with the multiple slaves consistency test. 2014-10-24 09:49:26 +02:00
antirez ebb3bd53c2 Diskless replication: child -> parent communication improved.
Child now reports full info to the parent including IDs of slaves in
failure state and exit code.
2014-10-23 23:10:33 +02:00
antirez b50e3215d2 Translate rio fdset target EWOULDBLOCK error into ETIMEDOUT.
EWOULDBLOCK with the fdset rio target is returned when we try to write
but the send timeout socket option triggered an error. Better to
translate the error in something the user can actually recognize as a
timeout.
2014-10-22 15:58:14 +02:00
antirez d4f6a1711d Diskless replication: set / reset socket send timeout.
We need to avoid that a child -> slaves transfer can continue forever.
We use the same timeout used as global replication timeout, which is
documented to also affect I/O operations during bulk transfers.
2014-10-22 15:53:45 +02:00
antirez 2309f15d89 anet.c: new API anetSendTimeout(). 2014-10-22 15:23:21 +02:00
antirez 456003af25 Diskless replication: less debugging printfs around. 2014-10-17 17:11:48 +02:00
antirez fd112f52dc rio.c fdset write() method fixed: wrong type for return value. 2014-10-17 17:02:44 +02:00
antirez 525c488f63 rio fdset target: handle short writes.
While the socket is set in blocking mode, we still can get short writes
writing to a socket.
2014-10-17 16:45:53 +02:00
antirez 74f90c6123 anet.c: API to set sockets back to blocking mode. 2014-10-17 16:30:32 +02:00
antirez 10aafdad56 Diskless replication: rio fdset target new supports buffering.
To perform a socket write() for each RDB rio API write call was
extremely unefficient, so now rio has minimal buffering capabilities.
Writes are accumulated into a buffer and only when a given limit is
reacehd are actually wrote to the N slaves FDs.

Trivia: rio lacked support for buffering since our targets were:

1) Memory buffers.
2) C standard I/O.

Both were buffered already.
2014-10-17 11:36:12 +02:00
antirez b1337b15b6 Diskless replication: Various fixes to backgroundSaveDoneHandlerSocket() 2014-10-17 10:43:56 +02:00
antirez 4b16263bd9 Diskless replication: don't send "\n" pings to slaves.
This is useful for normal replication in order to refresh the slave
when we are persisting on disk, but for diskless replication the
child is already receiving data while in WAIT_BGSAVE_END state.
2014-10-17 10:23:44 +02:00
antirez 25a3d9965e Diskless replication: remove 40 bytes EOF mark from end of RDB file. 2014-10-17 10:23:11 +02:00
antirez 0c5a06f6bb Diskless replication: swap inverted branches to compute read len. 2014-10-17 10:22:29 +02:00
antirez 80f7f63b64 Diskless replication: don't enter the read-payload branch forever. 2014-10-17 10:21:18 +02:00
antirez f70369335d syncReadLine(): actually enforce buffer size limits. 2014-10-16 17:35:29 +02:00
antirez 5ee2ccf48e Diskless replication: EOF:<mark> streaming support slave side. 2014-10-16 17:09:35 +02:00
antirez 43ae606430 Diskless replication: redis.conf and CONFIG SET/GET support. 2014-10-16 10:22:02 +02:00
antirez 42951ab301 Diskless replication: trigger a BGSAVE after a config change.
If we turn from diskless to disk-based replication via CONFIG SET, we
need a way to start a BGSAVE if there are slaves alerady waiting for a
BGSAVE to start. Normally with disk-based replication we do it as soon
as the previous child exits, but when there is a configuration change
via CONFIG SET, we may have slaves in WAIT_BGSAVE_START state without
an RDB background process currently active.
2014-10-16 10:15:18 +02:00
antirez 5f8360eb21 Diskless replication flag renamed repl_diskless -> repl_diskless_sync. 2014-10-16 10:00:50 +02:00
antirez e9e007555e Diskless replication: trigger diskless RDB transfer if needed. 2014-10-16 09:03:52 +02:00
antirez 3730d118a3 Diskless replication: handle putting the slave online. 2014-10-15 15:31:19 +02:00
antirez 7a1e0d9898 Diskless replication: read report from child. 2014-10-15 11:36:03 +02:00
antirez fbe7545545 Diskless replication: child writes report to parent. 2014-10-15 09:46:49 +02:00
antirez 93eed9ae01 redis-cli: add missing newline in error message. 2014-10-15 09:21:02 +02:00
antirez 2a436aaeab rio.c fdset target: tolerate (and report) a subset of FDs in error.
Fdset target is used when we want to write an RDB file directly to
slave's sockets. In this setup as long as there is a single slave that
is still receiving our payload, we want to continue sennding instead of
aborting. However rio calls should abort of no FD is ok.

Also we want the errors reported so that we can signal the parent who is
ok and who is broken, so there is a new set integers with the state of
each fd. Zero is ok, non-zero is the errno of the failure, if avaialble,
or a generic EIO.
2014-10-14 17:19:42 +02:00
antirez 1cd0d26c63 Diskless replication: parent-child pipe and a few TODOs. 2014-10-14 15:29:07 +02:00
antirez 75f0cd6520 Diskless replication: RDB -> slaves transfer draft implementation. 2014-10-14 10:11:29 +02:00
antirez 850ea57c37 rio.c: draft implementation of fdset target implemented. 2014-10-10 17:44:06 +02:00
antirez 29db3227ab rio.c refactoring before adding a new target. 2014-10-10 16:36:09 +02:00
antirez 16546f5aca Add some comments in syncCommand() to clarify RDB target. 2014-10-10 16:25:58 +02:00
antirez 5ba47b50ae Merge branch 'unstable' of github.com:/antirez/redis into unstable 2014-10-09 11:26:51 +02:00
antirez da838544ae 02_upload_tarball.sh: let me exit before updating site. 2014-10-09 11:26:32 +02:00
antirez 591b69c745 Fix DEBUG POPULATE warning for lack of casting. 2014-10-09 11:17:27 +02:00
antirez 5f6950caa8 Cluster: process gossip section only for known nodes.
With the exception of nodes sending MEET packets: we have to trust them
since they can send us MEET packets only when the cluster is initially
created or because sysadmin manual action.
2014-10-08 16:58:12 +02:00
antirez 36e34a656a Cluster: fix logic to detect we are among a minority.
In the cluster evaluation function we are supposed to set the cluster
state as "fail" if we are among a minority, however the code was not
detecting to be into a minority partition if exactly half the masters
were reachable, which is a minority.
2014-10-08 16:27:07 +02:00
antirez 908be1dbeb Cluster test: helpers/onlydots.tcl: detect EOF and exit. 2014-10-08 10:17:01 +02:00
antirez 5b47783d77 Cluster test: less console-spammy resharding test. 2014-10-08 10:12:40 +02:00
antirez 2df8341c75 Define different types of RDB childs.
We need to remember what is the saving strategy of the current RDB child
process, since the configuration may be modified at runtime via CONFIG
SET and still we'll need to understand, when the child exists, what to
do and for what goal the process was initiated: to create an RDB file
on disk or to write stuff directly to slave's sockets.
2014-10-08 09:09:01 +02:00
antirez 8beb98574a RDB file creation refactored to target non-disk target. 2014-10-07 12:56:23 +02:00
antirez 034ca98678 Cluster: nodes.conf added to git ignore list. 2014-10-07 09:52:40 +02:00
antirez edb3987a06 Cluster: more chatty slaves when failover is stalled. 2014-10-07 09:51:55 +02:00
antirez e4b0c8ec50 Linenoise README updated to match source code. 2014-10-06 09:49:44 +02:00
Salvatore Sanfilippo 3c6f9ac37c Merge pull request #1902 from mattsta/comment-fixes
ALL comment fixes
2014-10-06 09:44:54 +02:00
antirez 389ec305b3 Test: check that INCR object sharing works as expected. 2014-10-03 12:28:56 +01:00
antirez 16559b4615 INCR: Modify incremented object in-place when possible.
However we don't try to do this if the integer is already inside a range
representable with a shared integer.

The performance gain appears to be around ~15% in micro benchmarks,
however in the long run this also helps to improve locality, so should
have more, hard to measure, benefits.
2014-10-03 12:11:13 +01:00
Miguel ParramonandMatt Stancliff be0061635e redis.conf small grammar fix
😄

Closes #2034
2014-09-29 06:49:09 -04:00
Ezequiel LovelleandMatt Stancliff c3d252d995 Fix typo
Closes #2029
2014-09-29 06:49:09 -04:00
Aniruddh ChaturvediandMatt Stancliff 82154ffab5 Fix typo in unit test
Closes #2005
2014-09-29 06:49:09 -04:00
Gregory PetrosyanandMatt Stancliff fa8786f17a Fix typos in comments
Closes #2002
2014-09-29 06:49:09 -04:00
Matt Stancliff 678df67501 Add missing 'by' 2014-09-29 06:49:09 -04:00
Juarez BochiandMatt Stancliff a63b9c24f0 Fix typo in scripting.c comment
Closes #1960
2014-09-29 06:49:08 -04:00
T.J. SchuckandMatt Stancliff 38a5db6c9a Fix typo radis -> redis
Closes #1938
2014-09-29 06:49:08 -04:00
Matt Stancliff ef5fc599b4 Cleanup wording of dictScan() comment
Some language in the comment was difficult
to understand, so this commit: clarifies wording, removes
unnecessary words, and relocates some dependent clauses
closer to what they actually describe.

I also tried to break up longer chains of thought
(if X, then Y, and Q, and also F, so obviously M)
into more manageable chunks for ease of understanding.
2014-09-29 06:49:08 -04:00
Xiaojie ZhangandMatt Stancliff 9aba616866 Fix comment inconsistencies in ziplist.c
Closes #1523
2014-09-29 06:49:08 -04:00
Jan-Erik RedigerandMatt Stancliff cf737ff1c8 Fix sentinel.conf typo
Closes #1739
2014-09-29 06:49:08 -04:00
Matt Stancliff 1cedebb799 Remove trailing spaces from tests 2014-09-29 06:49:08 -04:00
Matt Stancliff 09cb281bc3 Fix spelling in some test cases 2014-09-29 06:49:08 -04:00
Matt Stancliff 28343966a4 Spell software correctly 2014-09-29 06:49:07 -04:00
Matt Stancliff 12d0195b30 Clean up text throughout project
- Remove trailing newlines from redis.conf
  - Fix comment misspelling
  - Clarifies zipEncodeLength usage and a C API mention (#1243, #1242)
  - Fix cluster typos (inspired by @papanikge #1507)
  - Fix rewite -> rewrite in a few places (inspired by #682)

Closes #1243, #1242, #1507
2014-09-29 06:49:07 -04:00
Michael ParkerandMatt Stancliff fc8f7ec765 Fix hash table size in comment for dictScan
Closes #1351
2014-09-29 06:49:07 -04:00
Ted NymanandMatt Stancliff 39be4bf496 Fix on-place -> in-place
Closes #1373
2014-09-29 06:49:07 -04:00
xuxiangandMatt Stancliff d9462d2780 Fix typo in redis.h
Closes #1386
2014-09-29 06:49:07 -04:00
BenandMatt Stancliff f62f00e062 Grammar and typo fixes in redis.conf
Closes #1441
2014-09-29 06:49:07 -04:00
Aaron RutkovskyandMatt Stancliff 3a82b8ac64 Fix typos
Closes #1513
2014-09-29 06:49:07 -04:00
Jan-Erik RedigerandMatt Stancliff 9f98b29cef Fix typo: ad -> and
Closes #1537
2014-09-29 06:49:06 -04:00
Agis AnastasopoulosandMatt Stancliff 4d2042212c Fix typo
Closes #1544
2014-09-29 06:49:06 -04:00
Dmitry ChestnykhandMatt Stancliff 45409d3d8b Fix typo in 00-RELEASENOTES
Closes #1673
2014-09-29 06:49:06 -04:00
Juarez BochiandMatt Stancliff 0d434a446a Fix typo
Closes #1682
2014-09-29 06:49:06 -04:00
Jan-Erik RedigerandMatt Stancliff aa628446f2 Fix typo in redis.conf
Closes #1713
2014-09-29 06:49:06 -04:00
zionwuandMatt Stancliff a2583466e4 Fix incorrect comments
error != success; and 0 != number of bytes written

Closes #1806
2014-09-29 06:49:06 -04:00
Jan-Erik RedigerandMatt Stancliff 0cb874819b Fix many small typos
Closes #1871
2014-09-29 06:49:06 -04:00
Matt Stancliff 6e0a280cb5 Update linenoise accessory files
The core linenoise code was being backported, but not
the README or example.  It's less confusing for users
if everything matches across directories.

Fix inspired by @thrig

Closes #1872
2014-09-29 06:49:05 -04:00
LynnandMatt Stancliff 74aeb27a26 Remove surplus double quotes
Closes #1877
2014-09-29 06:49:05 -04:00
Manuel MeurerandMatt Stancliff df484c7a2c Improve disable save comment in redis.conf
Closes #1897
2014-09-29 06:49:05 -04:00
antirez de10b8d9cc redis-trib should not abort easily on connection issues. 2014-09-26 16:57:42 +02:00
antirez d4222e6bee DEBUG POPULATE two args form implemented.
The old DEBUG POPULATE form for automatic creation of test keys is:

    DEBUG POPULATE <count>

Now an additional form is available:

    DEBUG POPULATE <count> <prefix>

When prefix is not specified, it defaults to "key", so the keys are
named incrementally from key:0 to key:<count-1>. Otherwise the specified
prefix is used instead of "key".

The command is useful in order to populate different Redis instances
with key names guaranteed to don't collide. There are other debugging
uses, for example it is possible to add additional N keys using a count
of N and a random prefix at every call.
2014-09-25 17:01:56 +02:00
antirez 35a8302b7c Linenoise lib updated. 2014-09-22 14:55:54 +02:00
antirez d61e820faf Fix 03_release_hash.sh commit message. 2014-09-19 19:20:15 +02:00
antirez 5807558c98 03_release_hahs.sh: let me edit before committing. 2014-09-19 19:18:48 +02:00
antirez f3107559be Quick, wrong, fix for create_tarball script. 2014-09-19 19:17:52 +02:00
antirez eba1e652f4 03_release_hash.sh: fix commit stage. 2014-09-19 18:20:47 +02:00
antirez 07bd52113e 02_upload_tarball.sh fixes. 2014-09-19 18:19:19 +02:00
antirez 44acd50a11 First version of release scripts. 2014-09-19 17:44:08 +02:00
Salvatore Sanfilippo 8fe3ecff09 Merge pull request #2014 from mattsta/fix-hiredis-leak
Fix hiredis getaddrinfo leak
2014-09-19 12:49:35 +02:00
Matt Stancliff 56c1f8805e Fix hiredis getaddrinfo leak
Fixed in Redis by 1a5e5b6, but since that part of code
is largely copy/paste from Redis, the fix needs to be
ported over too.

Closes #2012
2014-09-18 14:47:55 -04:00
antirez 747b57cfb6 anetPeerToString(): make unix sockets distinguishable.
Following the CLIENT LIST output format, we prefix the unix socket
address with a "/" so that it is different than an IPv4/6 address.
This makes parsing simpler.

Related to #2010.
2014-09-18 17:37:40 +02:00
antirez 6d53b2f34b anetPeerToString() refactoring and more explicit checks.
Related to PR #2010.
2014-09-18 17:22:22 +02:00
Salvatore Sanfilippo 6d699ea401 Merge pull request #2010 from yossigo/unixdomain-info-string
Unix domain sockets incorrectly listed as IPv6.
2014-09-18 17:21:22 +02:00
Yossi Gottlieb cceb58d233 Unix domain sockets incorrectly listed as IPv6. 2014-09-18 17:07:29 +03:00
antirez 2374496799 Cluster: claim ping_sent time even if we can't connect.
This fixes a potential bug that was never observed in practice since
what happens is that the asynchronous connect returns ok (to fail later,
calling the handler) every time, so a ping is queued, and sent_ping
happens to always be populated.

Howver technically connect(2) with a non blocking socket may return an
error synchronously, so before this fix the code was not correct.
2014-09-17 16:39:41 +02:00
antirez c89afc8e5d Cluster: new option to work with partial slots coverage. 2014-09-17 11:10:09 +02:00
antirez a2c740ea93 Better truncated AOF loading tests.
Now there are tests to write more data after loading a truncated AOF,
testing that the loaded data is correct, appending more, and testing
again.
2014-09-16 11:05:12 +02:00
antirez 878c089ec9 Seek at the end of AOF after truncate call.
It is not clear if files open in append only mode will automatically fix
their offset after a truncate(2) operation. This commit makes sure that
we reposition the AOF file descriptor offset at the end of the file
after a truncated AOF is loaded and trimmed to the last valid command.
2014-09-16 10:57:40 +02:00
antirez 0ba8fe1a43 On AOF end of file, truncate the AOF to last valid command.
Recently we introduced the ability to load truncated AOFs, but
unfortuantely the support was broken since the server, after loading the
truncated AOF, continues appending to the file that is corrupted at the
end. The problem is fixed only in the next AOF rewrite.

This commit fixes the issue by truncating the AOF to the last valid
opcode, and aborting if it is not possible to truncate the file
correctly.
2014-09-16 10:32:58 +02:00
antirez 046a00a03d Don't propagate SAVE.
This is a general fix (check that dirty delta is positive) but actually
should have as the only effect fixing the SAVE propagation to
AOF and slaves.
2014-09-16 10:12:50 +02:00
antirez b82d650afa Cluster test: unit 09, Pub/Sub across the cluster. 2014-09-15 18:11:45 +02:00
antirez 3064e9bd4b Cluster test: unit 08, update messages. 2014-09-15 16:48:26 +02:00
antirez f5efa9bbad Sentinel sentinelGetLeader() top comment improved. 2014-09-11 19:27:45 +02:00
antirez f4be6f16f2 Sentinel: fix computation of total number of votes.
The code to check the number of voters was never updated to follow the new
Sentinel specification, so the number of voters was computed using only
the set of Sentinels that provided a vote.

This means that there is a changing majority on partitions, even if
usually the issue is not triggered because of the configured quorum
check (what was broken was the other implicit check that requires anyway
half of the known sentinels to agree in order to start a failover).
2014-09-11 18:53:31 +02:00
antirez 12b56a969f luaRedisGenericCommand() cached argv handling simplified.
As discussed in issue #1945.
2014-09-10 15:45:59 +02:00
Salvatore Sanfilippo 99871803c8 Merge pull request #1993 from xiaost/limit-scan-iter-count
Limit the *SCAN command `dictScan` iterations
2014-09-10 09:57:54 +02:00
xiaost acfc19633d Limit the *SCAN command dictScan iterations
*SCAN will cause redis server to hang for seconds
after millions of keys was deleted by SCAN/DEL pairs
2014-09-09 17:53:30 +08:00
antirez b892ea70ae Tests for aof-load-truncated = yes. 2014-09-08 10:56:52 +02:00
antirez 9f40c25a08 AOF tests fixed turning aof-load-truncated to no.
When aof-load-truncated option was introduced, with a default of "yes",
the past behavior of the server to abort with trunncated AOF changed, so
we need to explicitly configure the tests to abort with truncated AOF
by setting the option to no.
2014-09-08 10:56:52 +02:00
antirez 31f79a46ff Make aof-load-truncated option actually configurable. 2014-09-08 10:56:52 +02:00
antirez de663966c6 Correct cleanup when aborting AOF loading.
Because of the new ability to start with a truncated AOF, we need
to correctly release all the memory on EOF error. Otherwise there is a
small leak, that is not really a problem, but causes a false positive in
the tests that detect memory leaks.
2014-09-08 10:56:52 +02:00
antirez 132550efc3 AOF ability to load truncated files. 2014-09-08 10:56:52 +02:00
antirez 7b2e5ff9f9 Test AOF format error detection. 2014-09-08 10:56:52 +02:00
antirez 0ab04287aa AOF loading: split handling of format errors from unexpected EOF. 2014-09-08 10:56:52 +02:00
antirez d9f0094154 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2014-09-04 11:46:18 +02:00
antirez 0a6cbabb26 Sentinel: don't set announce-ip if is empty. 2014-09-04 11:45:58 +02:00
antirez c9437fe5a3 Sentinel: clarify announce-ip/port options in sentinel.conf. 2014-09-04 11:30:11 +02:00
antirez cd576a1aab Sentinel: announce ip/port changes + rewrite.
The original implementation was modified in order to allow to
selectively announce a different IP or port, and to rewrite the two
options in the config file after a rewrite.
2014-09-04 11:23:31 +02:00
Dara Kongandantirez 3d939266be sentinel: Decouple bind address from address sent to other sentinels
There are instances such as EC2 where the bind address is private
(behind a NAT) and cannot be accessible from WAN.

https://groups.google.com/d/msg/redis-db/PVVvjO4nMd0/P3oWC036v3cJ
2014-09-04 10:54:21 +02:00
Salvatore Sanfilippo 0019479f86 Merge pull request #1982 from mattsta/fix-getrange-32bit
Increase size of range request in getrange
2014-09-03 15:10:35 +02:00
Matt Stancliff b20df972ed Return empty string if GETRANGE of empty string
Previously, GETRANGE of a key containing nothing ("")
would allocate a large (size_t)-1 return value causing
crashes on 32bit builds when it tried to allocate the
4 GB return string.
2014-09-02 18:56:28 -04:00
Matt Stancliff f0e306f4a0 Increase size of range request in getrange
32 bit builds don't have a big enough long to capture
the same range as a 64 bit build.  If we use "long long"
we get proper size limits everywhere.

Also updates size of unsigned comparison to fit new size of `end`.

Fixes #1981
2014-09-02 18:56:01 -04:00
Matt Stancliffandantirez 67e414c7b8 Sentinel: Abort Hello quicker if not connected
We can save a little work by aborting when we enter the function
if we're disconnected.
2014-09-01 16:34:06 +02:00
Paddy Byersandantirez a23502e5e8 Add regression test for issue #1939 2014-09-01 10:42:27 +02:00
Paddy Byersandantirez 86cde329d1 Store the length of the static argv when first allocated. 2014-09-01 10:42:27 +02:00
antirez 91e6b4d74e Fix dictRehash assert casting type.
Also related to #1929.
2014-08-26 10:32:44 +02:00
antirez 72aa797c89 Cast to right type in dictNext().
This closes issue #1929, the other part was fixed in the context of issue
2014-08-26 10:26:36 +02:00
antirez 064d5c96ac Use long for rehash and iterator index in dict.h.
This allows to support datasets with more than 2 billion of keys
(possible in very large memory instances, this bug was actually
reported).

Closes issue #1814.
2014-08-26 10:18:56 +02:00
antirez 293348d0de Missing va_end also added in hiredis copy of sds.c. 2014-08-26 10:05:32 +02:00
Salvatore Sanfilippo 26744da0d1 Merge pull request #1964 from mariano-perez-rodriguez/patch-2
Missing assert removal from sdsIncrLen()
2014-08-26 10:03:54 +02:00
Mariano Pérez Rodríguez 8bbc1897c4 Missing assert removal from sdsIncrLen()
Companion for 8eeb1802ec, but dealing with hiredis.
2014-08-25 15:24:15 -03:00
antirez 119a7000b5 Use modern typedef form in cluster.h. 2014-08-25 10:42:18 +02:00
antirez 8eeb1802ec No longer useful assert removed from sdsIncrLen(). 2014-08-25 10:36:26 +02:00
antirez 209f266cc5 Merge branch '1906-merge' into unstable 2014-08-25 10:27:53 +02:00
Matt Stancliffandantirez 1f8a6d806c Fix redis-benchmark abort output 2014-08-25 10:26:30 +02:00
Matt Stancliffandantirez 6b099c7de6 Deny CLIENT command in scripts
We don't want scripts doing CLIENT SETNAME
or CLIENT KILL or CLIENT LIST or CLIENT PAUSE.

Originally reported by Chris Wj then proper
action inspired by Itamar Haber.

Reference: https://groups.google.com/forum/#!topic/redis-db/09B2EYwyVgk
2014-08-25 10:25:30 +02:00
Matt Stancliffandantirez aa404fc11b redis-benchmark: Remove unused buffer
This just deletes old code that didn't get removed when
logic changed.  We were setting offsets that never
got read anywhere.

Since clients are now just cloned, we don't need to track
per-client buffer offsets anywhere because they are all
the same from the original client.
2014-08-25 10:19:43 +02:00
CharSyamandantirez b34a8c3b75 redis-benchmark: add auth option
Closes #1097
2014-08-25 10:17:33 +02:00
Matt Stancliffandantirez 7e63dd23f3 Rename two 'buf' vars to 'ip' for better clarity
Clearly ip[32] is wrong, but it's less clear that buf[32] was wrong
without further reading.
2014-08-25 10:16:20 +02:00
Eiichi Satoandantirez c38884ceac Sentinel: fix bufsize to support IPv6 address
Closes #1914
2014-08-25 10:15:43 +02:00
Jan-Erik Redigerandantirez ef57f94df0 Use correct github url to find commands.json
Once this is merged:
  - merge the latest changes to commands.json in antirez/redis-doc
  - re-run: utils/generate-command-help.rb > src/help.h

Then we'll have nice and easy tab-completed help in redis-cli again.

Closes #1909
2014-08-25 10:15:28 +02:00
Mariano Pérez Rodríguezandantirez 5afe1e37c7 Stop tests from leaving a black background
Uses ANSI "default background" color code after closing tests
so any non-black terminals don't remain polluted.

Fixes #1649
Closes #1912
2014-08-25 10:14:03 +02:00
Matt Stancliffandantirez 681de88df4 Sentinel: Reject config from STDIN
Sentinel needs to die with a more accurate error message
when attempted to open a config from STDIN.

See: https://groups.google.com/forum/#!topic/redis-db/sYx7VNMWaNM
2014-08-25 10:13:34 +02:00
Matt Stancliffandantirez 60c448b584 Cluster: Fix segfault if cluster config corrupt
This commit adds a size check after initial config
line parsing to make sure we have *at least* 8 arguments
per line.

Also, instead of asserting for cluster->myself, we just test
and error out normally (since the error does a hard exit anyway).

Closes #1597
2014-08-25 10:11:38 +02:00
Matt Stancliffandantirez 879e18b7ec Fix memory leak in cluster config parsing
The continue stop us from triggering the
free after the long line for loop, so add it
earlier.
2014-08-18 11:27:19 +02:00
Matt Stancliffandantirez 82bac1b729 Remove unused global variable
It has an important name, but nothing uses it...
2014-08-18 11:23:58 +02:00
antirez a6edfceaa8 Fix invalid expire error for SET family commands. 2014-08-18 11:15:50 +02:00
Cong Dingandantirez 18ca831830 Remove unused function
Closes #878
2014-08-18 11:12:26 +02:00
David Palmandantirez 100c3315be Extend range of bytesToHuman to TB and PB
Also adds a fallthrough case for when given
large values (like overflow numbers of 2^64 by mistake).

Closes #858
2014-08-18 11:11:00 +02:00
yoavandantirez 0a98b21f65 Add error check for writing RDB checksum
Closes #857
2014-08-18 11:09:06 +02:00
Dov Murikandantirez 233d24a737 redis-cli: fix prompt after shutdown command
Fix redis-cli prompt to state "not connected" after a SHUTDOWN command
is sent.
2014-08-18 11:05:36 +02:00
xiaoyuandantirez d786fb6e94 Clarify argument to dict macro
d is more clear because the type of argument is dict not dictht

Closes #513
2014-08-18 10:59:01 +02:00
Matt Stancliffandantirez 6a7a32a806 Clarify existing slot wording on cluster start 2014-08-18 10:58:00 +02:00
antirez c951c3ee5a Fix undefined behavior in ll2string().
The bug was found by @CAFxX, thanks!
See issue #1940.
2014-08-15 15:48:15 +02:00
antirez edca2b14d2 Remove warnings and improve integer sign correctness. 2014-08-13 11:44:38 +02:00
antirez 7e3ccddc6d Add -W to compilation flags. 2014-08-13 10:53:54 +02:00
antirez 68db7b1f56 Use unsigned integers in SDS header.
This raises the max string to 4GB without any downside.
2014-08-13 10:53:51 +02:00
Pieter Noordhuisandantirez cf85b5ba81 va_copy must be matched by va_end
Hat tip to @rfuchs. See: https://github.com/redis/hiredis/pull/178.

Fixes #1187
2014-08-12 11:50:02 +02:00
charsyamandantirez 3426866e0a Remove duplicate prototypes in redis.h
Also moves acceptHandler() to be near the other accept...() functions.

Closes #1105
2014-08-12 11:45:25 +02:00
NanXiaoandantirez 951eefae3c Use resolv library in Solaris
For some Solaris flavours, the inet_aton in in resolv library.
Not linking this library will introduce link error.

Improves compatability with older Solaris and still
works on new Solaris.

Closes #1092
2014-08-12 11:30:46 +02:00
Daniel Priceandantirez b57a4d07a2 Define AR to help with lua cross-compilation
Closes #997
2014-08-12 11:26:34 +02:00
antirez ded57795ff representRedisNodeFlags() moved into right code section.
The funciton was also modified in order to be more standalone and
produce an output without trailing spaces, making the reuse simpler.
The global variable was renamed in cammel case as most other Redis
globals, except the main ones we refer too many times, like 'server'.
2014-08-08 15:53:42 +02:00
charsyamandantirez de5465baf7 Refactor cluster flag printing
Less copy/paste code duplication.

Closes #952
2014-08-08 15:39:44 +02:00
Matt Stancliffandantirez da0132638f Cleanup double semicolons
Closes #1161
2014-08-08 14:54:02 +02:00
SungBin_Hongandantirez dec58464d8 Free memory in clusterLoadConfig error handler
Closes #1327
2014-08-08 14:40:32 +02:00
Chris Lambandantirez af62d9a687 redis-check-dump: Prevent segfault if can't malloc
Found by The Mayhem Team (Alexandre Rebert, Thanassis Avgerinos,
Sang Kil Cha, David Brumley, Manuel Egele) Cylab, Carnegie Mellon
University. See http://bugs.debian.org/716259 for more.

Signed-off-by: Chris Lamb <lamby@debian.org>

Fixes #1191
2014-08-08 14:10:34 +02:00
charsyamandantirez 354119e4c5 Remove unused LINE_BUFLEN definition
Closes #1129
2014-08-08 14:05:02 +02:00
Matt Stancliffandantirez 498ad7482b Reject MOVE to non-integer DBs
Previously, "MOVE key somestring" would move the key to
DB 0 which is just unexpected and wrong.
String as DB == error.

Test added too.

Modified by @antirez in order to use the getLongLongFromObject() API
instead of strtol().

Fixes #1428
2014-08-08 12:38:43 +02:00
Matt Stancliffandantirez 25791550e0 pubsub: Return integers for NUMSUB, not strings
Also adds test for numsub — due to tcl being tcl,
it doesn't capture the "numberness" of the fix,
but now we at least have one test case for numsub.

Closes #1561
2014-08-08 11:19:37 +02:00
Xiaojie Zhangandantirez 4bb6844e43 Avoid unnecessary decoding in ziplist.c
Closes #1519
2014-08-08 11:04:54 +02:00
Samuel Chouandantirez 7e9f24d694 Improve accuracy of HAVE_ATOMIC dependency check
[I had to split out the clang check due to
 clang *really* not liking the __GLIBC_PREREQ macro; -matt]

Closes #1456
2014-08-08 10:46:57 +02:00
Jim Howardandantirez a8d3e930bc Remove redundant event loop fd processing
Reason:
https://github.com/antirez/redis/pull/1450#issuecomment-30432925

Closes #1450
2014-08-08 10:43:54 +02:00
Xian Liandantirez f510cfb4ef memtest: Add missing free()
Closes #1614
2014-08-08 10:30:14 +02:00
Joshua Andersonandantirez 7a2450a7e5 src/Makefile: Reword "to run make test" message
Closes #1610
2014-08-08 10:20:22 +02:00
cubicdaiyaandantirez 23f08510d5 Use 'void' for zero-argument functions
According to the C standard,
it is desirable to give the type 'void'
to functions have no argument.

Closes #1631
2014-08-08 10:05:32 +02:00
Matt Stancliffandantirez 8db020e2a1 Fix assert technical correctness
dictAdd returns DICT_OK, not REDIS_OK. They both
have the same underlying values, so it works even though
the code is technically wrong.

Fixes #1512
2014-08-08 10:03:22 +02:00
Jan-Erik Redigerandantirez 6df1251baa install_server.sh: add missing bang
This was discovered by _bodya and reported in the IRC channel.
Everything worked fine as these scripts are always executed as shell
scripts.

Closes #1728
2014-08-07 17:14:08 +02:00
Edgars Irmejsandantirez 67c4fbed82 Change unixsocketperm comment to 700 from 755
According to unix manuals, "Connecting to the socket object requires
read/write permission." -- mode 755 is useless for anybody
other than the owner.

Fixes #1696
2014-08-07 17:13:01 +02:00
ripcurld00dandantirez 9b4efe6cf4 redis-check-dump: use names instead of magic nums
Use constants to avoid magic numbers in `types`, which is an array
that stores the names of `REDIS` types.

Closes #1681
2014-08-07 17:11:19 +02:00
Kashif Rasulandantirez c49378fe3e Fix issues raised by clang analyzer
Modified by @antirez since the original fix to genInfoString() looked
weak. Probably the clang analyzer complained about `section` being
possibly NULL, and strcasecmp() called with a NULL pointer. In the
practice this can never happen, still for the sake of correctness
the right fix is not to modify only the first call, but to set `section`
to the value of "default" if it happens to be NULL.

Closes #1660
2014-08-07 17:08:11 +02:00
antirez 7bb25f8a46 Force quit when receiving a second SIGINT.
Also quit ASAP when we are still loading a DB, since care is not needed
in this special condition, especially for a SIGINT.
2014-08-07 16:39:02 +02:00
Matt Stancliffandantirez d99a7246f8 Add graceful exit when Ctrl-C is received 2014-08-07 16:21:15 +02:00
Matt Stancliffandantirez c82e0b701f Fix intset midpoint selection
The classic (min+max)/2 is provably unsafe.  Fixed
as recommended in research:
http://googleresearch.blogspot.com/2006/06/extra-extra-read-all-about-it-nearly.html

Fix inspired by @wjin, but I used a different approach.
(later, I found @kuebler fixed the same issue too).

Fixes #1741, #1602
2014-08-07 16:17:37 +02:00
Matt Stancliffandantirez 87815ab5ba Fix key extraction for SORT
We only want to use the last STORE key, but we have to record
we actually found a STORE key so we can increment the final return
key count.

Test added to prevent further regression.

Closes #1883, #1645, #1647
2014-08-07 16:14:15 +02:00
Jan-Erik Redigerandantirez 53fdfda9e3 Handle large getrange requests
Previously the end was casted to a smaller type
which resulted in a wrong check and failed
with values larger than handled by unsigned.

Closes #1847, #1844
2014-08-07 12:40:44 +02:00
Matt Stancliffandantirez f17f8521f0 scripting: no eval with negative key count
Negative key count causes segfault in Lua functions.

Fixes #1842
Closes #1843
2014-08-07 12:38:36 +02:00
kingsumosandantirez 1a5e5b6bd4 cluster: fix node connection memory leak
Cluster leaks memory while connecting due to missing freeaddrinfo()

(Commit modified by @antirez. The freeaddrinfo() call was misplaced so
 in case of no address was bound, the memory leak was still there).

Closes #1801
2014-08-07 12:35:03 +02:00
Jan-Erik Redigerandantirez 05676c5d16 redis-cli: fix latency result output
(Cleaned up a little by @mattsta)

Closes #1774
2014-08-07 12:26:45 +02:00
Wei Jinandantirez 5b3930655f Remove redundant else/return block
Fixes #1741
2014-08-07 12:20:57 +02:00
antirez 9756f61fda Clarify AIX "#undef hz". 2014-08-07 12:14:27 +02:00
siahlandantirez 009cabf103 Add support for compiling on AIX
Closes #1900
2014-08-07 12:12:54 +02:00
Matt Stancliffandantirez 426553164d redis-cli: Add --no-raw option
Some people need formatted output even when they have no
interactive tty.

Fixes #760
2014-08-07 12:07:01 +02:00
Matt Stancliffandantirez 0042fb0eb0 redis-cli: stop showing incorrectly selected DB
Previously redis-cli would happily show "-1" or "99999"
as valid DB choices.

Now, if the SELECT call returned an error, we don't update
the DB number in the CLI.

Inspired by @anupshendkar in #1313

Fixes #566, #1313
2014-08-07 12:04:33 +02:00
Matt Stancliffandantirez bbc1cd0bd9 redis-cli: Re-attach selected DB after auth
Previously, if you did SELECT then AUTH, redis-cli
would show your SELECT'd db even though it didn't
happen.

Note: running into this situation is a (hopefully) very limited
used case of people using multiple DBs and AUTH all at the same
time.

Fixes antirez#1639
2014-08-07 11:58:28 +02:00
Salvatore Sanfilippo 78a012d81a Merge pull request #1896 from yossigo/bgsave-signal-fix
Fail SYNC if background save child aborted due to a signal.
2014-07-28 14:45:31 +02:00
Yossi Gottlieb a75a574141 Fail SYNC if background save child aborted due to a signal. 2014-07-28 14:43:30 +03:00
vpsandantirez c90af7cd14 * fixed doc URL for keyspace events 2014-07-28 12:49:33 +02:00
antirez f231edc672 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2014-07-28 12:49:04 +02:00
antirez ddd9fd417e Cluster test: unit 07, replicas migration. 2014-07-28 12:48:51 +02:00
antirez 75b92fe7c4 Cluster test: unit 06 cluster creation comment fixed. 2014-07-28 12:35:05 +02:00
Salvatore Sanfilippo d76c663dfe Merge pull request #1895 from mattrobenolt/patch-1
Fix spelling of "stand alone" in ascii art
2014-07-28 09:37:41 +02:00
Matt Robenolt 0f8df3df41 Fix spelling of "stand alone" in ascii art
Also unified the logic to match `genRedisInfoString`
2014-07-28 00:30:12 -07:00
antirez 351ad518b3 Cluster test: node reachability condition fixed. 2014-07-25 16:17:13 +02:00
antirez 628c5641c0 Cluster test: init test, be patient during restarts. 2014-07-25 16:10:18 +02:00
antirez 0d35b46761 Cluster test: 03 unit speedup, don't send WAIT to slaves. 2014-07-25 16:09:31 +02:00
antirez ad87ec4fb7 Cluster test: unit 06, slaves with stale data can't failover. 2014-07-25 15:40:25 +02:00
antirez 49c817c250 Example redis.conf: improve slaveof description. 2014-07-25 14:20:23 +02:00
antirez c22fc1ac6f Cluster test: new unit 05, slave selection. 2014-07-25 11:23:51 +02:00
antirez 0d9bcb1c12 Cluster: don't migrate to a master that never had slaves.
Replica migration algorithm modified so that slaves never try to migrate
to masters that were never configured to have slaves in the past.
We want the algorithm to take care of masters that remained without
*working* slaves, but that used to have slaves according to the cluster
configuration.
2014-07-25 11:02:09 +02:00
antirez 7eb47ae290 Fix util.c compilation by including stdint.h. 2014-07-23 18:02:02 +02:00
antirez 0ce352c19f Faster ll2string() implementation.
Based on ideas documented in this blog post:

https://www.facebook.com/notes/facebook-engineering/three-optimization-tips-for-c/10151361643253920

The original code was modified to handle signed integers, reformetted to
fit inside the Redis code base, and was stress-tested with a program
in order to validate the implementation against snprintf().

Redis was measured to be measurably faster from the point of view of
clients in real-world operations because of this change, since sometimes
number to string conversion is used extensively (for example every time
a GET results into an integer encoded object to be returned to the
user).
2014-07-23 14:58:23 +02:00
antirez 0adf4482f0 PFSELFTEST: less false positives.
This is just a quickfix, for the nature of the test the right way to fix
it is to average the error of N runs, since otherwise it is always
possible to get a false positive with a bad run, or to minimize too much
this possibility we may end testing with too much "large" error ranges.
2014-07-23 11:43:57 +02:00
antirez b9bb7e2d4a Test: check sorted set elements order after union.
This is not a regression but issue #1786 showed the need for this test.
2014-07-22 17:52:04 +02:00
antirez d74e422b5e Remove useless var and check in zunionInterGenericCommand(). 2014-07-22 17:38:22 +02:00
antirez 119813e968 ZUNIONSTORE reimplemented for speed.
The user @kjmph provided excellent ideas to improve speed of ZUNIONSTORE
(in certain cases by many order of magnitude), together with an
implementation of the ideas.

While the ideas were sounding, the implementation could be improved both
in terms of speed and clearness, so that's my attempt at reimplementing
the speedup proposed, trying to improve by directly using just a
dictionary with an embedded score inside, and reusing the single-pass
aggregate + order-later approach.

Note that you can't apply this commit without applying the previous
commit in this branch that adds a double in the dictEntry value union.

Issue #1786.
2014-07-22 17:38:22 +02:00
antirez d1cb6a0fc4 Add double field in dict.c entry value union. 2014-07-22 17:38:22 +02:00
antirez 89af463124 CLUSTER RESET: Flush dataset if node is a slave.
For non-empty masters, CLUSTER RESET is denied, and the user requires to
start to reset a node by explicitly clearing it with FLUSHALL.
However CLUSTER RESET when executed with slaves don't have this
restrictions since data is just a replica of the master, and with
read-only slaves it is also not possible to remove the data set. However
the node was turned from slave to master after a reset, without touching
the old slave data. This is 99.99% of times not appropriate and forces
full resets to follow this path to work with both slave and master
nodes:

    FLUSHALL
    CLUSTER RESET HARD
    FLUSHALL

Since we need the first flushall for masters, and the second for slaves.

This commit changes the behavior so that CLUSTER RESET removes the data set
of a slave node during a reset, in the moment it gets turned into a master,
so the new pattern is simply:

    FLUSHALL (that may fail for slaves)
    CLUSTER RESET
2014-07-22 15:29:57 +02:00
antirez 10088d5087 Cluster test: use larger keyspace in resharding test. 2014-07-21 16:29:49 +02:00
antirez 54f1c682e6 Cluster test: unit 04, consistency during resharding. 2014-07-21 16:26:05 +02:00
antirez 15336ffe86 Cluster: fix redis-trib --from all. 2014-07-21 15:30:25 +02:00
antirez 73a809b159 redis-trib: allow to reshard in non-interactive way.
The introduction of --from --to --slots --yes options allow to reshard
from cli in an automated way from scripts. The code is ugly and needs
refactoring as soon as we get it in RC / stable release.
2014-07-18 17:43:49 +02:00
antirez 8d4ff87b26 Cluster test: test that writes are retained during failovers.
In the test we use WAIT when the master and slave are up, and only later the
partition is created killing the master, so we are sure we don't incur
in failure modes that may lose writes in this test: the goal here is to
make sure that the elected slave was replicating correctly with the
master.
2014-07-18 17:13:46 +02:00
antirez 509c888dfa Test: Pub/Sub PING. 2014-07-18 12:03:31 +02:00
antirez 420584a46d Test: small integer sharing depends on maxmemory policy. 2014-07-18 10:55:08 +02:00
antirez 6b9b958e00 PING: backward compatible error for wrong number of args. 2014-07-18 10:15:51 +02:00
antirez b3c042cdb2 tryObjectEncoding(): use shared objects with maxmemory and non-LRU policy.
In order to make sure every object has its own private LRU counter, when
maxmemory is enabled tryObjectEncoding() does not use the pool of shared
integers. However when the policy is not LRU-based, it does not make
sense to do so, and it is much better to save memory using shared
integers.
2014-07-18 10:09:51 +02:00
antirez 27839e5ecb Variadic PING with support for Pub/Sub.
PING can now be called with an additional arugment, behaving exactly
like the ECHO command. PING can now also be called in Pub/Sub mode (with
one more more subscriptions to channels / patterns) in order to trigger
the delivery of an asynchronous pong message with the optional payload.

This fixes issue #420.
2014-07-16 17:47:17 +02:00
antirez 59cf0824d9 PubSub clients refactoring and new PUBSUB flag.
The code tested many times if a client had active Pub/Sub subscriptions
by checking the length of a list and dictionary where the patterns and
channels are stored. This was substituted with a client flag called
REDIS_PUBSUB that is simpler to test for. Moreover in order to manage
this flag some code was refactored.

This commit is believed to have no effects in the behavior of the
server.
2014-07-16 17:34:07 +02:00
antirez c7822bf382 RDB: load string objects directly as EMBSTR objects when possible. 2014-07-16 11:36:22 +02:00
Salvatore Sanfilippo 9e7e0cd613 Merge pull request #1870 from michael-grunder/object_arity
Fix OBJECT arity
2014-07-14 15:37:33 +02:00
michael-grunder ce8a68b1fd Fix OBJECT arity
Previously, the command definition for the OBJECT command specified
a minimum of two args (and that it was variadic), which meant that
if you sent this:

OBJECT foo

When cluster was enabled, it would result in an assertion/SEGFAULT
when Redis was attempting to extract keys.

It appears that OBJECT is not variadic, and only ever takes 3 args.

https://gist.github.com/michael-grunder/25960ce1508396d0d36a
2014-07-11 13:03:26 -07:00
antirez 6382936ebc Fix typo in LATENCY DOCTOR output. 2014-07-11 10:57:28 +02:00
antirez 3bf72d0d5f Test: more reliable AOF rewrite test under write load. 2014-07-10 16:42:43 +02:00
antirez a94aa39c86 LATENCY command / monitor basic tests. 2014-07-10 16:23:54 +02:00
antirez 0dcc754f68 LATENCY HISTORY returns empty array if no data is available.
Previously we returned an error.
2014-07-10 16:20:40 +02:00
antirez 79d2f70f5c Test: enable latency monitor in the default config.
This way as a side effect of running the test we also stress the latency
monitor data collection.
2014-07-10 15:04:34 +02:00
antirez e01195e90d Test: AOF rewrite during write load. 2014-07-10 11:25:12 +02:00
antirez 067e365042 Fixed a few missing newline in createLatencyReport(). 2014-07-09 16:45:40 +02:00
antirez fda2b30346 createLatencyReport(): compile before commit avoids commits. 2014-07-08 17:17:08 +02:00
antirez 924dc9731f createLatencyReport(), fix mount option name. 2014-07-08 17:12:21 +02:00
antirez 51e01c0450 createLatencyReport(): initialize all advices to zero. 2014-07-08 17:11:56 +02:00
antirez 7fb90a670e LATENCY DOCTOR first implementation complete. 2014-07-08 17:05:56 +02:00
antirez 57e931debc Fix analyzeLatencyForEvent() MAD computation. 2014-07-08 12:56:45 +02:00
antirez 2a232dfa9a LATENCY DOCTOR: initial draft and events summary output. 2014-07-08 11:31:46 +02:00
antirez 19853db892 Latency: low level time series analysis implemented. 2014-07-07 15:00:01 +02:00
antirez b2604dc58c LATENCY SAMPLES renamed LATENCY HISTORY. 2014-07-07 12:35:24 +02:00
antirez f8934657b2 LATENCY RESET implemented. 2014-07-07 12:34:54 +02:00
antirez 2de5bab368 Better "final read from parent" algorithm in rewriteAppendOnlyFile*(.
We now wait up to 1 second for diff data to come from the parent,
however we use poll(2) to wait for more data, and use a counter of
contiguous failures to get data for N times (set to 20 experimentally
after different tests) as an early stop condition to avoid wasting 1
second when the write traffic is too low.
2014-07-05 15:42:24 +02:00
antirez 895409ca78 Log AOF diff sizes in MBs instead of bytes. 2014-07-05 12:37:44 +02:00
antirez 93cd75bbe0 aofChildWriteDiffData() better handling of free blocks. 2014-07-05 01:11:28 +02:00
antirez ac33c049c6 redis-benchmark: abort when all clients are disconnected. 2014-07-04 17:52:18 +02:00
antirez 20a0959964 Fine tuning of aofReadDiffFromParent() calls trigger. 2014-07-04 17:51:03 +02:00
antirez cc24447a1a Use amount of I/O not ops to trigger aofReadDiffFromParent(). 2014-07-04 16:03:46 +02:00
antirez 62f9d34c56 aofChildWriteDiffData(): write as much as possible. 2014-07-04 15:54:20 +02:00
antirez 30c42f12b0 Handle write failure in AOF parent -> child ACK. 2014-07-04 15:37:49 +02:00
antirez 75ef41058d Use a timeout when reading parent ack from AOF child. 2014-07-04 15:35:01 +02:00
antirez 2fa9fb647f Send AOF diffs from parent to child to improve latency. 2014-07-04 15:27:23 +02:00
antirez 57852ff207 Use fsync instead of aof_fsync in final AOF sync.
This happens in the child process so we don't care about latency:
better to sync metadata as well.
2014-07-04 11:51:30 +02:00
antirez a99766fda1 Fixed conditional for aof-write-pending-fsync latency event selection. 2014-07-02 17:42:29 +02:00
antirez f0f9d583ad Dependencies updated. 2014-07-02 17:04:14 +02:00
antirez 1f665d6e82 Cast void* to char* to avoid waring in latencyCommand(). 2014-07-02 16:56:08 +02:00
antirez e3c0125b70 Properly initialize min/max in latency.c. 2014-07-02 16:55:34 +02:00
antirez a887af34e1 latencyStartMonitor() modified to avoid warnings. 2014-07-02 16:53:44 +02:00
antirez 53ae687d59 Latency monitor: specialize delayed aof writes events. 2014-07-02 16:45:45 +02:00
antirez a53c734094 LATENCY GRAPH: filling under the curve is more readable. 2014-07-02 16:37:53 +02:00
antirez aa16f87b87 LATENCY GRAPH implemented. 2014-07-02 16:31:22 +02:00
antirez 6f20482a86 latencyTimeSeries structure max field type fixed. 2014-07-02 16:14:28 +02:00
antirez 9d4d810861 Free labels in freeSparklineSequence(). 2014-07-02 12:49:14 +02:00
antirez 1766d91697 LATENCY LATEST: add the max field. 2014-07-02 12:40:38 +02:00
antirez e173f7a0e3 Latency monitor trheshold value is now configurable.
This commit adds both support for redis.conf and CONFIG SET/GET.
2014-07-02 12:28:17 +02:00
antirez cc4df5a6b8 ASCII sparklines generation API. 2014-07-02 10:13:53 +02:00
antirez ed4980243a License added to latency.h. 2014-07-02 10:06:58 +02:00
antirez b809676a9e Latency monitor turned off by default.
It is not a good idea to bloat the code with gettimeofday() calls if the
instance is working well, and turning monitoring on at runtime is a
joke.
2014-07-01 17:23:59 +02:00
antirez de88bc63d5 Latency monitor: more hooks around the code. 2014-07-01 17:19:08 +02:00
antirez f35abe2ff5 Latency monitor: don't add new samples in the same second.
Instead we update the old sample with the new latency if it is greater.
2014-07-01 17:12:09 +02:00
antirez 83beaa886c LATENCY LATEST implemented. 2014-07-01 16:17:33 +02:00
antirez 753b707d2a Latency monitor: command duration is in useconds. Convert. 2014-07-01 16:09:02 +02:00
antirez 551bee86b4 LATENCY SAMPLES implemented. 2014-07-01 16:07:13 +02:00
antirez 8612e6de88 Latency monitor: collect slow commands.
We introduce the distinction between slow and fast commands since those
are two different sources of latency. An O(1) or O(log N) command without
side effects (can't trigger deletion of large objects as a side effect of
its execution) if delayed is a symptom of inherent latency of the system.

A non-fast command (commands that may run large O(N) computations) if
delayed may just mean that the user is executing slow operations.

The advices LATENCY should provide in this two different cases are
different, so we log the two classes of commands in a separated way.
2014-07-01 11:47:08 +02:00
antirez d7a07a2012 Latency monitor: basic samples collection. 2014-07-01 11:30:15 +02:00
antirez 0afb7a48c0 Fix Solaris compilation due to ctime_r() call.
Introduced in Redis 2.8.10 because of a change in Sentinel.
This closes issue #1837.
2014-06-30 16:29:12 +02:00
antirez 54157bc49e Test: find_available_port: check that cluster port is free as well.
The function will only return ports that have also port+10000 free, so
that Redis Cluster instances can be executed at the returned port.
2014-06-30 12:08:24 +02:00
antirez 612bc784fc Test: fix instances.tcl restart_instance abort error. 2014-06-30 12:06:27 +02:00
antirez 683f41adf2 DEBUG CMDKEYS moved to COMMAND GETKEYS. 2014-06-27 12:22:15 +02:00
antirez 885b6fc577 COMMAND COUNT subcommand added. 2014-06-27 12:11:15 +02:00
antirez a92ae77740 COMMAND: fix argument parsing.
This fixes detection of wrong subcommand (that resulted in the default
all-commands output instead) and allows COMMAND INFO to be called
without arguments (resulting into an empty array) which is useful in
programmtically generated calls like the following (in Ruby):

    redis.commands("command","info",*mycommands)

Note: mycommands may be empty.
2014-06-27 12:05:54 +02:00
antirez 7fd0149d34 COMMANDS command renamed COMMAND. 2014-06-27 12:01:29 +02:00
antirez 9bf6921f3d COMMANDS command: remove static + aesthetic changes.
Static was removed since it is needed in order to get symbols in stack
traces. Minor changes in the source code were operated to make it more
similar to the existing Redis code base.
2014-06-27 11:59:48 +02:00
Matt Stancliffandantirez 183458f76a Cluster: Add COMMANDS command
COMMANDS returns a nested multibulk reply for each
command in the command table.  The reply for each
command contains:
  - command name
  - arity
  - array of command flags
  - start key position
  - end key position
  - key offset step
  - optional: if the keys are not deterministic and
    Redis uses an internal key evaluation function,
    the 6th field appears and is defined as a status
    reply of: REQUIRES ARGUMENT PARSING

Cluster clients need to know where the keys are in each
command to implement proper routing to cluster nodes.

Redis commands can have multiple keys, keys at offset steps, or other
issues where you can't always assume the first element after
the command name is the cluster routing key.

Using the information exposed by COMMANDS, client implementations
can have live, accurate key extraction details for all commands.

Also implements COMMANDS INFO [commands...] to return only a
specific set of commands instead of all 160+ commands live in Redis.
2014-06-27 11:54:26 +02:00
antirez 1f0c0df4a8 Fixed assert conditional in ROLE command test. 2014-06-26 22:13:46 +02:00
antirez 95b1979c32 No more trailing spaces in Redis source code. 2014-06-26 18:48:40 +02:00
antirez 97f1fc65cf CLIENT KILL: don't kill the master as a normal client.
Technically the problem is due to the client type API that does not
return a special value for the master, however fixing it locally in the
CLIENT KILL command is better currently because otherwise we would
introduce a new output buffer limit class as a side effect.
2014-06-26 18:43:09 +02:00
antirez 42231117a7 Remove infinite loop from PSYNC test.
Added for debugging and forgot there.
2014-06-26 18:30:03 +02:00
antirez e7887e6060 Test: hopefully more robust PSYNC test.
This is supposed to fix issue #1417, but we'll know if this is enough
only after a couple of runs of the CI test without false positives.
2014-06-26 16:00:27 +02:00
Salvatore Sanfilippo f86798ba6b Merge pull request #1838 from mattsta/powerpc-fixes
PowerPC compile-time improvements
2014-06-26 15:13:49 +02:00
Matt Stancliff a3e7a665ad Allow __powerpc__ to define HAVE_ATOMIC too
From mailing list post https://groups.google.com/forum/#!topic/redis-db/D3k7KmJmYgM

In the file “config.h”, the definition HAVE_ATOMIC is used to indicate
if an architecture on which redis is implemented supports atomic
synchronization primitives.  Powerpc  supports atomic synchronization
primitives, however, it is not listed as one of the architectures
supported in config.h. This patch  adds the __powerpc__ to the list of
architectures supporting these primitives. The improvement of redis
due to the atomic synchronization on powerpc is significant,
around 30% to 40%, over the default implementation using pthreads.

This proposal adds __powerpc__ to the list of architectures designated
to support atomic builtins.
2014-06-26 08:55:47 -04:00
Matt Stancliff a953c88381 Allow atomic memory count update with C11 builtins
From mailing list post https://groups.google.com/forum/#!topic/redis-db/QLjiQe4D7LA

In zmalloc.c the following primitives are currently used
to synchronize access to single global variable:
__sync_add_and_fetch
__sync_sub_and_fetch

In some architectures such as powerpc these primitives are overhead
intensive. More efficient C11 __atomic builtins are available with
newer GCC versions, see
http://gcc.gnu.org/onlinedocs/gcc-4.8.2/gcc/_005f_005fatomic-Builtins.html#_005f_005fatomic-Builtins

By substituting the following  __atomic… builtins:
__atomic_add_fetch
__atomic_sub_fetch

the performance improvement on certain architectures such as powerpc can be significant,
around 10% to 15%, over the implementation using __sync builtins while there is only slight uptick on
Intel architectures because it was already enforcing Intel Strongly ordered memory semantics.

The selection of __atomic built-ins can be predicated on the definition of ATOMIC_RELAXED
which Is available on in gcc 4.8.2 and later versions.
2014-06-26 08:52:53 -04:00
Matt Stancliff f9bca13a1a Use predefined macro for used_memory() update 2014-06-26 08:51:13 -04:00
antirez 9be3ee8283 Make unstable branch version unique and distinguishable. 2014-06-25 15:30:34 +02:00
antirez 75c57d53ea CLUSTER SLOTS: don't output failing slaves.
While we have to output failing masters in order to provide an accurate
map (that may be the one of a Redis Cluster in down state because not
all slots are served by a working master), to provide slaves in FAIL
state is not a good idea since those are not necesarely needed, and the
client will likely incur into a latency penalty trying to connect with a
slave which is down.

Note that this means that CLUSTER SLOTS does not provide a *complete*
map of slaves, however this would not be of any help since slaves may be
added later, and a client that needs to scale reads and requires to
stay updated with the list of slaves, need to do a refresh of the map
from time to time, anyway.
2014-06-25 15:19:35 +02:00
antirez a6fe4ca321 CLUSTER SLOTS: main loop should skip only slaves and zero slot masters. 2014-06-25 15:08:33 +02:00
Matt Stancliffandantirez e14829de30 Cluster: Add CLUSTER SLOTS command
CLUSTER SLOTS returns a Redis-formatted mapping from
slot ranges to IP/Port pairs serving that slot range.

The outer return elements group return values by slot ranges.

The first two entires in each result are the min and max slots for the range.

The third entry in each result is guaranteed to be either
an IP/Port of the master for that slot range - OR - null
if that slot range, for some reason, has no master

The 4th and higher entries in each result are replica instances
for the slot range.

Output comparison:
127.0.0.1:7001> cluster nodes
f853501ec8ae1618df0e0f0e86fd7abcfca36207 127.0.0.1:7001 myself,master - 0 0 2 connected 4096-8191
5a2caa782042187277647661ffc5da739b3e0805 127.0.0.1:7005 slave f853501ec8ae1618df0e0f0e86fd7abcfca36207 0 1402622415859 6 connected
6c70b49813e2ffc9dd4b8ec1e108276566fcf59f 127.0.0.1:7007 slave 26f4729ca0a5a992822667fc16b5220b13368f32 0 1402622415357 8 connected
2bd5a0e3bb7afb2b56a2120d3fef2f2e4333de1d 127.0.0.1:7006 slave 32adf4b8474fdc938189dba00dc8ed60ce635b0f 0 1402622419373 7 connected
5a9450e8279df36ff8e6bb1c139ce4d5268d1390 127.0.0.1:7000 master - 0 1402622418872 1 connected 0-4095
32adf4b8474fdc938189dba00dc8ed60ce635b0f 127.0.0.1:7002 master - 0 1402622419874 3 connected 8192-12287
5db7d05c245267afdfe48c83e7de899348d2bdb6 127.0.0.1:7004 slave 5a9450e8279df36ff8e6bb1c139ce4d5268d1390 0 1402622417867 5 connected
26f4729ca0a5a992822667fc16b5220b13368f32 127.0.0.1:7003 master - 0 1402622420877 4 connected 12288-16383

127.0.0.1:7001> cluster slots
1) 1) (integer) 0
   2) (integer) 4095
   3) 1) "127.0.0.1"
      2) (integer) 7000
   4) 1) "127.0.0.1"
      2) (integer) 7004
2) 1) (integer) 12288
   2) (integer) 16383
   3) 1) "127.0.0.1"
      2) (integer) 7003
   4) 1) "127.0.0.1"
      2) (integer) 7007
3) 1) (integer) 4096
   2) (integer) 8191
   3) 1) "127.0.0.1"
      2) (integer) 7001
   4) 1) "127.0.0.1"
      2) (integer) 7005
4) 1) (integer) 8192
   2) (integer) 12287
   3) 1) "127.0.0.1"
      2) (integer) 7002
   4) 1) "127.0.0.1"
      2) (integer) 7006
2014-06-25 15:03:41 +02:00
antirez f29b12d0bf Cluster: myself->ip autodiscovery.
Instead of having an hardcoded IP address in the node configuration, we
autodiscover it via MEET messages for automatic update when the node is
restarted with a different IP address.

This mechanism was discussed in the context of PR #1782.
2014-06-25 11:28:57 +02:00
antirez 46319094db Old form of CLIENT KILL should still allow suicide. 2014-06-24 12:49:28 +02:00
antirez e21e0ba3dc Sentinel test: more correct sentinels config reset.
In the initialization test for each instance we used to unregister the
old master and register it again to clear the config.
However there is a race condition doing this: as soon as we unregister
and re-register "mymaster", another Sentinel can update the new
configuration with the old state because of gossip "hello" messages.

So the correct procedure is instead, unregister "mymaster" from all the
sentinel instances, and re-register it everywhere again.
2014-06-23 14:07:47 +02:00
antirez e3bae84606 Sentinel implementation of ROLE. 2014-06-23 12:07:41 +02:00
antirez 54c46bafa3 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2014-06-23 11:56:16 +02:00
antirez be8f4d49d4 Silence different signs comparison warning in sds.c. 2014-06-23 11:50:24 +02:00
antirez 5c7286efbd Use Redis updated sds.c for deps/hiredis. 2014-06-23 11:44:50 +02:00
Matt Stancliffandantirez 5cd83ef539 Sentinel: bind source address
Some deployments need traffic sent from a specific address.  This
change uses the same policy as Cluster where the first listed bindaddr
becomes the source address for outgoing Sentinel communication.

Fixes #1667
2014-06-23 11:44:35 +02:00
Matt Stancliffandantirez d830dcb12d Add REDIS_BIND_ADDR access macro
We need to access (bindaddr[0] || NULL) in a few places, so centralize
access with a nice macro.
2014-06-23 11:44:34 +02:00
antirez 1506c64c8b Dummy zmalloc.h restored into deps/hiredis. 2014-06-23 11:44:34 +02:00
Matt Stancliffandantirez 28f32c99d0 hiredis: Update to latest version
This is hiredis f225c276be7fd0646019b51023e3f41566633dfe

This update includes all changes that diverged inside of Redis since
the last update.  This version also allows optional source address
binding for connections which we need for some Sentinel deployments.
2014-06-23 11:44:34 +02:00
Matt Stancliffandantirez ef897a41e8 Cancel SHUTDOWN if initial AOF is being written
Fixes #1826 (and many other reports of the same problem)
2014-06-23 11:44:34 +02:00
Salvatore Sanfilippo af7647b20d Merge pull request #1829 from mattsta/stop-aof-dataloss
Cancel SHUTDOWN if initial AOF is being written
2014-06-23 10:07:13 +02:00
antirez 1206bdf13f Basic tests for the ROLE command. 2014-06-23 09:08:51 +02:00
Matt Stancliff 72980c7005 Cancel SHUTDOWN if initial AOF is being written
Fixes #1826 (and many other reports of the same problem)
2014-06-21 11:14:05 -04:00
antirez fb2f637c4a Allow to call ROLE in LOADING state. 2014-06-21 11:39:43 +02:00
antirez 7970d53997 ROLE command: array len fixed for slave output. 2014-06-21 11:17:18 +02:00
antirez fceef8e0dd Jemalloc updated to 3.6.0.
Not a single bug in about 3 months, and our previous version was
too old (3.2.0).
2014-06-20 14:59:20 +02:00
antirez fe596d67e3 Jemalloc update script added. 2014-06-20 14:53:57 +02:00
antirez 22d17bc14f Cluster: clear NOADDR flag when updating node address. 2014-06-20 09:32:47 +02:00
antirez f62dfa0f50 Sentinel test: tolerate larger delays in init tests. 2014-06-19 15:58:45 +02:00
antirez d06d8d6ffa Sentinel test: unit 02, avoid some time related false positives. 2014-06-19 15:56:28 +02:00
antirez 41f12ac988 Sentinel: send hello messages ASAP after config change.
Eventual configuration convergence is guaranteed by our periodic hello
messages to all the instances, however when there are important notices
to share, better make a phone call. With this commit we force an hello
message to other Sentinal and Redis instances within the next 100
milliseconds of a config update, which is practically better than
waiting a few seconds.
2014-06-19 15:17:06 +02:00
antirez f16ad11c71 Sentinel test: add manual failover test. 2014-06-19 10:33:12 +02:00
antirez 94bc467328 Sentinel: handle SRI_PROMOTED flag correctly.
Lack of check of the SRI_PROMOTED flag caused Sentienl to act with the
promoted slave turned into a master during failover like if it was a
normal instance.

Normally this problem was not apparent because during real failovers the
old master is down so the bugged code path was not entered, however with
manual failovers via the SENTINEL FAILOVER command, the problem was
easily triggered.

This commit prevents promoted slaves from getting reconfigured, moreover
we now explicitly check that during a failover the slave turning into a
master is the one we selected for promotion and not a different one.
2014-06-19 10:28:27 +02:00
antirez 1c343ec29f Test: use higher level redis.tcl proc to read replies. 2014-06-18 16:25:53 +02:00
antirez 5260ce1260 Sentienl/Redis test: enable link reconnection in instances.tcl. 2014-06-18 15:55:04 +02:00
antirez 670d164fbe Test: Tcl client initial support for automatic reconnection. 2014-06-18 15:52:14 +02:00
antirez 9162b5f8f9 Test: tcl client, unset deferred var on close. 2014-06-18 15:07:08 +02:00
Salvatore Sanfilippo 36bbfeef7b Merge pull request #1824 from mattsta/fix-test-error-signal
Add correct exit value to failed tests
2014-06-18 14:59:43 +02:00
Matt Stancliff f7d9827330 Add correct exit value to failed tests 2014-06-18 08:10:04 -04:00
Salvatore Sanfilippo 3f8865bcdc Merge pull request #1822 from vito/patch-1
add missing signal.h include
2014-06-18 09:31:02 +02:00
Alex Suraci 9f8dcfe69a add missing signal.h include 2014-06-17 21:59:12 -07:00
Salvatore Sanfilippo 3a482689c5 Merge pull request #1658 from mattsta/redis-cli-latency-clean-interrupt
Add SIGINT handler to cli --intrinsic-latency
2014-06-17 23:38:02 +02:00
Matt Stancliff 20c2a38ad0 Add SIGINT handler to cli --intrinsic-latency
If we run a long latency session and want to Ctrl-C out of it,
it's nice to still get the summary output at the end.
2014-06-17 10:12:57 -04:00
antirez 2c17591224 Sentinel: send SLAVEOF with MULTI, CLIENT KILL, CONFIG REWRITE.
This implements the new Sentinel-Client protocol for the Sentinel part:
now instances are reconfigured using a transaction that ensures that the
config is rewritten in the target instance, and that clients lose the
connection with the instance, in order to be forced to: ask Sentinel,
reconnect to the instance, and verify the instance role with the new
ROLE command.
2014-06-17 11:03:21 +02:00
antirez bb2011d992 CLIENT KILL API modified.
Added a new SKIPME option that is true by default, that prevents the
client sending the command to be killed, unless SKIPME NO is sent.
2014-06-16 14:50:15 +02:00
antirez e06b3819ea CLIENT KILL: fix closing link of the current client. 2014-06-16 14:28:23 +02:00
antirez e7affd266c New features for CLIENT KILL. 2014-06-16 14:24:28 +02:00
antirez f26f79ea37 Assign an unique non-repeating ID to each new client.
This will be used by CLIENT KILL and is also a good way to ensure a
given client is still the same across CLIENT LIST calls.

The output of CLIENT LIST was modified to include the new ID, but this
change is considered to be backward compatible as the API does not imply
you can do positional parsing, since each filed as a different name.
2014-06-16 14:22:55 +02:00
antirez 56d26c2380 Client types generalized.
Because of output buffer limits Redis internals had this idea of type of
clients: normal, pubsub, slave. It is possible to set different output
buffer limits for the three kinds of clients.

However all the macros and API were named after output buffer limit
classes, while the idea of a client type is a generic one that can be
reused.

This commit does two things:

1) Rename the API and defines with more general names.
2) Change the class of clients executing the MONITOR command from "slave"
   to "normal".

"2" is a good idea because you want to have very special settings for
slaves, that are not a good idea for MONITOR clients that are instead
normal clients even if they are conceptually slave-alike (since it is a
push protocol).

The backward-compatibility breakage resulting from "2" is considered to
be minimal to care, since MONITOR is a debugging command, and because
anyway this change is not going to break the format or the behavior, but
just when a connection is closed on big output buffer issues.
2014-06-16 10:43:05 +02:00
antirez aa19fd612b Scripting: regression test for issue #1811. 2014-06-12 16:20:30 +02:00
antirez 96e0fe6232 Fix semantics of Lua calls to SELECT.
Lua scripts are executed in the context of the currently selected
database (as selected by the caller of the script).

However Lua scripts are also free to use the SELECT command in order to
affect other DBs. When SELECT is called frm Lua, the old behavior, before
this commit, was to automatically set the Lua caller selected DB to the
last DB selected by Lua. See for example the following sequence of
commands:

    SELECT 0
    SET x 10
    EVAL "redis.call('select','1')" 0
    SET x 20

Before this commit after the execution of this sequence of commands,
we'll have x=10 in DB 0, and x=20 in DB 1.

Because of the problem above, there was a bug affecting replication of
Lua scripts, because of the actual implementation of replication. It was
possible to fix the implementation of Lua scripts in order to fix the
issue, but looking closely, the bug is the consequence of the behavior
of Lua ability to set the caller's DB.

Under the old semantics, a script selecting a different DB, has no simple
ways to restore the state and select back the previously selected DB.
Moreover the script auhtor must remember that the restore is needed,
otherwise the new commands executed by the caller, will be executed in
the context of a different DB.

So this commit fixes both the replication issue, and this hard-to-use
semantics, by removing the ability of Lua, after the script execution,
to force the caller to switch to the DB selected by the Lua script.

The new behavior of the previous sequence of commadns is to just set
X=20 in DB 0. However Lua scripts are still capable of writing / reading
from different DBs if needed.

WARNING: This is a semantical change that will break programs that are
conceived to select the client selected DB via Lua scripts.

This fixes issue #1811.
2014-06-12 16:05:52 +02:00
antirez 38a06e86cf Cluser test: write multiple keys in unit 03.
The write-failover-read test is better if we write 100 keys so that we
are sure all the nodes are affected by the write.
2014-06-11 16:25:17 +02:00
antirez 1d5b12c54a Cluster test: rewrite config when resetting nodes. 2014-06-11 16:23:13 +02:00
antirez 73fefd0bc0 Scripting: Fix for a #1118 regression simplified.
It is more straightforward to just test for a numerical type avoiding
Lua's automatic conversion. The code is technically more correct now,
however Lua should automatically convert to number only if the original
type is a string that "looks like a number", and not from other types,
so practically speaking the fix is identical AFAIK.
2014-06-11 10:10:58 +02:00
Salvatore Sanfilippo 3ac1dad77a Merge pull request #1804 from mattsta/lua-regression-1118
Scripting: Fix regression from #1118
2014-06-11 10:09:02 +02:00
Matt Stancliff 76efe1225f Scripting: Fix regression from #1118
The new check-for-number behavior of Lua arguments broke
users who use large strings of just integers.

The Lua number check would convert the string to a number, but
that breaks user data because
Lua numbers have limited precision compared to an arbitrarily
precise number wrapped in a string.

Regression fixed and new test added.

Fixes #1118 again.
2014-06-10 14:26:13 -04:00
antirez 8ef79e72ac Cluster: fix an error message when logging failover auth denied. 2014-06-10 17:39:42 +02:00
antirez 58799718be Cluster: better comment for clusterSendFailoverAuthIfNeeded() epoch test. 2014-06-10 17:20:21 +02:00
antirez 605303d9c6 Cluster test: set config epochs when creating clusters. 2014-06-10 16:56:43 +02:00
antirez 61eb0eae83 Cluster: log granted failover authorizations. 2014-06-10 16:56:08 +02:00
antirez d5d92deb6c Cluster: log configEpoch updates to myself. 2014-06-10 16:38:36 +02:00
antirez 8204ab0098 Cluster: log when a master denies a failover auth. 2014-06-10 16:07:26 +02:00
antirez 64e6d7af47 Cluster test: unit 03, check epochs correctness. 2014-06-10 15:46:36 +02:00
antirez 4fd35a28d3 Cluster test: better failover detection in unit 03. 2014-06-10 15:46:00 +02:00
antirez 92dcae8fe4 Cluster test: added show-redis-logs command in the interactive console. 2014-06-10 15:22:40 +02:00
antirez 1617c36c35 Cluster test: new unit 03, failover loop stress testing. 2014-06-10 15:05:52 +02:00
antirez 2ed450c322 Cluster test: 02 unit more reliable waiting for slave sync. 2014-06-10 15:01:52 +02:00
antirez 9865ce0e64 Cluster test: unit 02 should wait for failover. 2014-06-10 14:18:54 +02:00
antirez 6785b39eba Cluster test: check master -> slave role switch. 2014-06-10 13:54:45 +02:00
antirez 9b3bc82c1a Cluster: cluster_my_epoch added to CLUSTER INFO output. 2014-06-10 11:35:40 +02:00
Salvatore Sanfilippo 08c7363647 Merge pull request #1743 from mattsta/cygwin-compile-fix
Cygwin compile fix
2014-06-09 11:42:14 +02:00
Salvatore Sanfilippo c7f93143f6 Merge pull request #1669 from mattsta/blpop-internally-added-keys
Fix blocking operations from missing new lists
2014-06-09 11:37:28 +02:00
antirez 6a13193d8f ROLE output improved for slaves.
Info about the replication state with the master added.
2014-06-07 17:38:20 +02:00
antirez d34c2fa3bb ROLE command added.
The new ROLE command is designed in order to provide a client with
informations about the replication in a fast and easy to use way
compared to the INFO command where the same information is also
available.
2014-06-07 17:27:49 +02:00
antirez 32d0a79f78 Cluster: check that configEpoch never goes back.
Since there are ways to alter the configEpoch outside of the failover
procedure (for exampel CLUSTER SET-CONFIG-EPOCH and via the configEpoch
collision resolution algorithm), make always sure, before replacing our
configEpoch with a new one, that it is greater than the current one.
2014-06-07 14:37:09 +02:00
antirez a2c2ef7de5 Cluster: SET-CONFIG-EPOCH should update currentEpoch.
SET-CONFIG-EPOCH, used by redis-trib at cluster creation time, failed to
update the currentEpoch, making it possible after a failover for a
server to set its configEpoch to a value smaller than the current one
(since configEpochs are obtained using currentEpoch).

The bug totally break the Redis Cluster algorithms and protocols
allowing for permanent split brain conditions about the slots
configuration as shown in issue #1799.
2014-06-07 14:25:47 +02:00
Salvatore Sanfilippo a2403227c7 Merge pull request #1772 from andygrunwald/typo-avarege-average
Fixed typo in word avarege in result message of --intrinsic-latency analyzer
2014-06-06 11:19:21 +02:00
Salvatore Sanfilippo 113be48221 Merge pull request #1780 from badboy/patch-8
Small typo fixed
2014-06-06 10:45:00 +02:00
Salvatore Sanfilippo e8ebd7d0e0 Merge pull request #1789 from yoav-steinberg/fix_eval_in_tests
Fix eval usage in tests to conform with eval semantics
2014-06-06 10:37:57 +02:00
Salvatore Sanfilippo 1e221d101c Merge pull request #1788 from zionwu/unstable
fix issue 1787
2014-06-06 10:33:11 +02:00
antirez 14fb0ac649 Don't process min-slaves-to-write for slaves.
Replication is totally broken when a slave has this option, since it
stops accepting updates from masters.

This fixes issue #1434.
2014-06-05 10:48:05 +02:00
antirez 134fd9eaf4 Tests for min-slaves-* feature. 2014-06-05 10:46:12 +02:00
antirez 3758f27bc1 Fixed dbuf variable scope in luaRedisGenericCommand().
I'm not sure if while the visibility is the inner block, the fact we
point to 'dbuf' is a problem or not, probably the stack var isx
guaranteed to live until the function returns. However obvious code is
better anyway.
2014-06-04 18:57:12 +02:00
antirez 3307db49bd Regression test for issue #1118. 2014-06-04 18:51:20 +02:00
antirez 072982d83c Scripting: better Lua number -> string conversion in luaRedisGenericCommand().
The lua_to*string() family of functions use a non optimal format
specifier when converting integers to strings. This has both the problem
of the number being converted in exponential notation, which we don't
use as a Redis return value when floating point numbers are involed,
and, moreover, there is a loss of precision since the default format
specifier is not able to represent numbers that must be represented
exactly in the IEEE 754 number mantissa.

The new code handles it as a special case using a saner conversion.

This fixes issue #1118.
2014-06-04 18:33:24 +02:00
zionwu dc8584696a fix issue 1787 2014-06-01 02:23:24 +08:00
antirez 8a588ac14d More trailing spaces in sentinel.c removed. 2014-05-28 15:46:05 +02:00
Jan-Erik Rediger b187c591e3 Small typo fixed 2014-05-28 09:46:01 +02:00
antirez efbe173962 Cluster test: add tmp dir to Git repo. 2014-05-26 18:08:12 +02:00
Salvatore Sanfilippo 3e0aef1e2e Merge pull request #1775 from mattsta/fix-test-against-new-PID-format
Fix test framework to detect proper server PID
2014-05-26 17:56:58 +02:00
Matt Stancliffandantirez 7a0c5fdf12 Disable recursive watchdog signal handler
If we are in the signal handler, we don't want to handle
the signal again.  In extreme cases, this can cause a stack overflow
and segfault Redis.

Fixes #1771
2014-05-26 17:53:33 +02:00
antirez 88c2307535 Cluster: always allow ok -> fail switch in clusterUpdateState().
There is a time defined by REDIS_CLUSTER_WRITABLE_DELAY where fail -> ok
switch is not possible after startup as a master for some time, however
the contrary (ok -> fail) should always be possible.
2014-05-26 16:24:12 +02:00
antirez 3495caf4fb Cluster test: catch FLUSHALL errors on node reset.
FLUSHALL will fail on read-only slaves, but there the command is not
needed in order to reset the instance with CLUSTER RESET so errors can
be ignored.
2014-05-26 11:00:11 +02:00
antirez 69fa133ec2 Sentinel example config: explain you don't need to specify slaves. 2014-05-26 10:17:12 +02:00
Matt Stancliff 6c16ecaaaa Fix test framework to detect proper server PID
Previously the PID format was:
[PID] Timestamp

But it recently changed to:
PID:X Timestamp

The tcl testing framework was grabbing the PID from \[\d+\], but
that's not valid anymore.

Now we grab the pid from "PID: <PID>" in the part of Redis startup
output to the right of the ASCII logo.
2014-05-23 13:54:29 -04:00
antirez d0566daeaf Cluster test: basic failover unit added. 2014-05-23 11:47:47 +02:00
antirez aa5dfb3c2c Cluster test: move basic read/write test into a procedure. 2014-05-23 11:41:50 +02:00
antirez a700bc74a8 Cluster test: more reliable 01-faildet unit.
Do things in a sequence that prevents failover during failure detection.
2014-05-23 11:40:34 +02:00
Andy Grunwald 94e3bb568a Fixed typo in word avarege in result message of --intrinsic-latency analyzer 2014-05-22 20:01:12 +02:00
antirez b239a32aae redisLogFromHandler() format changed to match new logs format. 2014-05-22 19:24:35 +02:00
antirez d98fa718e0 Tag every log line with role.
Every log contains, just after the pid, a single character that provides
information about the role of an instance:

S - Slave
M - Master
C - Writing child
X - Sentinel
2014-05-22 18:48:37 +02:00
antirez 39603a7e31 Cluster: slave validity factor is now user configurable.
Check the commit changes in the example redis.conf for more information.
2014-05-22 16:57:54 +02:00
antirez ce37488919 Test: AOF test false positive when running in slow hosts.
The bug was triggered by running the test with Valgrind (which is a lot
slower and more sensible to timing issues) after the recent changes
that made Redis more promptly able to reply with the -LOADING error.
2014-05-22 16:05:03 +02:00
antirez 17b05afda9 Test: dump.tcl fixed for RESTORE new error msg. 2014-05-22 15:56:17 +02:00
antirez 762b1ae2be Fix an error in redis-trib where we always talk with same node.
While iterating the list of nodes we want to set the slot as stable in
the current node, not always in the first node of the list.
2014-05-21 18:17:02 +02:00
antirez c68c78719f redis-trib fix improved: move keys from N nodes to owner. 2014-05-21 16:40:46 +02:00
Matt Stancliff 33f943b4cd Fix blocking operations from missing new lists
Behrad Zari discovered [1] and Josiah reported [2]: if you block
and wait for a list to exist, but the list creates from
a non-push command, the blocked client never gets notified.

This commit adds notification of blocked clients into
the DB layer and away from individual commands.

Lists can be created by [LR]PUSH, SORT..STORE, RENAME, MOVE,
and RESTORE.  Previously, blocked client notifications were
only triggered by [LR]PUSH.  Your client would never get
notified if a list were created by SORT..STORE or RENAME or
a RESTORE, etc.

Blocked client notification now happens in one unified place:
  - dbAdd() triggers notification when adding a list to the DB

Two new tests are added that fail prior to this commit.

All test pass.

Fixes #1668

[1]: https://groups.google.com/forum/#!topic/redis-db/k4oWfMkN1NU
[2]: #1668
2014-05-21 09:52:52 -04:00
antirez 56161ca0a4 redis-trib fix: use MIGRATE REPLACE when fixing slots.
This fixes issue #1765.
2014-05-21 12:15:06 +02:00
antirez 7f772355f4 Regression test for issue #1764. 2014-05-20 16:20:16 +02:00
antirez ce2b2f22d9 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2014-05-20 16:15:13 +02:00
Salvatore Sanfilippo ce7c47265b Merge pull request #1764 from michael-grunder/lua_cache_segfault
Fix LUA_OBJCACHE segfault.
2014-05-20 16:14:34 +02:00
antirez 4ddc77041f Remove trailing spaces from scripting.c 2014-05-20 16:11:22 +02:00
antirez 01e3f9ba1d Remove trailing spaces from sentinel.c. 2014-05-20 14:22:42 +02:00
michael-grunder ea0e2524aa Fix LUA_OBJCACHE segfault.
When scanning the argument list inside of a redis.call() invocation
for pre-cached values, there was no check being done that the
argument we were on was in fact within the bounds of the cache size.

So if a redis.call() command was ever executed with more than 32
arguments (current cache size #define setting) redis-server could
segfault.
2014-05-19 13:18:13 -07:00
antirez a9e62ab9fa HyperLogLog regression test for issue #1762. 2014-05-19 15:44:04 +02:00
Salvatore Sanfilippo 04d901aed5 Merge pull request #1762 from trink/unstable
Correct the HyperLogLog stale cache flag to prevent unnecessary computat...
2014-05-19 15:39:30 +02:00
antirez 194b78050e Cluster test: better failure detection test and framework improvements. 2014-05-19 15:26:19 +02:00
antirez a7da78e472 Cluster test: failure detection initial tests. 2014-05-19 11:39:15 +02:00
antirez 4c04744734 Cluster test: proper initialization at unit startup. 2014-05-19 11:24:15 +02:00
Mike Trinkala ba52cd06c8 Correct the HyperLogLog stale cache flag to prevent unnecessary computations.
Set the MSB as documented.
2014-05-18 07:26:26 -07:00
antirez 67133d2f48 Cluster: use clusterSetNodeAsMaster() during slave failover.
clusterHandleSlaveFailover() was reimplementing what
clusterSetNodeAsMaster() without any good reason.
2014-05-15 17:03:28 +02:00
antirez 8c6e92c3bc Cluster: clear todo_before_sleep flags when executing actions.
Thanks to this change, when there is some code like:

    clusterDoBeforeSleep(CLUSTER_TODO_UPDATE_STATE|...);
    ... and later before returning to the event loop ...
    clusterUpdateState();

The clusterUpdateState() function will clar the flag and will not be
repeated in the clusterBeforeSleep() function. This especially important
for config save/fsync flags which are slow to execute and not a good
idea to repeat without a good reason.

This is implemented for all the CLUSTER_TODO flags.
2014-05-15 16:33:13 +02:00
antirez 7b87cda70e Fixed typo in CLUSTER RESET implementation. 2014-05-15 12:33:57 +02:00
antirez 796f4ae9f7 CLUSTER RESET implemented.
The new command is able to reset a cluster node so that it starts again
as a fresh node. By default the command performs a soft reset (the same
as calling it as CLUSTER RESET SOFT), and the following steps are
performed:

1) All slots are set as unassigned.
2) The list of known nodes is flushed.
3) Node is set as master if it is a slave.

When an hard reset is performed with CLUSTER RESET HARD the following
additional operations are performed:

4) A new Node ID is created at random.
5) Epochs are set to 0.

CLUSTER RESET is useful both when the sysadmin wants to reconfigure a
node with a different role (for example turning a slave into a master)
and for testing purposes.

It also may play a role in automatically provisioned Redis Clusters,
since it allows to reset a node back to the initial state in order to be
reconfigured.
2014-05-15 11:43:06 +02:00
antirez 8b9d5ecbd1 Remove trailing spaces from cluster.c file. 2014-05-15 10:18:36 +02:00
antirez 0b7aa26164 Cluster test: added function assert_cluster_state. 2014-05-14 15:21:57 +02:00
antirez 60e5d1724c Cluster: don't accept cluster bus connections during startup. 2014-05-14 12:05:00 +02:00
antirez 6baac558d8 Cluster: better handling of stolen slots.
The previous code handling a lost slot (by another master with an higher
configuration for the slot) was defensive, considering it an error and
putting the cluster in an odd state requiring redis-cli fix.

This was changed, because actually this only happens either in a
legitimate way, with failovers, or when the admin messed with the config
in order to reconfigure the cluster. So the new code instead will try to
make sure that the keys stored match the new slots map, by removing all
the keys in the slots we lost ownership from.

The function that deletes the keys from the lost slots is called only
if the node does not lose all its slots (resulting in a reconfiguration
as a slave of the node that got ownership). This is an optimization
since the replication code will anyway flush all the instance data in
a faster way.
2014-05-14 10:46:37 +02:00
antirez 27ca133d35 cluster.tcl: fix redis links leak in refresh_nodes_map. 2014-05-14 09:10:03 +02:00
antirez cdf2271c5b cluster.tcl: saner error handling.
Better handling of connection errors in order to update the table and
recovery, populate the startup nodes table after fetching the list of
nodes.

More work to do about it, it is still not as reliable as
redis-rb-cluster implementation which is the minimal reference
implementation for Redis Cluster clients.
2014-05-14 00:15:52 +02:00
antirez bae30479fb redis.tcl: return I/O error message when peer closes connection. 2014-05-14 00:14:35 +02:00
antirez 832a298005 Cluster: fixed data_age computation / check integer overflow. 2014-05-12 17:46:15 +02:00
Matt Stancliff 7c4decb101 Fix lack of strtold under Cygwin
Renaming strtold to strtod then casting
the result is the standard way of dealing with
no strtold in Cygwin.
2014-05-12 11:11:09 -04:00
Matt Stancliff 3e0e51dd9f Fix lack of SA_ONSTACK under Cygwin
Fixes #232
2014-05-12 11:10:24 -04:00
antirez 2692339138 Cluster: forced failover implemented.
Using CLUSTER FAILOVER FORCE it is now possible to failover a master in
a forced way, which means:

1) No check to understand if the master is up is performed.
2) No data age of the slave is checked. Evan a slave with very old data
   can manually failover a master in this way.
3) No chat with the master is attempted to reach its replication offset:
   the master can just be down.
2014-05-12 16:34:20 +02:00
antirez 005f564eb3 Cluster: bypass data_age check for manual failovers.
Automatic failovers only happen in Redis Cluster if the slave trying to
be elected was disconnected from its master for no more than 10 times
the node-timeout value. However there should be no such a check for
manual failovers, since these are initiated by the sysadmin that, in
theory, knows what she is doing when a slave is selected to be promoted.
2014-05-12 16:12:12 +02:00
Akos Vandraandantirez b252fab06c Fixed possible buffer overflow bug if RDB file is corrupted.
(Note: commit message modified by @antirez for clarity).
2014-05-12 11:48:14 +02:00
Akos Vandraandantirez 433e835d3e fixed possible buffer overflow error 2014-05-12 11:19:07 +02:00
antirez 658ad301cc redis-trib create: use CONFIG SET-CONFIG-EPOCH before joining the cluster.
This way there is no need for the conflict resolution algo to be used in
order to start with a cluster where each node has a different
configEpoch.
2014-05-12 11:06:37 +02:00
antirez 63d1f9e570 Sentinel: Add "dir /tmp" directive in example sentinel.conf. 2014-05-12 10:46:25 +02:00
antirez 715a6d3a78 redis-trib import: trap MIGRATE errors. 2014-05-12 10:36:33 +02:00
antirez 939c586ef7 redis-trib.rb: MIGRATE hardcoded timeout set to 15 sec.
Will be configurable / adaptive at some point but let's start with a
saner value compared to 1 sec which is not a good idea for big data
structures stored into a single key.
2014-05-12 10:22:24 +02:00
antirez 5c78f87666 RESTORE: reply with -BUSYKEY special error code.
The error when the target key is busy was a generic one, while it makes
sense to be able to distinguish between the target key busy error and
the others easily.
2014-05-12 10:01:59 +02:00
antirez 2a48bd4a37 Cluster: initial ability to import data from standalone instance. 2014-05-10 17:59:31 +02:00
antirez 71d0e7e0ea CLUSTER MEET: better error messages when address is invalid.
Fixes issue #1734.
2014-05-09 16:36:59 +02:00
antirez 74435aba47 redis-trib: allow support for mandatory options. 2014-05-09 16:11:11 +02:00
antirez 72ff03346f DEBUG POPULATE: call dictExpand() to avoid useless rehashing. 2014-05-09 15:02:29 +02:00
antirez 8a170c817d Cluster: bulk-accept new nodes connections.
The same change was operated for normal client connections. This is
important for Cluster as well, since when a node rejoins the cluster,
when a partition heals or after a restart, it gets flooded with new
connection attempts by all the other nodes trying to form a full
mesh again.
2014-05-09 11:52:59 +02:00
antirez 3625b52791 Cluster: clusterAcceptHandler() comments updated to match the code. 2014-05-09 11:44:46 +02:00
antirez 2102778606 Sentinel: log when a failover will be attempted again.
When a Sentinel performs a failover (successful or not), or when a
Sentinel votes for a different Sentinel trying to start a failover, it
sets a min delay before it will try to get elected for a failover.

While not strictly needed, because if multiple Sentinels will try
to failover the same master at the same time, only one configuration
will eventually win, this serialization is practically very useful.
Normal failovers are cleaner: one Sentinel starts to failover, the
others update their config when the Sentinel performing the failover
is able to get the selected slave to move from the role of slave to the
one of master.

However currently this timeout was implicit, so users could see
Sentinels not reacting, after a failed failover, for some time, without
giving any feedback in the logs to the poor sysadmin waiting for clues.

This commit makes Sentinels more verbose about the delay: when a master
is down and a failover attempt is not performed because the delay has
still not elaped, something like that will be logged:

    Next failover delay: I will not start a failover
    before Thu May  8 16:48:59 2014
2014-05-08 16:38:53 +02:00
antirez 931beae9b0 Sentinel: generate +config-update-from event when a new config is received.
This event makes clear, before the switch-master event is generated,
that a Sentinel received a configuration update from another Sentinel.
2014-05-08 15:59:34 +02:00
antirez 0b0f872f3f REDIS_ENCODING_EMBSTR_SIZE_LIMIT set to 39.
The new value is the limit for the robj + SDS header + string +
null-term to stay inside the 64 bytes Jemalloc arena in 64 bits
systems.
2014-05-07 17:05:09 +02:00
antirez 76c31d425e Scripting test: check that Lua can call commands rewirting argv.
SPOP, tested in the new test, is among the commands rewritng the
client->argv argument vector (it gets rewritten as SREM) for command
replication purposes.

Because of recent optimizations to client->argv caching in the context
of the Lua internal Redis client, it is important to test for SPOP to be
callable from Lua without bad effects to the other commands.
2014-05-07 16:12:32 +02:00
antirez 088b9eadc4 Test: handle new osx 'leaks' error.
Sometimes the process is still there but no longer in a state that can
be checked (after being killed). This used to happen after a call to
SHUTDOWN NOSAVE in the scripting unit, causing a false positive.
2014-05-07 16:12:32 +02:00
antirez 4f686555ce Scripting: objects caching for Lua c->argv creation.
Reusing small objects when possible is a major speedup under certain
conditions, since it is able to avoid the malloc/free pattern that
otherwise is performed for every argument in the client command vector.
2014-05-07 16:12:32 +02:00
antirez 1e4ba6e7e6 Scripting: Use faster API for Lua client c->argv creation.
Replace the three calls to Lua API lua_tostring, lua_lua_strlen,
and lua_isstring, with a single call to lua_tolstring.

~ 5% consistent speed gain measured.
2014-05-07 16:12:32 +02:00
antirez 76fda9f8e1 Scripting: don't call lua_gc() after Lua script run.
Calling lua_gc() after every script execution is too expensive, and
apparently does not make the execution smoother: the same peak latency
was measured before and after the commit.

This change accounts for scripts execution speedup in the order of 10%.
2014-05-07 16:12:32 +02:00
antirez 48c49c4851 Scripting: cache argv in luaRedisGenericCommand().
~ 4% consistently measured speed improvement.
2014-05-07 16:12:32 +02:00
antirez 3318b74705 Fixed missing c->bufpos reset in luaRedisGenericCommand().
Bug introduced when adding a fast path to avoid copying the reply buffer
for small replies that fit into the client static buffer.
2014-05-07 16:12:32 +02:00
antirez c49955fd77 Scripting: replace tolower() with faster code in evalGenericCommand().
The function showed up consuming a non trivial amount of time in the
profiler output. After this change benchmarking gives a 6% speed
improvement that can be consistently measured.
2014-05-07 16:12:32 +02:00
antirez 0ef4f44c5a Scripting: luaRedisGenericCommand() fast path for buffer-only replies.
When the reply is only contained in the client static output buffer, use
a fast path avoiding the dynamic allocation of an SDS string to
concatenate the client reply objects.
2014-05-07 16:12:32 +02:00
antirez 8226be61ec Define HAVE_ATOMIC for clang. 2014-05-07 16:12:32 +02:00
antirez 40abeb1f40 Scripting: simpler reply buffer creation in luaRedisGenericCommand().
It if faster to just create the string with a single sdsnewlen() call.
If c->bufpos is zero, the call will simply be like sdsemtpy().
2014-05-07 16:12:32 +02:00
antirez 1c130c6b03 Test: cluster/base, check that we can write/read from cluster. 2014-05-02 16:37:12 +02:00
antirez 3bc119c155 Cluster: Tcl cluster client: handle MOVED/ASK. 2014-05-02 15:35:08 +02:00
antirez fcd2065f8e Cluster: Tcl cluster client: slots-nodes map and close method.
Now the client is able to actually run commands in a Redis Cluster
assuming the slots->nodes map is stable.
2014-05-02 10:56:02 +02:00
antirez 5344357f80 Cluster: Tcl cluster client: build nodes representation. 2014-05-02 10:19:28 +02:00
antirez 8b7e23bdde Cluster: Tcl cluster client: get nodes description. 2014-05-02 09:55:27 +02:00
antirez bc8ea04a7d Cluster: Tcl cluster client key -> hashslot. 2014-04-30 18:55:28 +02:00
antirez e8357d0f85 Cluster test: Tcl cluster library initial skeleton. 2014-04-30 15:47:19 +02:00
antirez 1db45ba58c Cluster test: check for state=ok after slot allocation. 2014-04-30 09:29:03 +02:00
antirez 11d9ecb71d CLUSTER SET-CONFIG-EPOCH implemented.
Initially Redis Cluster accepted that after cluster creation all the
nodes were at configEpoch 0, evolving from zero as failovers happen.

However later the semantic was made more strict in order to make sure a
cluster has always all the master nodes with a different configEpoch,
which is more robust in some corner case (especially resulting from
errors by the system administrator).

To assign different configEpochs to different nodes at startup was a
task performed naturally by the config conflicts resolution algorithm
(see the Cluster specification). However this works well only for small
clusters or when there are actually just a few collisions, since it is
designed for exceptional cases.

When a large cluster is created hundred of nodes can be at epoch 0, so
the conflict resolution code is slow to provide an unique config to each
node. For this reason this new command was introduced. It can be called
only when a node is totally fresh: no other nodes known, and configEpoch
set to zero, so it is safe even against misuses.

redis-trib will use the new command in order to start the cluster
already setting an incremental unique config to every node.
2014-04-29 19:15:16 +02:00
antirez 7b5ce1ffb1 Cluster test: slots allocation. 2014-04-29 18:40:43 +02:00
antirez 4a3db25504 Cluster test: use 20 instances.
This makes tests a bit slower, but it is better to test things at a
decent scale instead of using just a few nodes, and for a few tests we
actually need so many nodes.
2014-04-29 16:20:43 +02:00
antirez e8631a6991 Cluster / Sentinel test: instances count moved to run.tcl. 2014-04-29 16:17:15 +02:00
antirez 9e422f74a6 Cluster test: config epoch conflict resolution. 2014-04-29 15:39:59 +02:00
antirez 2c55622333 Cluster test: auto-discovery to form full mesh. 2014-04-29 15:00:11 +02:00
antirez 2555b2f4bd Cluster test: check that every node has a different ID. 2014-04-29 10:42:32 +02:00
antirez e1b129811a Cluster test: basic cluster nodes info access functions. 2014-04-29 10:42:17 +02:00
antirez 0bcc7cb4bf CLIENT LIST speedup via peerid caching + smart allocation.
This commit adds peer ID caching in the client structure plus an API
change and the use of sdsMakeRoomFor() in order to improve the
reallocation pattern to generate the CLIENT LIST output.

Both the changes account for a very significant speedup.
2014-04-28 17:36:57 +02:00
antirez f9a4a80f49 Use sdscatfmt() in getClientInfoString() to make it faster. 2014-04-28 16:55:43 +02:00
antirez 2d76736a2e Added new sdscatfmt() %u and %U format specifiers.
This commit also fixes a bug in the implementation of sdscatfmt()
resulting from stale references to the SDS string header after
sdsMakeRoomFor() calls.
2014-04-28 16:38:17 +02:00
antirez 53575c4708 sdscatfmt() added to SDS library.
sdscatprintf() relies on printf() family libc functions and is sometimes
too slow in critical code paths. sdscatfmt() is an alternative which is:

1) Far less capable.
2) Format specifier uncompatible.
3) Faster.

It is suitable to be used in those speed critical code paths such as
CLIENT LIST output generation.
2014-04-28 16:23:17 +02:00
antirez 93e7a130fc Test: fixed scripting.tcl test false positive. 2014-04-24 21:44:32 +02:00
antirez e29d330724 Process events with processEventsWhileBlocked() when blocked.
When we are blocked and a few events a processed from time to time, it
is smarter to call the event handler a few times in order to handle the
accept, read, write, close cycle of a client in a single pass, otherwise
there is too much latency added for clients to receive a reply while the
server is busy in some way (for example during the DB loading).
2014-04-24 21:44:32 +02:00
antirez 3a3458ee7b Accept multiple clients per iteration.
When the listening sockets readable event is fired, we have the chance
to accept multiple clients instead of accepting a single one. This makes
Redis more responsive when there is a mass-connect event (for example
after the server startup), and in workloads where a connect-disconnect
pattern is used often, so that multiple clients are waiting to be
accepted continuously.

As a side effect, this commit makes the LOADING, BUSY, and similar
errors much faster to deliver to the client, making Redis more
responsive when there is to return errors to inform the clients that the
server is blocked in an not interruptible operation.
2014-04-24 21:44:32 +02:00
antirez cac4bae11a AE_ERR -> ANET_ERR in acceptUnixHandler().
No actual changes since the value is the same.
2014-04-24 21:43:22 +02:00
antirez c3f85c0107 Redis Cluster test framework skeleton. 2014-04-24 18:01:41 +02:00
antirez 7d9b45b4a1 While ANET_ERR is -1, check syscall retval for -1 itself. 2014-04-24 17:03:07 +02:00
antirez e3cf812c9e clusterLoadConfig() REDIS_ERR retval semantics refined.
We should return REDIS_ERR to signal we can't read the configuration
because there is no config file only after checking errno, othewise
we risk to rewrite an existing file that was not accessible for some
other reason.
2014-04-24 16:23:03 +02:00
antirez db06108bc1 Lock nodes.conf to avoid multiple processes using the same file.
This was a common source of problems among users.
The solution adopted is not bullet-proof as if the user deletes the
nodes.conf file manually, and starts a new instance with the same
nodes.conf file path, two instances will use the same file. However
following this reasoning the user may drop a nuclear bomb into the
datacenter as well.
2014-04-24 16:04:10 +02:00
antirez 897adc1c8c Sentinel test files / directories layout improved.
The test now runs in a self-contained directory.
The general abstractions to run the tests in an environment where
mutliple instances are executed at the same time was extrapolated into
instances.tcl, that will be reused to test Redis Cluster.
2014-04-24 11:08:22 +02:00
antirez e6b6813656 Test: spawn_instance now supports additional config. 2014-04-24 10:50:51 +02:00
Salvatore Sanfilippo 32c917964e Merge pull request #1677 from mattsta/expire-before-delete
Check key expiration before deleting
2014-04-23 16:13:49 +02:00
Kevin Menardandantirez 12e435d2a4 Fixed typos. 2014-04-23 15:34:51 +02:00
Glauber Costaandantirez 7dd4432798 fix null pointer access with no file pointer
I happen to be working on a system that lacks urandom. While the code does try
to handle this case and artificially create some bytes if the file pointer is
empty, it does try to close it unconditionally, leading to a segfault.
2014-04-23 12:07:25 +02:00
Salvatore Sanfilippo e0918a332d Merge pull request #1701 from kingsumos/node_description
fix cluster node description showing wrong slot allocation
2014-04-23 11:37:47 +02:00
antirez cb4e2ee9e7 Missing return REDIS_ERR added to processMultibulkBuffer().
When we set a protocol error we should return with REDIS_ERR to let the
caller know it should stop processing the client.

Bug found in a code auditing related to issue #1699.
2014-04-23 10:19:43 +02:00
kingsumos a69178fdd2 fix cluster node description showing wrong slot allocation 2014-04-22 11:44:53 -04:00
antirez 20c040d364 redis-cli help.h updated. 2014-04-22 16:14:38 +02:00
antirez 9eeb03e5f7 generate-command-help.rb updated with new hyperloglog group. 2014-04-22 16:13:58 +02:00
yoav fdaab02347 Merge remote-tracking branch 'upstream/unstable' into unstable 2014-04-22 10:01:21 +03:00
antirez 9caa1ae96a Fuzzy test for ZREMRANGEBYLEX added. 2014-04-18 13:02:16 +02:00
antirez ab3afe2f4d ZREMRANGEBYLEX memory leak removed calling zslFreeLexRange(). 2014-04-18 13:01:04 +02:00
antirez 85a2f2354e PFCOUNT multi-key test added. 2014-04-18 12:36:33 +02:00
antirez 5eb7ac0c92 Speedup hllRawSum() processing 8 bytes per iteration.
The internal HLL raw encoding used by PFCOUNT when merging multiple keys
is aligned to 8 bits (1 byte per register) so we can exploit this to
improve performances by processing multiple bytes per iteration.

In benchmarks the new code was several times faster with HLLs with many
registers set to zero, while no slowdown was observed with populated
HLLs.
2014-04-17 18:05:27 +02:00
antirez 192a213274 Speedup SUM(2^-reg[m]) in HyperLogLog computation.
When the register is set to zero, we need to add 2^-0 to E, which is 1,
but it is faster to just add 'ez' at the end, which is the number of
registers set to zero, a value we need to compute anyway.
2014-04-17 17:53:20 +02:00
antirez 0feb2aabca PFCOUNT support for multi-key union. 2014-04-17 17:32:59 +02:00
antirez fcd2155b6f HyperLogLog low level merge extracted from PFMERGE. 2014-04-17 17:08:43 +02:00
antirez 78954ca3a2 ZREMRANGEBYLEX implemented. 2014-04-17 14:49:25 +02:00
antirez 8827dc4eec Always pass sorted set range objects by reference. 2014-04-17 14:30:12 +02:00
antirez 95098b7230 ZREMRANGE* commands refactored into a single generic function. 2014-04-17 14:19:14 +02:00
antirez c0ccd4da7d Sorted set lex ranges stress tester. 2014-04-17 10:25:58 +02:00
antirez 5c48432340 Basic ZRANGEBYLEX / ZLEXCOUNT tests. 2014-04-17 00:08:11 +02:00
antirez bcab07f7fc Pass by pointer and release of lex ranges.
Given that the code was written with a 2 years pause... something
strange happened in the middle. So there was no function to free a
lex range min/max objects, and in some places the range was passed by
value.
2014-04-16 23:55:58 +02:00
antirez 8b5e0b213e ZLEXCOUNT implemented.
Like ZCOUNT for lexicographical ranges.
2014-04-16 12:17:00 +02:00
antirez cffeafe391 More HyperLogLog tests. 2014-04-16 09:17:38 +02:00
antirez 8e8f8189eb HyperLogLog invalid representation error code set to INVALIDOBJ. 2014-04-16 09:10:30 +02:00
antirez 0bbdaca6a0 PFDEBUG TODENSE added.
Converts HyperLogLogs from sparse to dense. Used for testing.
2014-04-16 09:05:42 +02:00
antirez 402110f9fd User-defined switch point between sparse-dense HLL encodings. 2014-04-15 17:46:51 +02:00
antirez d541f65d66 PFSELFTEST improved with sparse encoding checks. 2014-04-15 10:10:38 +02:00
antirez dde8dff73f PFDEBUG ENCODING added. 2014-04-14 19:35:00 +02:00
antirez 54f0156e8c Set HLL_SPARSE_MAX to 3000.
After running a few benchmarks, 3000 looks like a reasonable value to
keep HLLs with a few thousand elements small while the CPU cost is
still not huge.

This covers all the cases where the dense representation would use N
orders of magnitude more space, like in the case of many HLLs with
carinality of a few tens or hundreds.

It is not impossible that in the future this gets user configurable,
however it is easy to pick an unreasoable value just looking at savings
in the space dimension without checking what happens in the time
dimension.
2014-04-14 16:15:55 +02:00
antirez 848d0461f9 Error message for invalid HLL objects unified. 2014-04-14 16:11:54 +02:00
antirez 81ceef7d22 PFMERGE fixed to work with sparse encoding. 2014-04-14 16:09:32 +02:00
antirez 9df77fc0c4 Mark PFDEBUG as write command in the commands table.
It is safer since it is able to have side effects.
2014-04-14 15:57:50 +02:00
antirez 3bc35f9ce9 Correctly replicate PFDEBUG GETREG.
Even if it is a debugging command, make sure that when it forces a
change in encoding, the command is propagated.
2014-04-14 15:57:19 +02:00
antirez ba0afb4566 Added assertion in hllSparseAdd() when promotion to dense occurs.
If we converted to dense, a register must be updated in the dense
representation.
2014-04-14 15:55:21 +02:00
antirez e9cd51c7eb hllSparseAdd(): speed optimization.
Mostly by reordering opcodes check conditional by frequency of opcodes
in larger sparse-encoded HLLs.
2014-04-14 15:42:05 +02:00
antirez 681bf7468b Detect corrupted sparse HLLs in hllSparseSum(). 2014-04-14 15:20:26 +02:00
antirez db40da0a47 hllSparseAdd(): faster code removing conditional.
Bottleneck found profiling. Big run time improvement found when testing
after the change.
2014-04-14 12:58:46 +02:00
antirez 4e0a99ba51 Comment typo in hllSparseAdd(). first -> fits. 2014-04-14 12:12:53 +02:00
antirez 5532b5308a Merge adjacent VAL opcodes in hllSparseAdd().
As more values are added splitting ZERO or XZERO opcodes, try to merge
adjacent VAL opcodes if they have the same value.
2014-04-14 12:11:39 +02:00
antirez 837ca39081 More robust HLL_SPARSE macros protecting 'p' with parens.
Now the macros will work with arguments such as "ptr+1".
2014-04-14 11:49:53 +02:00
antirez 142d133c8a hllSparseAdd() opcode seek stop condition fixed. 2014-04-14 11:04:11 +02:00
antirez 1ee18db922 Fixed error message generation in PFDEBUG GETREG.
Bulk length for registers was emitted too early, so if there was a bug
the reply looked like a long array with just one element, blocking the
client as result.
2014-04-14 10:25:19 +02:00
antirez 82c31f750d Fixed memmove() count in hllSparseAdd(). 2014-04-14 09:40:07 +02:00
antirez 3b20003503 hllSparseAdd(): more correct dense conversion conditional.
We want to promote if the total string size exceeds the resulting size
after the upgrade.
2014-04-14 09:36:32 +02:00
antirez b7571b7453 hllSparseToDense(): sanity check added.
The function checks if all the HLL_REGISTERS were processed during the
convertion from sparse to dense encoding, returning REDIS_OK or
REDIS_ERR to signal a corruption problem.

A bug in PFDEBUG GETREG was fixed: when the object is converted to the
dense representation we need to reassign the new pointer to the header
structure pointer.
2014-04-14 09:27:01 +02:00
antirez f9dc3cb04d PFDEBUG DECODE added.
Provides a human readable description of the opcodes composing a
run-length encoded HLL (sparse encoding).
The command is only useful for debugging / development tasks.
2014-04-14 09:00:53 +02:00
antirez 261da523e8 PFDEBUG added, PFGETREG removed.
PFDEBUG will be the interface to do debugging tasks with a key
containing an HLL object.
2014-04-13 23:01:21 +02:00
antirez e8e717e145 hllSparseToDense API changed to take ref to object.
The new API takes directly the object doing everything needed to
turn it into a dense representation, including setting the new
representation as object->ptr.
2014-04-13 22:59:27 +02:00
antirez 2067644a8c hllSparseAdd() sanity check for span != 0 added. 2014-04-13 10:19:12 +02:00
antirez 80140fa006 Fix hllSparseAdd() new sequence replacement when next is NULL.
sdsIncrLen() must be called anyway even if we are replacing the last
oppcode of the sparse representation.
2014-04-12 23:55:44 +02:00
antirez 3c3c16561a Fix seqlen computation in hllSparseAdd(). 2014-04-12 23:52:36 +02:00
antirez a9e057e095 Abstract hllSparseAdd() / hllDenseAdd() via hllAdd(). 2014-04-12 23:42:56 +02:00
antirez 0b7d08efb9 hllSparseSum(): multiply 1 * runlen for zero entries. 2014-04-12 16:47:50 +02:00
antirez d9314079ca Macro HLL_SPARSE_XZERO_LEN fixed. 2014-04-12 16:46:08 +02:00
antirez f5c03044a6 Fix HLL sparse object creation #2.
Two vars initialized to wrong values in createHLLObject().
2014-04-12 16:37:50 +02:00
antirez b5659cb0a6 Increment pointer while iterating sparse HLL object. 2014-04-12 11:02:14 +02:00
antirez 1ccb661569 Fix HLL sparse object creation.
The function didn't considered the fact that each XZERO opcode is
two bytes.
2014-04-12 10:59:12 +02:00
antirez a79386b1af Create HyperLogLog objects with sparse encoding. 2014-04-12 10:56:18 +02:00
antirez 1fc04a6221 HyperLogLog sparse to dense conversion function. 2014-04-12 10:55:42 +02:00
antirez c756936b1d HyperLogLog sparse representation initial implementation.
Code never tested, but the basic layout is shaped in this commit.
Also missing:

1) Sparse -> Dense conversion function.
2) New HLL object creation using the sparse representation.
3) Implementation of PFMERGE for the sparse representation.
2014-04-11 17:34:32 +02:00
antirez 8ea5b46d30 hllCount() refactored to support multiple representations. 2014-04-11 10:25:07 +02:00
antirez 1efc1e052d hllAdd() refactored into two functions.
Also dense representation access macro renamed accordingly.
2014-04-11 09:47:52 +02:00
antirez d55474e558 HyperLogLog refactoring to support different encodings.
Metadata are now placed at the start of the representation as an header.
There is a proper structure to access the representation.
Still work to do in order to truly abstract the implementation from the
representation, commands still work assuming dense representation.
2014-04-11 09:26:45 +02:00
Matt Stancliff 88530804e8 Add test for deleting an expired key
Verify proper expire-before-delete behavior.

This test passes with the expire-before-delete commit and fails
without it.
2014-04-10 18:32:19 -04:00
Matt Stancliff 83d2830372 Check key expiration before deleting
Deleting an expired key should return 0, not success.

Fixes #1648
2014-04-10 17:08:02 -04:00
antirez 9c037ba85f HyperLogLog sparse representation slightly modified.
After running a few simulations with different alternative encodings,
it was found that the VAL opcode performs better using 5 bits for the
value and 2 bits for the run length, at least for cardinalities in the
range of interest.
2014-04-10 16:36:31 +02:00
antirez da2fbcf93d HyperLogLog sparse representation description and macros. 2014-04-09 18:56:00 +02:00
antirez 67bb2c46b2 Add casting to match printf format.
adjustOpenFilesLimit() and clusterUpdateSlotsWithConfig() that were
assuming uint64_t is the same as unsigned long long, which is true
probably for all the systems out there that we target, but still GCC
emitted a warning since technically they are two different types.
2014-04-07 08:58:06 +02:00
yoav 4930d903fc Fix eval usage in tests to conform with eval semantics 2014-04-06 17:20:01 +03:00
antirez 3a6a1e42f1 ZRANGEBYLEX and ZREVRANGEBYLEX implementation. 2014-04-05 11:41:43 +02:00
antirez d5be696db8 PFCOUNT: always unshare/decode the object.
This will be a non-op most of the times since the object will be
unshared / decoded, however it is more technically correct to start this
way since the object may be decoded even in the read-only code path.
2014-04-04 17:25:55 +02:00
antirez 1c12bcbcfb tryObjectEncoding() refactoring.
We also avoid to re-create an object that is already in EMBSTR encoding.
2014-04-04 17:25:35 +02:00
antirez 433ce7f85c Changed HyperLogLog hash seed to a non-zero value.
Using a seed of zero has the side effect of having the empty string
hashing to what is a very special case in the context of HyperLogLog: a
very long run of zeroes.

This did not influenced the correctness of the result with 16k registers
because of the harmonic mean, but still it is inconvenient that a so
obvious value maps to a so special hash.

The seed 0xadc83b19 is used instead, which is the first 64 bits of the
SHA1 of the empty string.

Reference: issue #1657.
2014-04-04 09:36:32 +02:00
antirez 352208ff68 Initial HyperLogLog tests. 2014-04-03 22:16:05 +02:00
antirez d2ca4bb62d Return "WRONGTYPE" error on PF* type mismatch. 2014-04-03 22:10:20 +02:00
antirez 349c978189 Fix PFADD infinite loop.
We need to guarantee that the last bit is 1, otherwise an element may
hash to just zeroes with probability 1/(2^64) and trigger an infinite
loop.

See issue #1657.
2014-04-03 19:31:26 +02:00
antirez b612affba3 Make hll-gnuplot-graph.rb callable from cli. 2014-04-03 16:38:11 +02:00
antirez ce637b2fef Remove HyperLogLog type checking duplicated code. 2014-04-03 13:20:34 +02:00
antirez aaaed66c56 PFGETREG added for testing purposes.
The new command allows to get a dump of the registers stored
into an HyperLogLog data structure for testing / debugging purposes.
2014-04-03 10:45:30 +02:00
antirez 9682295f68 PFCOUNT: unshare the object when cached cardinality is modified. 2014-04-03 10:37:32 +02:00
antirez be9860d0e9 PFSELFTEST improved to test the approximation error. 2014-04-03 10:18:31 +02:00
antirez cf34507b87 hll-gnuplot-graph.rb improved with new filter.
The function to generate graphs is also more flexible as now includes
step and max value. The step of the samples generation function is no
longer limited to min step of 1000.
2014-04-02 12:30:11 +02:00
antirez 096b5e921e HyperLogLog: added magic / version.
This will allow future changes like compressed representations.
Currently the magic is not checked for performance reasons but this may
change in the future, for example if we add new types encoded in strings
that may have the same size of HyperLogLogs.
2014-04-02 09:58:47 +02:00
Salvatore Sanfilippo cef505d739 Merge pull request #1646 from raydog/unstable
Change HLL* to PF* in a few spots
2014-04-02 08:12:27 +02:00
Raymond Myers bf066c875f Fixed pfadd/pfcount commands emitting hll* events instead of pf* events 2014-04-01 14:59:13 -07:00
Raymond Myers f0868e080d Change HLL* to PF* in error messages 2014-04-01 14:54:31 -07:00
antirez 4ab162a559 Include redis.h before other stuff in hyperloglog.c.
Otherwise fmacros.h is included later and this may break compilation on
different systems.
2014-04-01 15:52:15 +02:00
antirez 5afcca34ce HyperLogLog API prefix modified from "P" to "PF".
Using both the initials of Philippe Flajolet instead of just "P".
2014-03-31 22:48:01 +02:00
antirez ba4e20835a Makefile.dep updated with hyperloglog.o deps. 2014-03-31 19:51:34 +02:00
antirez e887c62e45 HyperLogLog: make API use the P prefix in honor of Philippe Flajolet. 2014-03-31 19:29:40 +02:00
antirez f1b7608128 HLLMERGE fixed by adding a... missing loop! 2014-03-31 16:03:05 +02:00
antirez 0c9f06a237 hll-gnuplot-graph.rb: added new filter "all". 2014-03-31 15:45:06 +02:00
antirez ec1ee66256 HyperLogLog apply bias correction using a polynomial.
Better results can be achieved by compensating for the bias of the raw
approximation just after 2.5m (when LINEARCOUNTING is no longer used) by
using a polynomial that approximates the bias at a given cardinality.

The curve used was found using this web page:

    http://www.xuru.org/rt/PR.asp

That performs polynomial regression given a set of values.
2014-03-31 15:41:38 +02:00
antirez f2277475b2 HLLMERGE implemented.
Merge N HLL data structures by selecting the max value for every
M[i] register among the set of HLLs.
2014-03-31 14:39:44 +02:00
antirez 4ab45183fc HLLCOUNT is technically a write command
When we update the cached value, we need to propagate the command and
signal the key as modified for WATCH.
2014-03-31 12:29:24 +02:00
antirez 8aeb0c196a HLLADD: propagate write when only variable name is given.
The following form is given:

    HLLADD myhll

No element is provided in the above case so if 'myhll' var does not
exist the result is to just create an empty HLL structure, and no update
will be performed on the registers.

In this case, the DB should still be set dirty and the command
propagated.
2014-03-31 12:21:08 +02:00
antirez 69a93194fd hll-gnuplot-graph.rb: Use |error| when filter is :max 2014-03-31 11:58:13 +02:00
antirez a8fb1a328d Ignore txt files inside utils/hyperloglog.
Those are generated to trace graphs using gnuplot.
2014-03-31 10:21:14 +02:00
antirez 60e60f4ee0 HyperLogLog: use LINEARCOUNTING up to 3m.
The HyperLogLog original paper suggests using LINEARCOUNTING for
cardinalities < 2.5m, however for P=14 the median / max error
curves show that a value of '3' is the best pick for m = 16384.
2014-03-31 10:09:55 +02:00
antirez 7f9d289e10 hll-gnuplot-graph.rb added to plot HyperLogLog error graphs. 2014-03-31 10:09:43 +02:00
antirez 307a189900 HyperLogLog approximated cardinality caching.
The more we add elements to an HyperLogLog counter, the smaller is
the probability that we actually update some register.

From this observation it is easy to see how it is possible to use
caching of a previously computed cardinality and reuse it to serve
HLLCOUNT queries as long as no register was updated in the data
structure.

This commit does exactly this by using just additional 8 bytes for the
data structure to store a 64 bit unsigned integer value cached
cardinality. When the most significant bit of the 64 bit integer is set,
it means that the value computed is no longer usable since at least a
single register was modified and we need to recompute it at the next
call of HLLCOUNT.

The value is always stored in little endian format regardless of the
actual CPU endianess.
2014-03-30 19:26:16 +02:00
antirez 543ede03f2 String value unsharing refactored into proper function.
All the Redis functions that need to modify the string value of a key in
a destructive way (APPEND, SETBIT, SETRANGE, ...) require to make the
object unshared (if refcount > 1) and encoded in raw format (if encoding
is not already REDIS_ENCODING_RAW).

This was cut & pasted many times in multiple places of the code. This
commit puts the small logic needed into a function called
dbUnshareStringValue().
2014-03-30 18:32:17 +02:00
antirez aaf6db459b Use endian neutral hash function for HyperLogLog.
We need to be sure that you can save a dataset in a Redis instance,
reload it in a different architecture, and continue to count in the same
HyperLogLog structure.

So 32 and 64 bit, little or bit endian, must all guarantee to output the
same hash for the same element.
2014-03-30 00:55:49 +01:00
antirez 4628ac0065 HyperLogLog internal representation modified.
The new representation is more obvious, starting from the LSB of the
first byte and using bits going to MSB, and passing to next byte as
needed.

There was also a subtle error: first two bits were unused, everything
was carried over on the right of two bits, even if it worked because of
the code requirement of always having a byte more at the end.

During the rewrite the code was made safer trying to avoid undefined
behavior due to shifting an uint8_t for more than 8 bits.
2014-03-29 16:04:27 +01:00
antirez 5317a582cf Remove a few useless operations from hllCount() fast path. 2014-03-29 12:17:56 +01:00
antirez 3ed947fb30 HLLCOUNT 3x faster taking fast path for default params. 2014-03-29 12:12:44 +01:00
antirez 28dce36f76 Use processor base types in HLL_(GET|SET)_REGISTER.
This speedups the macros by a noticeable factor.
2014-03-29 08:37:01 +01:00
antirez ac8fbe8829 HyperLogLog: use precomputed table for 2^(-M[i]). 2014-03-28 22:49:24 +01:00
antirez fdf81b2d35 hll-err.rb: speedup using pipelining. 2014-03-28 22:25:30 +01:00
antirez f29123364b hll-err.rb added to test error rate of Redis HyperLogLog. 2014-03-28 18:35:21 +01:00
antirez f90a4af3d7 HyperLogLog algorithm fixed in two ways.
There was an error in the computation of 2^register, and the sequence of
zeroes computed after the hashing did not included the "1".
2014-03-28 18:24:05 +01:00
antirez ded86076b3 HLLCOUNT implemented. 2014-03-28 17:37:18 +01:00
antirez 156929ee97 HLLADD implemented. 2014-03-28 16:24:35 +01:00
antirez 5660ff1cc1 hllAdd() low level HyperLogLog "add" implemented. 2014-03-28 14:42:30 +01:00
antirez e3234116ad HyperLogLog: redefine constants using "P". 2014-03-28 14:09:28 +01:00
antirez e73839e7d5 HLL_SET_REGISTER fixed.
There was an error in the first version of the macro.
Now the HLLSELFTEST test reports success.
2014-03-28 13:56:07 +01:00
antirez f22397dd7f Use REDIS_HLL_REGISTER_MAX when possible. 2014-03-28 12:16:39 +01:00
antirez 1c88c5941b HLL_(SET|GET)_REGISTER types fixed. 2014-03-28 12:15:46 +01:00
antirez 552eb5407a HLLSELFTEST command implemented.
To test the bitfield array of counters set/get macros from the Redis Tcl
suite is hard, so a specialized command that is able to test the
internals was developed.
2014-03-28 12:11:55 +01:00
antirez 0609380603 HyperLogLog: initial sketch of registers access. 2014-03-28 11:18:48 +01:00
antirez 8f52173b2c Cluster: last_vote_epoch -> lastVoteEpoch.
Use cammel case for epochs that are persisted on disk.
2014-03-27 15:01:24 +01:00
antirez 7fb14b73ba Cluster: save/restore vars that must persist after recovery.
This fixes issue #1479.
2014-03-27 14:56:29 +01:00
antirez 6dd2dbbd36 Cluster: handshake "already known" error logged to VERBOSE.
This is not really an error but something that always happens for
example when creating a new cluster, or if the sysadmin rejoins manually
a node that is already known.

Since useless logs don't help, moved to VERBOSE level.
2014-03-26 16:35:38 +01:00
antirez 3cf6f1f54f Cluster: clusterHandleConfigEpochCollision() fixed.
New config epochs must always be obtained incrementing the currentEpoch,
that is itself guaranteed to be >= the max configEpoch currently known
to the node.
2014-03-26 12:31:28 +01:00
antirez 80d4c52cdf Cluster: better logging for clusterUpdateSlotsConfigWith(). 2014-03-26 12:09:38 +01:00
antirez eb746ec408 Cluster: CLUSTER SETSLOT implementation comment updated.
Update the comment since the implementation details changed.
2014-03-25 17:50:46 +01:00
antirez 0064b1a583 Cluster: redis-trib cluster allocation more even across nodes.
redis-trib used to allocate slots not considering fractions of nodes
when computing the slots_per_node amount. So the fractional part was
carried over till the end of the allocation, where the last node
received a few more slots than any other (or a lot more if the cluster
was composed of many nodes).

The computation was changed to allocate slots more evenly when they are
not exactly divisible for the number of masters we have.
2014-03-25 17:44:39 +01:00
antirez 6c527a89a0 Cluster: configEpoch collisions resolution.
The slave election in Redis Cluster guarantees that slaves promoted to
masters always end with unique config epochs, however failures during
manual reshardings, software bugs and operational errors may in theory
cause two nodes to have the same configEpoch.

This commit introduces a mechanism to eventually always end with different
configEpochs if a collision ever happens.

As a (wanted) side effect, this also ensures that after a new cluster
is created, all nodes will end with a different configEpoch automatically.
2014-03-25 17:19:58 +01:00
antirez c1041c570f Cluster: stay within 80 cols. 2014-03-25 16:07:14 +01:00
antirez 6540e9eeaa Fix off by one bug in freeMemoryIfNeeded() eviction pool.
Bug found by the continuous integration test running the Redis
with valgrind:

==6245== Invalid read of size 8
==6245==    at 0x4C2DEEF: memcpy@GLIBC_2.2.5 (mc_replace_strmem.c:876)
==6245==    by 0x41F9E6: freeMemoryIfNeeded (redis.c:3010)
==6245==    by 0x41D2CC: processCommand (redis.c:2069)

memmove() size argument was accounting for an extra element, going
outside the bounds of the array.
2014-03-25 10:32:15 +01:00
antirez 9e0b9f12b2 Test: do not complain when "leaks" can't run because process died. 2014-03-25 09:33:37 +01:00
antirez 6e33c908dd adjustOpenFilesLimit() refactoring.
In this commit:
* Decrement steps are semantically differentiated from the reserved FDs.
  Previously both values were 32 but the meaning was different.
* Make it clear that we save setrlimit errno.
* Don't explicitly handle wrapping of 'f', but prevent it from
  happening.
* Add comments to make the function flow more readable.

This integrates PR #1630
2014-03-25 09:05:28 +01:00
Salvatore Sanfilippo 72c5ebcba4 Merge pull request #1630 from mattsta/fix-infinite-loop-ulimit
Fix infinite loop ulimit
2014-03-25 08:42:39 +01:00
antirez 35667d75c3 Fixed undefined variable value with certain code paths.
In sentinelFlushConfig() fd could be undefined when the following if
statement was true:

        if (rewrite_status == -1) goto werr;

This could cause random file descriptors to get closed.
2014-03-24 21:07:44 +01:00
Salvatore Sanfilippo fc0fb0bad3 Merge pull request #1635 from mattsta/lruclock-instead-of-getlruclock
Use LRU_CLOCK() instead of function getLRUClock()
2014-03-24 21:04:03 +01:00
Matt Stancliff 78782ed59f Use LRU_CLOCK() instead of function getLRUClock()
lookupKey() uses LRU_CLOCK(), so it seems object
creation should use LRU_CLOCK() too.
2014-03-24 14:39:26 -04:00
Salvatore Sanfilippo 7f1c3607ec Merge pull request #1606 from mattsta/fix-disk-full-dataloss
Fix data loss when save AOF/RDB with no free space
2014-03-24 19:26:25 +01:00
Matt Stancliff 4290455145 Sentinel: Notify user when config can't be saved 2014-03-24 13:54:14 -04:00
Matt Stancliff b47b343fab Fix data loss when save AOF/RDB with no free space
Previously, the (!fp) would only catch lack of free space
under OS X.  Linux waits to discover it can't write until
it actually writes contents to disk.

(fwrite() returns success even if the underlying file
has no free space to write into.  All the errors
only show up at flush/sync/close time.)

Fixes antirez/redis#1604
2014-03-24 13:54:14 -04:00
Salvatore Sanfilippo 906c4d77c0 Merge pull request #1617 from mattsta/remove-unused-warning
Cluster: remove variable causing warning
2014-03-24 18:33:22 +01:00
Salvatore Sanfilippo 8e6625e6ae Merge pull request #1629 from mattsta/fix-trib-master-assignment
Cluster: Restore proper trib master iteration
2014-03-24 18:31:55 +01:00
Salvatore Sanfilippo a006fcb8a7 Merge pull request #1628 from mattsta/fix-trib-create
Cluster: Fix trib create when masters==replicas
2014-03-24 18:26:17 +01:00
Matt Stancliff 386a46946b Fix potentially incorrect errno usage
errno may be reset by the previous call to redisLog, so capture
the original value for proper error reporting.
2014-03-24 13:21:15 -04:00
Matt Stancliff 3b54ee6ea4 Add REDIS_MIN_RESERVED_FDS define for open fds
Also update the original REDIS_EVENTLOOP_FDSET_INCR to
include REDIS_MIN_RESERVED_FDS. REDIS_EVENTLOOP_FDSET_INCR
exists to make sure more than (maxclients+RESERVED) entries
are allocated, but we can only guarantee that if we include
the current value of REDIS_MIN_RESERVED_FDS as a minimum
for the INCR size.
2014-03-24 13:15:35 -04:00
Salvatore Sanfilippo 896e15f3e3 Merge pull request #1627 from badboy/lru-fix
Fixed a few typos.
2014-03-24 18:13:39 +01:00
Salvatore Sanfilippo 68a3af5f9c Merge pull request #1609 from badboy/install_server-fix
Finally fix the `install_server.sh` script.
2014-03-24 18:10:50 +01:00
Matt Stancliff e942f3ce0f Cluster: Restore proper trib master iteration
This got removed in 2e5c394 during a new feature addition.

The prior commit had "break if masters.length == masters_count"
but we are guaranteed to aready have that condition met since
otherwise we would haven't gotten this far.

Without this break statement, it's possible some masters may
be forgotten and have zero replicas while other masters have
more than their requested number of replicas.

Thanks to carlos for pointing out this regression at:
https://groups.google.com/forum/#!topic/redis-db/_WVVqDw5B7c
2014-03-24 10:17:44 -04:00
Matt Stancliff df4bdbf688 Cluster: Fix trib create when masters==replicas
This bug was introduced in 2e5c394f during a refactor.

It took me a while to understand what was going on with
the code, so I've refactored it further by:
  - Replacing boolean values with meaningful symbols
  - Replacing 'i' with a meaningful variable name
  - Adding the proper abort check
  - Factoring out now duplicated conditionals
  - Adding optional verbose logging (we're inside *four*
    different looping constructs, so it takes a while to
    figure out where all the moving parts are)
  - Updating comment for the section

This fixes a problem when the number of master instances
equaled the number of replica instances.  Before, when
there were equal numbers of both, nodes_count would go to
zero, but the while loop would spin in i < @replicas because
i would never be updated (because the nodes_list of each ip
was length == 0, which triggered an endless loop of
next -> i = 0 -> 0 < 1? -> true -> next -> i = 0 ...)

Thanks to carlo who found this problem at:
https://groups.google.com/forum/#!topic/redis-db/_WVVqDw5B7c
2014-03-24 10:17:38 -04:00
Matt Stancliff 90b844212d Fix infinite loop on startup if ulimit too low
Fun fact: rlim_t is an unsigned long long on all platforms.

Continually subtracting from a rlim_t makes it get smaller
and smaller until it wraps, then you're up to 2^64-1.

This was causing an infinite loop on Redis startup if
your ulimit was extremely (almost comically) low.

The case of (f > oldlimit) would never be met in a case like:

    f = 150
    while (f > 20) f -= 128

Since f is unsigned, it can't go negative and would
take on values of:

    Iteration 1: 150 - 128 => 22
    Iteration 2:  22 - 128 => 18446744073709551510
    Iterations 3-∞: ...

To catch the wraparound, we use the previous value of f
stored in limit.rlimit_cur.  If we subtract from f and
get a larger number than the value it had previously,
we print an error and exit since we don't have enough
file descriptors to help the user at this point.

Thanks to @bs3g for the inspiration to fix this problem.
Patches existed from @bs3g at antirez#1227, but I needed to repair a few other
parts of Redis simultaneously, so I didn't get a chance to use them.
2014-03-24 10:17:33 -04:00
Matt Stancliff 4a25983f8f Improve error handling around setting ulimits
The log messages about open file limits have always
been slightly opaque and confusing.  Here's an attempt to
fix their wording, detail, and meaning.  Users will have a
better understanding of how to fix very common problems
with these reworded messages.

Also, we handle a new error case when maxclients becomes less
than one, essentially rendering the server unusable.  We
now exit on startup instead of leaving the user with a server
unable to handle any connections.

This fixes antirez#356 as well.
2014-03-24 10:17:33 -04:00
Matt Stancliff 491532a713 Replace magic 32 with REDIS_EVENTLOOP_FDSET_INCR
32 was the additional number of file descriptors Redis
would reserve when managing a too-low ulimit.  The
number 32 was in too many places statically, so now
we use a macro instead that looks more appropriate.

When Redis sets up the server event loop, it uses:
    server.maxclients+REDIS_EVENTLOOP_FDSET_INCR

So, when reserving file descriptors, it makes sense to
reserve at least REDIS_EVENTLOOP_FDSET_INCR FDs instead
of only 32.  Currently, REDIS_EVENTLOOP_FDSET_INCR is
set to 128 in redis.h.

Also, I replaced the static 128 in the while f < old loop
with REDIS_EVENTLOOP_FDSET_INCR as well, which results
in no change since it was already 128.

Impact: Users now need at least maxclients+128 as
their open file limit instead of maxclients+32 to obtain
actual "maxclients" number of clients.  Redis will carve
the extra REDIS_EVENTLOOP_FDSET_INCR file descriptors it
needs out of the "maxclients" range instead of failing
to start (unless the local ulimit -n is too low to accomidate
the request).
2014-03-24 10:17:33 -04:00
Matt Stancliff c138631cd1 Fix maxclients error handling
Everywhere in the Redis code base, maxclients is treated
as an int with (int)maxclients or `maxclients = atoi(source)`,
so let's make maxclients an int.

This fixes a bug where someone could specify a negative maxclients
on startup and it would work (as well as set maxclients very high)
because:

    unsigned int maxclients;
    char *update = "-300";
    maxclients = atoi(update);
    if (maxclients < 1) goto fail;

But, (maxclients < 1) can only catch the case when maxclients
is exactly 0.  maxclients happily sets itself to -300, which isn't
-300, but rather 4294966996, which isn't < 1, so... everything
"worked."

maxclients config parsing checks for the case of < 1, but maxclients
CONFIG SET parsing was checking for case of < 0 (allowing
maxclients to be set to 0).  CONFIG SET parsing is now updated to
match config parsing of < 1.

It's tempting to add a MINIMUM_CLIENTS define, but... I didn't.

These changes were inspired by antirez#356, but this doesn't
fix that issue.
2014-03-24 10:17:33 -04:00
antirez 93253c2762 Sample and cache RSS in serverCron().
Obtaining the RSS (Resident Set Size) info is slow in Linux and OSX.
This slowed down the generation of the INFO 'memory' section.

Since the RSS does not require to be a real-time measurement, we
now sample it with server.hz frequency (10 times per second by default)
and use this value both to show the INFO rss field and to compute the
fragmentation ratio.

Practically this does not make any difference for memory profiling of
Redis but speeds up the INFO call significantly.
2014-03-24 12:00:20 +01:00
antirez 30639c8ca9 sdscatvprintf(): Try to use a static buffer.
For small content the function now tries to use a static buffer to avoid
a malloc/free cycle that is too costly when the function is used in the
context of performance critical code path such as INFO output generation.

This change was verified to have positive effects in the execution speed
of the INFO command.
2014-03-24 10:20:33 +01:00
antirez d3efe04c47 Cache uname() output across INFO calls.
Uname was profiled to be a slow syscall. It produces always the same
output in the context of a single execution of Redis, so calling it at
every INFO output generation does not make too much sense.

The uname utsname structure was modified as a static variable. At the
same time a static integer was added to check if we need to call uname
the first time.
2014-03-24 10:00:08 +01:00
antirez a9caca0424 sdscatvprintf(): guess buflen using format length.
sdscatvprintf() uses a loop where it tries to output the formatted
string in a buffer of the initial length, if there was not enough room,
a buffer of doubled size is tried and so forth.

The initial guess for the buffer length was very poor, an hardcoded
"16". This caused the printf to be processed multiple times without a
good reason. Given that printf functions are already not fast, the
overhead was significant.

The new heuristic is to use a buffer 4 times the length of the format
buffer, and 32 as minimal size. This appears to be a good balance for
typical uses of the function inside the Redis code base.

This change improved INFO command performances 3 times.
2014-03-24 09:44:11 +01:00
antirez 6972f18cbd Add test-lru.rb to utils.
This is a program useful to evaluate the Redis LRU algorithm behavior.
2014-03-21 09:52:05 +01:00
antirez 4d2e8fa189 Use getLRUClock() instead of server.lruclock to create objects.
Thanks to Matt Stancliff for noticing this error. It was in the original
code but somehow I managed to remove the change from the commit...
2014-03-21 09:08:20 +01:00
antirez 5fa3248bad The default maxmemory policy is now noeviction.
This is safer as by default maxmemory should just set a memory limit
without any key to be deleted, unless the policy is set to something
more relaxed.
2014-03-21 08:03:34 +01:00
Jan-Erik Rediger 4fdd7a0546 Fixed a few typos. 2014-03-20 23:16:38 +01:00
antirez a98369929e Use 24 bits for the lru object field and improve resolution.
There were 2 spare bits inside the Redis object structure that are now
used in order to enlarge 4x the range of the LRU field.

At the same time the resolution was improved from 10 to 1 second: this
still provides 194 days before the LRU counter overflows (restarting from
zero).

This is not a problem since it only causes lack of eviction precision for
objects not touched for a very long time, and the lack of precision is
only temporary.
2014-03-20 17:56:27 +01:00
antirez f4da796c53 Default LRU samples is now 5. 2014-03-20 17:05:42 +01:00
antirez c641b670c3 Use new dictGetRandomKeys() API to get samples for eviction.
The eviction quality degradates a bit in my tests, but since the API is
faster, it allows to raise the number of samples, and overall is a win.
2014-03-20 16:52:12 +01:00
antirez 82b53c650c struct dictEntry -> dictEntry. 2014-03-20 16:20:37 +01:00
antirez 5317f5e99a Added dictGetRandomKeys() to dict.c: mass get random entries.
This new function is useful to get a number of random entries from an
hash table when we just need to do some sampling without particularly
good distribution.

It just jumps at a random place of the hash table and returns the first
N items encountered by scanning linearly.

The main usefulness of this function is to speedup Redis internal
sampling of the key space, for example for key eviction or expiry.
2014-03-20 15:50:46 +01:00
antirez 22c9cfaf57 LRU eviction pool implementation.
This is an improvement over the previous eviction algorithm where we use
an eviction pool that is persistent across evictions of keys, and gets
populated with the best candidates for evictions found so far.

It allows to approximate LRU eviction at a given number of samples
better than the previous algorithm used.
2014-03-20 11:57:29 +01:00
antirez 6d5790d682 Fix OBJECT IDLETIME return value converting to seconds.
estimateObjectIdleTime() returns a value in milliseconds now, so we need
to scale the output of OBJECT IDLETIME to seconds.
2014-03-20 11:55:18 +01:00
antirez ad6b0f70b2 Obtain LRU clock in a resolution dependent way.
For testing purposes it is handy to have a very high resolution of the
LRU clock, so that it is possible to experiment with scripts running in
just a few seconds how the eviction algorithms works.

This commit allows Redis to use the cached LRU clock, or a value
computed on demand, depending on the resolution. So normally we have the
good performance of a precomputed value, and a clock that wraps in many
days using the normal resolution, but if needed, changing a define will
switch behavior to an high resolution LRU clock.
2014-03-20 11:47:12 +01:00
antirez 1faf82663f Specify lruclock in redisServer structure via REDIS_LRU_BITS.
The padding field was totally useless: removed.
2014-03-20 11:37:27 +01:00
antirez d77e231682 Specify LRU resolution in milliseconds. 2014-03-20 11:33:25 +01:00
antirez fe30847016 Set LRU parameters via REDIS_LRU_BITS define. 2014-03-20 11:22:47 +01:00
antirez e150ec7d0c Unify stats reset for CONFIG RESETSTAT / initServer().
Now CONFIG RESETSTAT makes sure to reset all the fields, and in the
future it will be simpler to avoid missing new fields.
2014-03-19 12:55:49 +01:00
Matt Stancliff 67ed5f00aa Cluster: remove variable causing warning
GCC-4.9 warned about this, but clang didn't.

This commit fixes warning:
sentinel.c: In function 'sentinelReceiveHelloMessages':
sentinel.c:2156:43: warning: variable 'master' set but not used [-Wunused-but-set-variable]
     sentinelRedisInstance *ri = c->data, *master;
2014-03-18 15:35:09 -04:00
antirez b9e90a70fa Sentinel: sentinelRefreshInstanceInfo() minor refactoring.
Test sentinel.tilt condition on top and return if it is true.
This allows to remove the check for the tilt condition in the remaining
code paths of the function.
2014-03-18 15:35:47 +01:00
antirez 258d377d8e Sentinel test: 02 unit better coverage + refactoring. 2014-03-18 15:18:51 +01:00
antirez 58f104e2ee Sentinel test: foreach_instance_id implements 'break'. 2014-03-18 15:06:52 +01:00
antirez 2586ea7637 Sentinel: instance_is_killed proc added to sentinel.tcl. 2014-03-18 14:58:27 +01:00
antirez 218cc5fc39 Sentinel: propagate down-after-ms changes to slaves and sentinels. 2014-03-18 14:37:44 +01:00
antirez bb6d850160 Sentinel: down-after-milliseconds is not master-specific.
addReplySentinelRedisInstance() modified so that this field is displayed
for all the kind of instances: Sentinels, Masters, Slaves.
2014-03-18 11:21:17 +01:00
antirez ae0b7680b3 Sentinel failure detection implementation improved.
Failure detection in Sentinel is ping-pong based. It used to work by
remembering the last time a valid PONG reply was received, and checking
if the reception time was too old compared to the current current time.

PINGs were sent at a fixed interval of 1 second.

This works in a decent way, but does not scale well when we want to set
very small values of "down-after-milliseconds" (this is the node
timeout basically).

This commit reiplements the failure detection making a number of
changes. Some changes are inspired to Redis Cluster failure detection
code:

* A new last_ping_time field is added in representation of instances.
  If non zero, we have an active ping that was sent at the specified
  time. When a valid reply to ping is received, the field is zeroed
  again.
* last_ping_time is not reset when we reconnect the link or send a new
  ping, so from our point of view it represents the time we started
  waiting for the instance to reply to our pings without receiving a
  reply.
* last_ping_time is now used in order to check if the instance is
  timed out. This means that we can have a node timeout of 100
  milliseconds and yet the system will work well since the new check is
  not bound to the period used to send pings.
* Pings are now sent every second, or often if the value of
  down-after-milliseconds is less than one second. With a lower limit of
  10 HZ ping frequency.
* Link reconnection code was improved. This is used in order to try to
  reconnect the link when we are at 50% of the node timeout without a
  valid reply received yet. However the old code triggered unnecessary
  reconnections when the node timeout was very small. Now that should be
  ok.

The new code passes the tests but more testing is needed and more unit
tests stressing the failure detector, so currently this is merged only
in the unstable branch.
2014-03-17 18:33:45 +01:00
antirez 3a2ff55617 Sentinel: use CLIENT SETNAME when connecting to Redis.
This makes debugging / monitoring of Sentinels simpler since you can
identify sentinels in CLIENT LIST output of Redis instances.
2014-03-15 14:59:23 +01:00
Jan-Erik Rediger 2209d077d3 Finally fix the install_server.sh script.
Includes changes from a dozen bug reports and pull requests.
Was tested on Ubuntu, Debian and CentOS.
2014-03-15 14:43:50 +01:00
Salvatore Sanfilippo c65b75e740 Merge pull request #1608 from mattsta/fix-sentinel-current-epoch-segfault
Fix segfault from accessing array out of bounds
2014-03-14 22:56:24 +01:00
Matt Stancliff 584052ee6b Fix segfault from accessing array out of bounds
argc == 2; argv[2] == crash
2014-03-14 17:38:05 -04:00
antirez ed813863f0 Sentinel: be safe under crash-recovery assumptions.
Sentinel's main safety argument is that there are no two configurations
for the same master with the same version (configuration epoch).

For this to be true Sentinels require to be authorized by a majority.
Additionally Sentinels require to do two important things:

* Never vote again for the same epoch.
* Never exchange an old vote for a fresh one.

The first prerequisite, in a crash-recovery system model, requires to
persist the master->leader_epoch on durable storage before to reply to
messages. This was not the case.

We also make sure to persist the current epoch in order to never reply
to stale votes requests from other Sentinels, after a recovery.

The configuration is persisted by making use of fsync(), this is
considered in the context of this code a good enough guarantee that
after a restart our durable state is restored, however this may not
always be the case depending on the kind of hardware and operating
system used.
2014-03-14 14:58:44 +01:00
antirez 365094028b Sentinel: fake PUBLISH command to receive HELLO messages.
Now the way HELLO messages are received is unified.
Now it is no longer needed for Sentinels to converge to the higher
configuration for a master to be able to chat via some Redis instance,
the are able to directly exchanges configurations.

Note that this commit does not include the (trivial) change needed to
send HELLO messages to Sentinel instances as well, since for an error I
committed the change in the previous commit that refactored hello
messages processing into a separated function.
2014-03-14 11:07:42 +01:00
antirez 9dfe426fc8 Sentinel: HELLO processing refactored into sentinelProcessHelloMessage(). 2014-03-14 11:07:42 +01:00
antirez 133fccb03f Cluster: flag the transaction as dirty for the new redirections. 2014-03-13 15:11:53 +01:00
antirez 429aff4ef4 Linenoise updated, multiline mode enabled in redis-cli. 2014-03-13 15:11:08 +01:00
antirez cc11d103c0 redis-trib: call MIGRATE via r.client.call as fix for redis-rb API changes.
See issue #1593.

Thanks to @badboy for suggesting the direct client.call fix.
2014-03-11 16:10:13 +01:00
antirez df32eb6827 redis-trib: new subcommand 'call'. Exec command in all nodes.
Example:

./redis-trib.rb call 192.168.1.11:7000 config get cluster-node-timeout
2014-03-11 14:58:55 +01:00
antirez 2e5c394fa8 redis-trib: create subcommand is now able to assign spare slaves.
Example: if the user will try to configure a cluster with 9 nodes,
asking for 1 slave for master, redis-trib will configure a 4 masters
cluster with 1 slave each as usually, but this time will assign the
spare node as a slave of one of the masters.
2014-03-11 14:17:28 +01:00
antirez e26f4486b0 Cluster: update node configEpoch on UPDATE messages.
The UPDATE message contains the configEpoch of the node configuration
advertised in the packet. Update it if needed.
2014-03-11 11:53:09 +01:00
antirez a2ff90919f Cluster: set slot error if we receive an update for a busy slot.
By manually modifying nodes configurations in random ways, it is possible
to create the following scenario:

A is serving keys for slot 10
B is manually configured to serve keys for slot 10

A receives an update from B (or another node) where it is informed that
the slot 10 is now claimed by B with a greater configuration epoch,
however A still has keys from slot 10.

With this commit A will put the slot in error setting it in IMPORTING
state, so that redis-trib can detect the issue.
2014-03-11 11:49:47 +01:00
antirez 1ed0ad77f0 Cluster: clarified a comment in clusterUpdateSlotsConfigWith(). 2014-03-11 11:32:40 +01:00
antirez 8287945ff8 Cluster: flush importing/migrating state when master is turned into slave. 2014-03-11 11:22:06 +01:00
antirez 2e8e0ad44e Cluster: clusterCloseAllSlots() added. 2014-03-11 11:16:18 +01:00
antirez 8eae54aa1e DEBUG ERROR implemented.
The new "error" subcommand of the DEBUG command can reply with an user
selected error, specified as its sole argument:

    DEBUG ERROR "LOADING please wait..."

The error is generated just prefixing the command argument with a "-"
character, and replacing newlines with spaces (since error replies can't
include newlines).

The goal of the command is to help in Client libraries unit tests by
making simple to simulate a command call triggering a given error.
2014-03-10 23:01:55 +01:00
antirez 2705306ba1 DEBUG CMDKEYS: provide some guarantee to getKeysFromCommand().
getKeysFromCommand() is designed to be called with the command arguments
passing the basic arity checks described in the command table.

DEBUG CMDKEYS must provide the same guarantees for calling
getKeysFromCommand() to be safe.
2014-03-10 16:43:38 +01:00
antirez 5b864617bc Cluster: make sortGetKeys() able to handle multiple STORE options.
It does not make sense to pass multiple store options, so, better to
handle it ;-)
2014-03-10 16:39:07 +01:00
antirez c4ef1d6494 DEBUG CMDKEYS added for getKeysFromCommand() testing.
Examples:

    redis 127.0.0.1:6379> debug cmdkeys set foo bar
    1) "foo"
    redis 127.0.0.1:6379> debug cmdkeys mget a b c
    1) "a"
    2) "b"
    3) "c"
    redis 127.0.0.1:6379> debug cmdkeys zunionstore foo 2 a b
    1) "a"
    2) "b"
    3) "foo"
    redis 127.0.0.1:6379> debug cmdkeys ping
    (empty list or set)
2014-03-10 16:36:08 +01:00
antirez 3e1d772677 Cluster: don't allow BY option of SORT as well.
There is the exception of a "constant" BY pattern that is used in order
to signal to don't sort at all. In this case no lookup is needed so it
is possible to support this case in Cluster mode.
2014-03-10 16:28:18 +01:00
antirez 04cf02e8dc Cluster: SORT get keys helper implemented. 2014-03-10 16:26:08 +01:00
antirez 21765c8588 Cluster: evalGetKeys() fixed: was not setting keys count. 2014-03-10 16:23:42 +01:00
antirez 03344196f3 Cluster: don't allow GET option in cluster mode.
The commit also refactors a bit the error handling during SORT option
parsing.
2014-03-10 16:10:50 +01:00
antirez 8caecc9ab4 Fixed memory leak in SORT LIMIT option argument parsing on error. 2014-03-10 15:44:41 +01:00
antirez ef5e7fbaa2 Cluster: getKeysFromCommand() top comment improved. 2014-03-10 15:31:01 +01:00
antirez c0e818ab08 Cluster: evalGetKey() added for EVAL/EVALSHA.
Previously we used zunionInterGetKeys(), however after this function was
fixed to account for the destination key (not needed when the API was
designed for "diskstore") the two set of commands can no longer be served
by an unique keys-extraction function.
2014-03-10 15:26:13 +01:00
antirez caf7b9b425 Cluster: getKeysFromCommand() and related: top-comments added. 2014-03-10 15:24:38 +01:00
antirez 787b297046 Cluster: getKeysFromCommand() API cleaned up.
This API originated from the "diskstore" experiment, not for Redis
Cluster itself, so there were legacy/useless things trying to
differentiate between keys that are going to be overwritten and keys
that need to be fetched from disk (preloaded).

All useless with Cluster, so removed with the result of code
simplification.
2014-03-10 13:18:41 +01:00
antirez 55b88e0044 Cluster: some zunionInterGetKeys() comment trimmed.
Everything was pretty clear again from the initial statements.
2014-03-10 11:43:56 +01:00
Salvatore Sanfilippo aca6cb529b Merge pull request #1586 from mattsta/fix-zunioninterstorekeys
Fix key extraction for z{union,inter}store
2014-03-10 11:39:45 +01:00
antirez c1a7d3e61f Cluster: abort on port too high error.
It also fixes multi-line comment style to be consistent with the rest of
the code base.

Related to #1555.
2014-03-10 10:41:27 +01:00
Salvatore Sanfilippo 442b06db54 Merge pull request #1555 from mattsta/cluster-port-error-out
Cluster port error out
2014-03-10 10:37:50 +01:00
antirez ed8c55237b Cluster: be explicit about passing NULL as bind addr for connect.
The code was already correct but it was using that bindaddr[0] is set to
NULL as a side effect of current implementation if no bind address is
configured. This is not guarnteed to hold true in the future.
2014-03-10 10:33:53 +01:00
antirez 3e8a92ef8d Cluster: log error when anetTcpNonBlockBindConnect() fails. 2014-03-10 10:32:28 +01:00
Salvatore Sanfilippo 3b0edb80ec Merge pull request #1567 from mattsta/fix-cluster-join
Bind source address for cluster communication
2014-03-10 10:28:32 +01:00
antirez 0f1f25784f Cluster: better timeout and retry time for failover.
When node-timeout is too small, in the order of a few milliseconds,
there is no way the voting process can terminate during that time, so we
set a lower limit for the failover timeout of two seconds.

The retry time is set to two times the failover timeout time, so it is
at least 4 seconds.
2014-03-10 09:57:52 +01:00
Matt Stancliff f0782a6e86 Fix key extraction for z{union,inter}store
The previous implementation wasn't taking into account
the storage key in position 1 being a requirement (it
was only counting the source keys in positions 3 to N).

Fixes antirez/redis#1581
2014-03-07 16:33:20 -05:00
antirez 0f2597092f Typo in sentinel.conf, exists -> exits. 2014-03-07 18:03:51 +01:00
antirez 6984692060 Cluster: fix conditional generating TRYAGAIN error. 2014-03-07 16:18:00 +01:00
antirez 36676c2318 Redis Cluster: support for multi-key operations. 2014-03-07 13:19:09 +01:00
Salvatore Sanfilippo bbf39b7a3a Merge pull request #1576 from Hailei/fix-lruidletime-comment
Fix REDIS_LRU_CLOCK_MAX's value
2014-03-06 18:14:36 +01:00
antirez b74c899da3 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2014-03-06 18:06:30 +01:00
Matt Stancliffandantirez e8bae92e54 Reset op_sec_last_sample_ops when reset requested
This value needs to be set to zero (in addition to
stat_numcommands) or else people may see
a negative operations per second count after they
run CONFIG RESETSTAT.

Fixes antirez/redis#1577
2014-03-06 18:00:08 +01:00
Matt Stancliffandantirez 385c25f70f Remove redundant IP length definition
REDIS_CLUSTER_IPLEN had the same value as
REDIS_IP_STR_LEN.  They were both #define'd
to the same INET6_ADDRSTRLEN.
2014-03-06 17:55:43 +01:00
Matt Stancliffandantirez d2040ab9b1 Remove some redundant code
Function nodeIp2String in cluster.c is exactly
anetPeerToString with a pre-extracted fd.
2014-03-06 17:55:39 +01:00
Matt Stancliffandantirez 59cf0b1902 Fix return value check for anetTcpAccept
anetTcpAccept returns ANET_ERR, not AE_ERR.

This isn't a physical error since both ANET_ERR
and AE_ERR are -1, but better to be consistent.
2014-03-06 17:55:31 +01:00
Salvatore Sanfilippo 54e99fb226 Merge pull request #1578 from badboy/patch-5
Small typo fixed
2014-03-06 17:40:04 +01:00
antirez 9b401819c0 Cast saveparams[].seconds to long for %ld format specifier. 2014-03-05 11:26:18 +01:00
Jan-Erik Rediger 5f5118bdad Small typo fixed 2014-03-05 00:41:02 +01:00
Matt Stancliff e5b1e7be64 Bind source address for cluster communication
The first address specified as a bind parameter
(server.bindaddr[0]) gets used as the source IP
for cluster communication.

If no bind address is specified by the user, the
behavior is unchanged.

This patch allows multiple Redis Cluster instances
to communicate when running on the same interface
of the same host.
2014-03-04 17:36:45 -05:00
antirez 8d011492a0 Sentinel test: set less time sensitive defaults.
This commit sets the failover timeout to 30 seconds instead of the 180
seconds default, and allows to reconfigure multiple slaves at the same
time.

This makes tests less sensible to timing, with the result that there are
less false positives due to normal behaviors that require time to
succeed or to be retried.

However the long term solution is probably some way in order to detect
when a test failed because of timing issues (for example split brain
during leader election) and retry it.
2014-03-04 17:10:29 +01:00
antirez 47750998a6 Sentinel: more aggressive failover start desynchronization.
Sentinel needs to avoid split brain conditions due to multiple sentinels
trying to get voted at the exact same time.

So far some desynchronization was provided by fluctuating server.hz,
that is the frequency of the timer function call. However the
desynchonization provided in this way was not enough when using many
Sentinel instances, especially when a large quorum value is used in
order to force a greater degree of agreement (more than N/2+1).

It was verified that it was likely to trigger a split brain
condition, forcing the system to try again after a timeout.
Usually the system will succeed after a few retries, but this is not
optimal.

This commit desynchronizes instances in a more effective way to make it
likely that the first attempt will be successful.
2014-03-04 17:09:36 +01:00
antirez 08da025f56 CONFIG REWRITE should be logged at WARNING level. 2014-03-04 16:39:47 +01:00
antirez efb092baa6 Sentinel test: debugging console improved. 2014-03-04 15:55:36 +01:00
antirez 7d97a4c99b Sentinel test: initial debugging console. 2014-03-04 12:05:49 +01:00
antirez 3072a1e781 Sentinel test: be more patient in create_redis_master_slave_cluster. 2014-03-04 11:20:53 +01:00
antirez dd8d883c9c Sentiel test: add test start time in output. 2014-03-04 11:17:27 +01:00
zhanghailei 138695d990 refer to updateLRUClock's comment REDIS_LRU_CLOCK_MAX is 22 bits,but #define REDIS_LRU_CLOCK_MAX ((1<<21)-1) only 21 bits 2014-03-04 12:20:31 +08:00
zhanghailei c0f8665414 FIXED a typo more thank should be more than 2014-03-04 11:21:34 +08:00
zhanghailei 4b9ac6edd0 According to context,the size should be 16 rather than 64 2014-03-04 11:21:34 +08:00
antirez c5edd91716 Cluster: invalidate current transaction on redirections. 2014-03-03 17:11:51 +01:00
antirez 35e8bc305d Sentinel test: use 1000 as retry in initial 00 unit test. 2014-03-03 13:26:18 +01:00
antirez 1734f7aedb Sentinel test: initial tests in 03 unit. 2014-03-03 13:25:14 +01:00
antirez db9e718c8a Sentinel test: foreach_instance_id now supports 'continue'. 2014-03-03 13:23:32 +01:00
antirez 69cc3b692f Sentienl test: fixed typo in unit 03 top comment. 2014-03-03 13:01:11 +01:00
antirez e41a3edfab Merge branch 'cli_improved_bigkeys' of git://github.com/michael-grunder/redis into unstable 2014-03-03 11:20:54 +01:00
antirez 12a88d575d Document why we update peak memory in INFO. 2014-03-03 11:19:54 +01:00
antirez 0c1bb1313c Merge branch 'unstable' of github.com:/antirez/redis into unstable 2014-03-03 11:17:37 +01:00
Salvatore Sanfilippo 68d6d1597f Merge pull request #1572 from mattsta/consistent-max-memory-info
Force INFO used_memory_peak to match peak memory
2014-03-03 11:16:51 +01:00
antirez 8dea2029a4 Fix configEpoch assignment when a cluster slot gets "closed".
This is still code to rework in order to use agreement to obtain a new
configEpoch when a slot is migrated, however this commit handles the
special case that happens when the nodes are just started and everybody
has a configEpoch of 0. In this special condition to have the maximum
configEpoch is not enough as the special epoch 0 is not unique (all the
others are).

This does not fixes the intrinsic race condition of a failover happening
while we are resharding, that will be addressed later.
2014-03-03 11:12:11 +01:00
Matt Stancliff f1c9a203b2 Force INFO used_memory_peak to match peak memory
used_memory_peak only updates in serverCron every server.hz,
but Redis can use more memory and a user can request memory
INFO before used_memory_peak gets updated in the next
cron run.

This patch updates used_memory_peak to the current
memory usage if the current memory usage is higher
than the recorded used_memory_peak value.

(And it only calls zmalloc_used_memory() once instead of
twice as it was doing before.)
2014-02-28 17:47:41 -05:00
antirez a89c8bb87c Sentinel test: Makefile target added. 2014-02-28 16:00:00 +01:00
michael-grunder 806788d009 Improved bigkeys with progress, pipelining and summary
This commit reworks the redis-cli --bigkeys command to provide more
information about our progress as well as output summary information
when we're done.

 - We now show an approximate percentage completion as we go
 - Hiredis pipelining is used for TYPE and SIZE retreival
 - A summary of keyspace distribution and overall breakout at the end
2014-02-27 12:01:57 -08:00
antirez 8b8c1cd4c2 BITPOS fuzzy testing. 2014-02-27 15:27:05 +01:00
antirez b21f4d63de Basic BITPOS tests. 2014-02-27 15:01:45 +01:00
antirez 76a6e82d89 warnigns -> warnings in redisBitpos(). 2014-02-27 13:17:23 +01:00
antirez 0e31eaa27f More consistent BITPOS behavior with bit=0 and ranges.
With the new behavior it is possible to specify just the start in the
range (the end will be assumed to be the first byte), or it is possible
to specify both start and end.

This is useful to change the behavior of the command when looking for
zeros inside a string.

1) If the user specifies both start and end, and no 0 is found inside
   the range, the command returns -1.

2) If instead no range is specified, or just the start is given, even
   if in the actual string no 0 bit is found, the command returns the
   first bit on the right after the end of the string.

So for example if the string stored at key foo is "\xff\xff":

    BITPOS foo (returns 16)
    BITPOS foo 0 -1 (returns -1)
    BITPOS foo 0 (returns 16)

The idea is that when no end is given the user is just looking for the
first bit that is zero and can be set to 1 with SETBIT, as it is
"available". Instead when a specific range is given, we just look for a
zero within the boundaries of the range.
2014-02-27 12:53:03 +01:00
antirez 38c620b3b5 Initial implementation of BITPOS.
It appears to work but more stress testing, and both unit tests and
fuzzy testing, is needed in order to ensure the implementation is sane.
2014-02-27 12:44:27 +01:00
antirez addd4de9c1 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2014-02-27 10:14:03 +01:00
antirez 8d95a47408 Added two more BITCOUNT tests stressing misaligned access. 2014-02-27 10:07:29 +01:00
antirez 2a7847a3b5 BITCOUNT fuzzy test with random start/end added.
It was verified in practice that this test is able to stress much more
the implementation by introducing errors that were only trivially to
detect with different offsets but impossible to detect starting always
at zero and counting bits the full length of the string.
2014-02-27 10:00:17 +01:00
antirez 746ce35f5f Fix misaligned word access in redisPopcount(). 2014-02-27 09:46:20 +01:00
Salvatore Sanfilippo 5edc7f76ed Merge pull request #1565 from mattsta/fix-cluster-ipv6-addr
Fix IP representation in clusterMsgDataGossip
2014-02-25 22:07:46 +01:00
Matt Stancliff d769cad4bf Fix IP representation in clusterMsgDataGossip 2014-02-25 16:02:28 -05:00
antirez 55c059e270 Sentinel test: add stub for unit 04. 2014-02-25 15:36:51 +01:00
antirez fe9a489b29 Sentinel test: added TODO items in 02 unit. 2014-02-25 15:21:53 +01:00
antirez 55e36e1132 Merge branch 'bigkeys_scan' of git://github.com/michael-grunder/redis into unstable 2014-02-25 14:59:57 +01:00
antirez 339322fe58 Sentinel test: check role at end of unit 01. 2014-02-25 14:59:36 +01:00
michael-grunder 013a4ce242 Update --bigkeys to use SCAN
This commit changes the findBigKeys() function in redis-cli.c to use the new
SCAN command for iterating the keyspace, rather than RANDOMKEY.  Because we
can know when we're done using SCAN, it will exit after exhausting the keyspace.
2014-02-25 05:41:30 -08:00
antirez fef88681f2 Sentinel test: kill masters instead of using DEBUG SLEEP in all tests. 2014-02-25 14:33:44 +01:00
antirez a2c76ffb1c redis-cli: also remove useless uint8_t. 2014-02-25 13:47:37 +01:00
antirez ba993cc685 redis-cli: don't use uint64_t where actually not needed.
The computation is just something to take the CPU busy, no need to use a
specific type. Since stdint.h was not included this prevented
compilation on certain systems.
2014-02-25 13:44:31 +01:00
antirez 5580350a7b redis-cli: check argument existence for --pattern. 2014-02-25 12:38:29 +01:00
antirez c1d67ea9b4 redis-cli: --intrinsic-latency run mode added. 2014-02-25 12:37:52 +01:00
antirez dcac007b81 redis-cli: added comments to split program in parts. 2014-02-25 12:24:45 +01:00
antirez 386467acfb Sentinel test: restart instances left killed by previous unit.
An unit can abort in the middle for an error. The next unit should not
assume that the instances are in a clean state, and must restart what
was left killed.
2014-02-25 08:48:46 +01:00
antirez a9360c62e8 Sentinel test: jump to next unit on test failure.
Sentinel tests are designed to be dependent on the previous tests in the
same unit, so usually we can't continue with the next test in the same
unit if a previous test failed.
2014-02-25 08:33:41 +01:00
antirez 044b627549 Sentinel test: test majority crashing Sentinels.
The test was previously performed by removing the master from the
Sentinel monitored masters. The test with the Sentinels crashed is
more similar to real-world partitions / failures.
2014-02-25 08:29:12 +01:00
antirez 630fb3539f Sentinel test: restart_instance should refresh pid attrib.
Also kill_instance was modified to warn when a test will try to kill the
same instance multiple times for error.
2014-02-25 08:23:48 +01:00
antirez d3a3ef0bc1 Sentinel test: more stuff mored from 00-base to init.
The area a number of mandatory tests to craete a stable setup for
testing that is not too sensitive to timing issues. All those tests
moved to includes/init-tests, and marked as (init).
2014-02-24 17:21:50 +01:00
antirez b15411df98 Sentinel: log quorum with +monitor event. 2014-02-24 17:10:20 +01:00
antirez 29f4df8018 Sentinel test: removed useless code to set SDOWN timeout.
The new common initialization code used to start a new unit already set
the timeout to 2000 milliseconds.
2014-02-24 16:57:52 +01:00
antirez 6b373edb77 Sentinel: generate +monitor events at startup. 2014-02-24 16:33:55 +01:00
antirez 3b7a757468 Sentinel: log +monitor and +set events.
Now that we have a runtime configuration system, it is very important to
be able to log how the Sentinel configuration changes over time because
of API calls.
2014-02-24 16:33:43 +01:00
antirez 25cebf7285 Sentinel: added missing exit(1) after checking for config file. 2014-02-24 16:22:52 +01:00
antirez 540536c055 Sentinel test: tmp dir and gitignore added. 2014-02-24 11:51:31 +01:00
antirez 09dec3613e Sentinel test: minor fixes to --pause-on-error. 2014-02-23 18:02:52 +01:00
antirez afd3db17a0 Sentinel test: --pause-on-error option added.
Pause the test with running instances available for state inspection on
error.
2014-02-23 17:57:56 +01:00
antirez a929867cca Sentinel test: added empty units to fill later. 2014-02-23 17:50:59 +01:00
Salvatore Sanfilippo e163332858 Merge pull request #1545 from mattsta/fix-redis-cli-sync
Deny SYNC and PSYNC in redis-cli
2014-02-23 17:47:28 +01:00
antirez b1c1386374 Sentinel: IDONTKNOW error removed.
This error was conceived for the older version of Sentinel that worked
via master redirection and that was not able to get configuration
updates from other Sentinels via the Pub/Sub channel of masters or
slaves.

This reply does not make sense today, every Sentinel should reply with
the best information it has currently. The error will make even more
sense in the future since the plan is to allow Sentinels to update the
configuration of other Sentinels via gossip with a direct chat without
the prerequisite that they have at least a monitored instance in common.
2014-02-22 17:34:46 +01:00
antirez 8c254415f7 Sentinel test: framework improved and conf-update unit added.
It is now possible to kill and restart sentinel or redis instances for
more real-world testing.

The 01 unit tests the capability of Sentinel to update the configuration
of Sentinels rejoining the cluster, however the test is pretty trivial
and more tests should be added.
2014-02-22 17:27:49 +01:00
Salvatore Sanfilippo 1d7d1d9b1f Merge pull request #1559 from mattsta/more-detailed-process-title
Add cluster or sentinel to proc title
2014-02-21 09:32:13 +01:00
Matt Stancliff 2c273e3591 Add cluster or sentinel to proc title
If you launch redis with `redis-server --sentinel` then
in a ps, your output only says "redis-server IP:Port" — this
patch changes the proc title to include [sentinel] or
[cluster] depending on the current server mode:
e.g.  "redis-server IP:Port [sentinel]"
      "redis-server IP:Port [cluster]"
2014-02-20 23:58:54 -05:00
antirez d7da507683 Sentinel test: move init tests as includes.
Most units will start with these two basic tests to create an
environment where the real tests are ran.
2014-02-20 16:58:23 +01:00
antirez 5765444454 Sentinel test: ability to run just a subset of test files. 2014-02-20 16:28:41 +01:00
antirez 7d7b3810e7 Sentinel: report instances role switch events.
This is useful mostly for debugging of issues.
2014-02-20 12:13:52 +01:00
Matt Stancliff ce68caea37 Cluster: error out quicker if port is unusable
The default cluster control port is 10,000 ports higher than
the base Redis port.  If Redis is started on a too-high port,
Cluster can't start and everything will exit later anyway.
2014-02-19 17:30:07 -05:00
Matt Stancliff b20ae393f1 Fix "can't bind to address" error reporting.
Report the actual port used for the listening attempt instead of
server.port.

Originally, Redis would just listen on server.port.
But, with clustering, Redis uses a Cluster Port too,
so we can't say server.port is always where we are listening.

If you tried to launch Redis with a too-high port number (any
port where Port+10000 > 65535), Redis would refuse to start, but
only print an error saying it can't connect to the Redis port.

This patch fixes much confusions.
2014-02-19 17:26:33 -05:00
antirez e087d8a20d Sentinel test: some reliability fixes to 00-base tests. 2014-02-19 10:26:23 +01:00
antirez a88a057a1f Sentinel test: check that role matches at end of 00-base. 2014-02-19 10:08:49 +01:00
antirez 2a08c7e5ac Sentinel test: ODOWN and agreement. 2014-02-19 09:44:38 +01:00
antirez 136537dcb0 Sentinel test: check reconfig of slaves and old master. 2014-02-18 17:03:56 +01:00
antirez 8e553ec67c Sentinel test: basic failover tested. Framework improvements. 2014-02-18 16:31:52 +01:00
antirez c7b7439528 Sentinel test: basic tests for MONITOR and auto-discovery. 2014-02-18 11:53:54 +01:00
antirez c4fbc1d336 Sentinel test: info fields, master-slave setup, fixes. 2014-02-18 11:38:49 +01:00
antirez 19b863c7fa Prefix test file names with numbers to force exec order. 2014-02-18 11:07:42 +01:00
antirez 141bac4c79 Sentinel test: provide basic commands to access instances. 2014-02-18 11:04:55 +01:00
antirez 7cec9e48ce Sentinel: SENTINEL_SLAVE_RECONF_RETRY_PERIOD -> RECONF_TIMEOUT
Rename define to match the new meaning.
2014-02-18 10:27:38 +01:00
antirez 18b8bad53c Sentinel: fix slave promotion timeout.
If we can't reconfigure a slave in time during failover, go forward as
anyway the slave will be fixed by Sentinels in the future, once they
detect it is misconfigured.

Otherwise a failover in progress may never terminate if for some reason
the slave is uncapable to sync with the master while at the same time
it is not disconnected.
2014-02-18 08:50:57 +01:00
antirez af788b5852 Sentinel: initial testing framework.
Nothing tested at all so far... Just the infrastructure spawning N
Sentinels and N Redis instances that the test will use again and again.
2014-02-17 17:38:04 +01:00
antirez 34c404e069 Test: colorstr moved to util.tcl. 2014-02-17 17:36:50 +01:00
antirez a1dca2efab Test: code to test server availability refactored.
Some inline test moved into server_is_up procedure.
Also find_available_port was moved into util since it is going
to be used for the Sentinel test as well.
2014-02-17 16:44:57 +01:00
antirez ede33fb912 Get absoulte config file path before processig 'dir'.
The code tried to obtain the configuration file absolute path after
processing the configuration file. However if config file was a relative
path and a "dir" statement was processed reading the config, the absolute
path obtained was wrong.

With this fix the absolute path is obtained before processing the
configuration while the server is still in the original directory where
it was executed.
2014-02-17 16:44:53 +01:00
antirez e1b77b61f3 Sentinel: better specify startup errors due to config file.
Now it logs the file name if it is not accessible. Also there is a
different error for the missing config file case, and for the non
writable file case.
2014-02-17 16:44:49 +01:00
antirez 51bd9da1fd Update cached time in rdbLoad() callback.
server.unixtime and server.mstime are cached less precise timestamps
that we use every time we don't need an accurate time representation and
a syscall would be too slow for the number of calls we require.

Such an example is the initialization and update process of the last
interaction time with the client, that is used for timeouts.

However rdbLoad() can take some time to load the DB, but at the same
time it did not updated the time during DB loading. This resulted in the
bug described in issue #1535, where in the replication process the slave
loads the DB, creates the redisClient representation of its master, but
the timestamp is so old that the master, under certain conditions, is
sensed as already "timed out".

Thanks to @yoav-steinberg and Redis Labs Inc for the bug report and
analysis.
2014-02-13 15:13:26 +01:00
antirez 7e8abcf693 Log when CONFIG REWRITE goes bad. 2014-02-13 14:32:44 +01:00
antirez f2bdf601be Test: regression for issue #1549.
It was verified that reverting the commit that fixes the bug, the test
no longer passes.
2014-02-13 12:26:38 +01:00
antirez 21e6b0fbe9 Fix script cache bug in the scripting engine.
This commit fixes a serious Lua scripting replication issue, described
by Github issue #1549. The root cause of the problem is that scripts
were put inside the script cache, assuming that slaves and AOF already
contained it, even if the scripts sometimes produced no changes in the
data set, and were not actaully propagated to AOF/slaves.

Example:

    eval "if tonumber(KEYS[1]) > 0 then redis.call('incr', 'x') end" 1 0

Then:

    evalsha <sha1 step 1 script> 1 0

At this step sha1 of the script is added to the replication script cache
(the script is marked as known to the slaves) and EVALSHA command is
transformed to EVAL. However it is not dirty (there is no changes to db),
so it is not propagated to the slaves. Then the script is called again:

    evalsha <sha1 step 1 script> 1 1

At this step master checks that the script already exists in the
replication script cache and doesn't transform it to EVAL command. It is
dirty and propagated to the slaves, but they fail to evaluate the script
as they don't have it in the script cache.

The fix is trivial and just uses the new API to force the propagation of
the executed command regardless of the dirty state of the data set.

Thank you to @minus-infinity on Github for finding the issue,
understanding the root cause, and fixing it.
2014-02-13 12:10:43 +01:00
antirez fc08c8599f AOF write error: retry with a frequency of 1 hz. 2014-02-12 16:27:59 +01:00
antirez fe8352540f AOF: don't abort on write errors unless fsync is 'always'.
A system similar to the RDB write error handling is used, in which when
we can't write to the AOF file, writes are no longer accepted until we
are able to write again.

For fsync == always we still abort on errors since there is currently no
easy way to avoid replying with success to the user otherwise, and this
would violate the contract with the user of only acknowledging data
already secured on disk.
2014-02-12 16:11:36 +01:00
antirez db6d628c3e Cluster: clusterDelNode(): remove node from master's slaves. 2014-02-11 10:34:25 +01:00
antirez 5e0e03be41 Cluster: UPDATE messages are the norm and verbose.
Logging them at WARNING level was of little utility and of sure disturb.
2014-02-11 10:18:24 +01:00
antirez 8251d2d150 Cluster: redis-trib fix: handling of another trivial case. 2014-02-11 10:13:18 +01:00
antirez 4a64286c36 Cluster: configEpoch assignment in SETNODE improved.
Avoid to trash a configEpoch for every slot migrated if this node has
already the max configEpoch across the cluster.

Still work to do in this area but this avoids both ending with a very
high configEpoch without any reason and to flood the system with fsyncs.
2014-02-11 10:09:17 +01:00
antirez 72f7abf6a2 Cluster: clusterSetStartupEpoch() made more generally useful.
The actual goal of the function was to get the max configEpoch found in
the cluster, so make it general by removing the assignment of the max
epoch to currentEpoch that is useful only at startup.
2014-02-11 10:00:14 +01:00
antirez 44f7afe28a Cluster: always increment the configEpoch in SETNODE after import.
Removed a stale conditional preventing the configEpoch from incrementing
after the import in certain conditions. Since the master got a new slot
it should always claim a new configuration.
2014-02-11 09:50:37 +01:00
antirez a1349728ea Cluster: on resharding upgrade version of receiving node.
The node receiving the hash slot needs to have a version that wins over
the other versions in order to force the ownership of the slot.

However the current code is far from perfect since a failover can happen
during the manual resharding. The fix is a work in progress but the
bottom line is that the new version must either be voted as usually,
set by redis-trib manually after it makes sure can't be used by other
nodes, or reserved configEpochs could be used for manual operations (for
example odd versions could be never used by slaves and are always used
by CLUSTER SETSLOT NODE).
2014-02-11 00:36:05 +01:00
antirez 6dc26795aa Cluster: fsync at every SETSLOT command puts too pressure on disks.
During slots migration redis-trib can send a number of SETSLOT commands.
Fsyncing every time is a bit too much in production as verified
empirically.

To make sure configs are fsynced on all nodes after a resharding
redis-trib may send something like CLUSTER CONFSYNC.

In this case fsyncs were not providing too much value since anyway
processes can crash in the middle of the resharding of an hash slot, and
redis-trib should be able to recover from this condition anyway.
2014-02-10 23:54:08 +01:00
antirez 218358bbbd Cluster: conditions to clear "migrating" on slot for SETSLOT ... NODE changed.
If the slot is manually assigned to another node, clear the migrating
status regardless of the fact it was previously assigned to us or not,
as long as we no longer have keys for this slot.

This avoid a race during slots migration that may leave the slot in
migrating status in the source node, since it received an update message
from the destination node that is already claiming the slot.

This way we are sure that redis-trib at the end of the slot migration is
always able to close the slot correctly.
2014-02-10 23:51:47 +01:00
antirez 3107e7ca60 Cluster: remove debugging xputs from redis-trib. 2014-02-10 19:14:05 +01:00
antirez 1ae50a9b1d Cluster: redis-trib fix: cover new case of open slot.
The case is the trivial one a single node claiming the slot as
migrating, without nodes claiming it as importing.
2014-02-10 19:10:23 +01:00
antirez 59e03a8f35 redis-trib: log event after we have reference to 'master'. 2014-02-10 18:48:40 +01:00
antirez bf670e0745 Cluster: don't update slave's master if we don't know it.
There is no way we can update the slave's node->slaveof pointer if we
don't know the master (no node with such an ID in our tables).
2014-02-10 18:33:34 +01:00
antirez a3755ae9ee Cluster: ignore slot config changes if we are importing it. 2014-02-10 18:04:43 +01:00
antirez 6fc53e16ad Cluster: update configEpoch after manually messing with slots. 2014-02-10 18:01:58 +01:00
antirez be0bb19fd3 Cluster: redis-trib, more info about open slots error. 2014-02-10 17:44:16 +01:00
antirez 1a73c992a3 Cluster: fixed inverted arguments in logging function call. 2014-02-10 17:21:10 +01:00
antirez 32563b4a5f Cluster: clear the FAIL status for masters without slots.
Masters without slots don't participate to the cluster but just do
redirections, no need to take them in FAIL state if they are back
reachable.
2014-02-10 17:18:27 +01:00
Matt Stancliff 21648473aa Auto-enter slaveMode when SYNC from redis-cli
If someone asks for SYNC or PSYNC from redis-cli,
automatically enter slaveMode (as if they ran
redis-cli --slave) and continue printing the replication
stream until either they Ctrl-C or the master gets disconnected.
2014-02-10 11:10:31 -05:00
antirez 5b2082ead3 Cluster: replica migration should only work for masters serving slots. 2014-02-10 17:08:37 +01:00
antirez f106a79309 Cluster: redis-trib del-node variable typo fixed. 2014-02-10 16:59:09 +01:00
antirez f885fa8bac Cluster: clusterReadHandler() fixed to work with new message header. 2014-02-10 16:27:37 +01:00
antirez 344a065d51 Cluster: don't propagate PUBLISH two times.
PUBLISH both published messages via Cluster bus and replication when
cluster was enabled, resulting in duplicated message in the slave.
2014-02-10 16:00:27 +01:00
antirez 7bf7b7350c Cluster: signature changed to "RCmb" (Redis Cluster message bus).
Sounds better after all.
2014-02-10 15:55:21 +01:00
antirez dced9c0619 Cluster: discard bus messages with version != 0. 2014-02-10 15:54:22 +01:00
antirez 007e1c7cb2 Cluster: added signature + version in bus packets. 2014-02-10 15:53:09 +01:00
antirez 100cd7b21e Added a release notes file good for "unstable". 2014-02-10 15:38:54 +01:00
88 changed files with 1259 additions and 4572 deletions
+11 -643
View File
@@ -1,648 +1,16 @@
Redis 3.0 release notes
=======================
Hello! This file is just a placeholder, since this is the "unstable" branch
of Redis, the place where all the development happens.
--------------------------------------------------------------------------------
Upgrade urgency levels:
There is no release notes for this branch, it gets forked into another branch
every time there is a partial feature freeze in order to eventually create
a new stable release.
LOW: No need to upgrade unless there are new features you want to use.
MODERATE: Program an upgrade of the server, but it's not urgent.
HIGH: There is a critical bug that may affect a subset of users. Upgrade!
CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
--------------------------------------------------------------------------------
Usually "unstable" is stable enough for you to use it in development environments
however you should never use it in production environments. It is possible
to download the latest stable release here:
--[ Redis 3.0.3 ] Release date: 17 Jul 2015
http://download.redis.io/releases/redis-stable.tar.gz
Upgrade urgency: LOW for Redis and Sentinel.
More information is available at http://redis.io
* [FIX] Fix blocking operations timeout precision when HZ is at its default
value (not increased) and there are thousands of clients connected
at the same time. This bug affected Sidekiq users that experienced
a very long delay for BLPOP and similar commands to return for
timeout. Check commit b029ff1 for more info. (Salvatore Sanfilippo)
* [FIX] MIGRATE "creating socket: Invalid argument" error fix. Check
issues #2609 and #2612 for more info. (Salvatore Sanfilippo)
* [FIX] Be able to connect to the master even when the slave is bound to
just the loopback interface and has no valid public address in the
network the master is reacahble. (Salvatore Sanfilippo)
* [FIX] ZADD with options encoding promotion fixed. (linfangrong)
* [FIX] Reset aof_delayed_fsync on CONFIG RESETSTATS. (Tom Kiemes)
* [FIX] PFCOUNT key parsing in cluster fixed. (MOON_CLJ)
* [FIX] Fix Solaris compilation of Redis 3.0. (Jan-Erik Rediger)
* [NEW] Variadic EXISTS command. Now the command accepts multiple arguments
and returns the total count of existing keys.
--[ Redis 3.0.2 ] Release date: 4 Jun 2015
Upgrade urgency: HIGH for Redis because of a security issue.
LOW for Sentinel.
* [FIX] Critical security issue fix by Ben Murphy: http://t.co/LpGTyZmfS7
* [FIX] SMOVE reply fixed when src and dst keys are the same. (Glenn Nethercutt)
* [FIX] Lua cmsgpack lib updated to support str8 type. (Sebastian Waisbrot)
* [NEW] ZADD support for options: NX, XX, CH. See new doc at redis.io.
(Salvatore Sanfilippo)
* [NEW] Senitnel: CKQUORUM and FLUSHCONFIG commands back ported.
(Salvatore Sanfilippo and Bill Anderson)
--[ Redis 3.0.1 ] Release date: 5 May 2015
Upgrade urgency: LOW for Redis and Cluster, MODERATE for Sentinel.
* [FIX] Sentinel memory leak due to hiredis fixed. (Salvatore Sanfilippo)
* [FIX] Sentinel memory leak on duplicated instance. (Charsyam)
* [FIX] Redis crash on Lua reaching output buffer limits. (Yossi Gottlieb)
* [FIX] Sentinel flushes config on +slave events. (Bill Anderson)
--[ Redis 3.0.0 ] Release date: 1 Apr 2015
>> What's new in Redis 3.0 compared to Redis 2.8?
* Redis Cluster: a distributed implementation of a subset of Redis.
* New "embedded string" object encoding resulting in less cache
misses. Big speed gain under certain work loads.
* AOF child -> parent final data transmission to minimize latency due
to "last write" during AOF rewrites.
* Much improved LRU approximation algorithm for keys eviction.
* WAIT command to block waiting for a write to be transmitted to
the specified number of slaves.
* MIGRATE connection caching. Much faster keys migraitons.
* MIGRATE new options COPY and REPLACE.
* CLIENT PAUSE command: stop processing client requests for a
specified amount of time.
* BITCOUNT performance improvements.
* CONFIG SET accepts memory values in different units (for example
you can use "CONFIG SET maxmemory 1gb").
* Redis log format slightly changed reporting in each line the role of the
instance (master/slave) or if it's a saving child log.
* INCR performance improvements.
>> Refactoring changes (no new features nor bug fixes)
* Blocking operations full refactoring (blocked.c)
* Client output buffer memory tracking refactored.
Changes between RC6 and 3.0.0 stable:
>> General changes
* Fixes to diskless replication. (Oran Agra)
* Test for BLPOP replication on role change. (Salvatore Sanfilippo)
* prepareClientToWrite() error handling improvements. (Salvatore Sanfilippo)
* Remove dict.c no longer used function. (Salvatore Sanfilippo)
>> Cluster changes
None
>> Sentinel changes
None
--[ Redis 3.0.0 RC6 (version 2.9.106) ] Release date: 24 mar 2015
Upgrade urgency: HIGH because of bugs related to Redis Custer and replication.
This is the 6th release candidate of Redis 3.0.0. This release fixes important
issues discovered during stress testing, and implements safest behavior
for blocking operations during clients reshardings, and a new much needed
functionality of Redis Cluster manual failovers.
In order to fix certain bugs quite a bit of refactoring was needed which
is usually non advisabble in a Release Candidate, but needed in order to
end with a clean fix.
>> General changes
* [FIX] Redis (non clustered & clustered) replication bug involving blocking
operations: see issue #2473. (Salvatore Sanfilippo)
>> Cluster changes
* [FIX] clientsArePaused() fix crashing the old master during manual failover.
(Salvatore Sanfilippo)
* [FIX] Lua scripts replication in Redis Cluster was totally broken.
(Salvatore Sanfilippo)
* [FIX] Redirect clients blocked into list operations when the hash slot
they are blocked into is migrated to another instance or the cluster
state turns into "fail". (Salvatore Sanfilippo)
* [NEW] TAKEOVER option for CLUSTER FAILOVER implemented. It is now possible
to fix a cluster manually in the minority side of the partition, for
example in order to allow for multi DC setups & recovery.
(Salvatore Sanfilippo)
>> Sentinel changes
No changes in Sentinel.
--[ Redis 3.0.0 RC5 (version 2.9.105) ] Release date: 20 mar 2015
Upgrade urgency: Moderate for Redis Cluster users, low otherwise.
This is the 5th release candidate of Redis 3.0.0, released in order to fix
a moderate bug in Redis Cluster. This RC does not shift in the future the
Redis 3.0.0 final release which is scheduled in a few days (we are in the
process of finishing the documentation for Redis Cluster).
>> General changes
* [FIX] Fix LATENCY command crash. (Salvatore Sanfilippo, thx to Ingmar)
* [FIX] Config: missing activerehashing option support in CONFIG SET added.
(Salvatore Sanfilippo, thx to Bill Anderson)
* [FIX] Fix for backtrace generation issue. (Mariano Pérez Rodríguez, Matt Stancliff, Salvatore Sanfilippo)
* [NEW] Redis-cli --latency-dist backported from unstable.
(Salvatore Sanfilippo)
>> Cluster changes
* [FIX] Avoid redundant SELECT in MIGRATE. (Tommy Wang, Salvatore Sanfilippo)
* [FIX] More robust slave check in CLUSTER REPLICATE. (Salvatore Sanfilippo)
* [FIX] Fixed possible Redis Cluster node crash due to wrong separation of
concerns between getNodeByQuery() and Cluster global state update
fnuction. (Salvatore Sanfilippo, thx to Ingmar)
* [NEW] Add command CLUSTER MYID to easily featch instance ID. (Michel Martens)
>> Sentinel changes
* [NEW] Support for CLIENT command added. It was missing in the command table.
(Leandro López)
--[ Redis 3.0.0 RC4 (version 2.9.104) ] Release date: 13 feb 2015
Upgrade urgency: High for Redis if you use LRU eviction, low otherwise.
This is the 4th release candidate of Redis 3.0.0, it fixes problems with
LRU eviction that are not present in older release (2.8.x is not affected)
and adds new tools to inspect latency and load-test LRU.
>> General changes
* [FIX] redis-cli CSV output NIL spurious newline removed. (Matt Collier)
* [FIX] Memory efficiency test in unit test is now much faster: it affacted
the total "make test" execution time in a bad way. (Salvatore
Sanfilippo)
* [FIX] Fixes and improvements to dict.c and LRU eviction. Redis 3.0.0 new
LRU eviction had bugs creating high latency spikes when LRU was
happening during the keys dictionary rehashing. This bug is not
present into 2.8, was 3.0 specific. As a side effect of this issue
dict.c is now improved, and LRU algorithm is more precise (better
approximates true LRU). This was a joint effort, see issue
#2306 for details. (Oran Agra, Sun He, Salvatore Sanfilippo).
Thanks to Charsyam for spotting an integer overflow.
* [NEW] New latency tool: redis-cli --latency-dist is able to show an
xterm-256 based spectrum of latencies over time. (Salvatore Sanfilippo)
* [NEW] redis-cli --lru-test implemented (cache workload simulator). (Salvatore
Sanfilippo)
* [NEW] redis-cli --stat now shows LOAD when Redis is loading data.
* [NEW] Support "1G" etc. units in CONFIG SET. (Chris Lamb, Salvatore
Sanfilippo)
>> Cluster changes
* None.
>> Sentinel changes
* None.
--[ Redis 3.0.0 RC3 (version 2.9.103) ] Release date: 30 jan 2015
Upgrade urgency: High for Redis Cluster users, low otherwise.
This is the third release candidate for Redis 3.0.0, the new RC fixes
several critical issues with Redis Cluster.
>> General changes
* [FIX] AOF bug unlikely to happen in practice and mostly harmless: child
process segfaults when parent is not reachable via pipe. (Sun He)
* [FIX] Scripting engine now reports an error when misused with Lua debug
hooks, instead of crashing. (Salvatore Sanfilippo)
>> Cluster changes
* [FIX] Several issues with Redis Cluster internal nodes objects handling.
(Matt Stancliff & Salvatore Sanfilippo)
* [FIX] Improvements in the Cluster test.
(Matt Stancliff & Salvatore Sanfilippo).
* [FIX] Cluster memory leaks / double frees (Matt Stancliff).
* [FIX] /dev/urandom surrogate for generation of unique IDs in a more
cheap way. (Salvatore Sanfilippo)
* [FIX] Fixes and improvements to PING / PONG packets gossip sections
in order to improve (and fix) failure detection and speedup
cluster info propagation. (Salvatore Sanfilippo)
* [NEW] CLUSTER count-failure-reports command added. (Salvatore Sanfilippo)
>> Sentinel changes
No changes for Sentinel.
--[ Redis 3.0.0 RC2 (version 2.9.102) ] Release date: 13 jan 2015
Upgrade urgency: LOW.
This is the second release candidate of Redis 3.0.0. The major changes
are back porting of things implemented into the unstable branch while
this was still possible (with the new development model adopted only
bug fixes will be merged in the future).
RC2 also fixes a few Redis Cluster non critical bugs.
>> General changes
* [FIX] A number of minor bug fixes.
* [NEW] Diskless replication backportede.
* [NEW] Lua bitops and updated cmsgpack backported.
* [NEW] Transparent Huge Pages warnings and reporting backported.
>> Cluster changes
* [FIX] Fix PUBLISH cluster bus message count field.
* [FIX] It is no longer possible to write outside node hash slots using Lua.
* [FIX] Valgrind warnings (no actual bugs).
* [FIX] Less strict in acceptiong myself->ip if it's not populated.
* [NEW] Better testing of Lua scripts.
>> Sentinel changes
No changes to Sentinel.
--[ Redis 3.0.0 RC1 (version 2.9.101) ] Release date: 9 oct 2014
This is the first release candidate of Redis 3.0.0.
>> General changes
* [FIX] An very large number of small fixes, old and new, merged in the
context of a the issue #1906. Please see the issue page here
for exact credits: https://github.com/antirez/redis/pull/1906
of each commit. (Matt Stancliff and many others).
* [FIX] SAVE is no longer propagated to AOF / slaves.
* [FIX] GETRANGE test no longer fails for 32 bit builds (Matt Stancliff).
* [FIX] Limit SCAN latency when the hash table is in an odd state (very few
populted buckets because rehashing is in progress). (Xiaost and
Salvatore Sanfilippo)
* [NEW] Redis is now able to load truncated AOF files without requiring a
redis-check-aof utility run. The default now is to load truncated
(but apparently not corrupted) AOFs, you can change this in redis.conf.
(Salvatore Sanfilippo).
* [NEW] DEBUG POPULATE two args form implemented. It is now possible to
call it with DEBUG POPULATE <count> <prefix>. Default prefix
is "key:" as usually.
* [NEW] INCR: Modify incremented object in-place when possible. This results
in speed improvements + possibly better memory locality.
>> Cluster changes
* [FIX] Cluster: claim ping_sent time even if we can't connect.
* [FIX] redis-trib should not abort easily on connection issues.
* [FIX] Cluster test: less console-spammy resharding test.
* [FIX] Fix logic to detect we are among a minority.
* [FIX] Process gossip section only for known nodes.
* [NEW] Redis Cluster is stable and tested enough, there is a clear MVP,
so it was promoted from beta to stable.
* [NEW] New unit 09, Pub/Sub across the cluster.
* [NEW] New unit 08, update messages.
* [NEW] New cluster option to work with partial slots coverage.
* [NEW] More chatty cluster slaves when failover is stalled. They log reason
with rate limiting, only when reason changes or a given time
has elapsed.
>> Sentinel changes
* [FIX] Sentinel critical bug fixed: the absolute majority was computed in a
wrong way because of a programming error. Now the implementation does
what the specification says and the majority to authorize a failover
(that should not be confused with the ODOWN quorum) is the majority of
*all* the Sentinels ever seen for a given master, regardless of their
current state.
* [FIX] Resolved a memory leak in the hiredis library causing a memory leak
in Redis Sentinel when a monitored instance or another Sentinel is
unavailable. Every reconnection attempt will leak a small amount of
memory, but in the long run the process can reach a considerable size.
* [NEW] Sentinel: ability to announce itself with an arbitrary IP/port to work
in the context of natted networks. However this is probably still
not enough since there is no equivalent mechanism for slaves listed
in the master INFO output. (Dara Kong and Salvatore Sanfilippo)
--[ Redis 3.0.0 Beta 8 (version 2.9.57) ] Release date: 29 jul 2014
This is the 8th beta of Redis 3.0.0.
>> General changes
* [FIX] Solaris compilation issues. (Matt Stancliff, Salvatore Sanfilippo)
* [FIX] Allow shared integer objects if maxmemory policy is not LRU based.
(Salvatore Sanfilippo)
* [FIX] PFSELFTEST: less false positives. (Salvatore Sanfilippo)
* [FIX] Fail SYNC if background save child aborted due to a signal. (Yossi
Gottlieb)
* [NEW] Latency framework backported from unstable branch. (Salvatore
Sanfilippo)
* [NEW] AOF rewrite improved for latency. (Salvatore Sanfilippo)
* [NEW] Pub/Sub PING. (Salvatore Sanfilippo)
* [NEW] Much faster ZUNIONSTORE. (Kyle Hubert, Salvatore Sanfilippo)
* [NEW] Faster ll2string() implementation. (Salvatore Sanfilippo)
>> Cluster changes
* [FIX] CLUSTER RESET: Flush slave dataset on reset. (Salvatore Sanfilippo)
* [FIX] Replica migration: don't migrate to masters that never had slaves
in the past, but only to masters that remained orphaned after
failure events. (Salvatore Sanfilippo)
* [NEW] redis-trib: allow to reshard in non-interactive way. (Salvatore
Sanfilippo)
* [NEW] Cluster test: unit 04, check consistency during resharding. (Salvatore
Sanfilippo)
* [NEW] Cluster test: unit 05, slave selection. (Salvatore Sanfilippo)
* [NEW] Cluster test: unit 06, slaves with stale data can't failover. (Salvatore
Sanfilippo)
* [NEW] Cluster test: unit 07, replicas migration. (Salvatore Sanfilippo)
>> Sentinel changes
* No Sentinel changes in this release.
--[ Redis 3.0.0 Beta 7 (version 2.9.56) ] Release date: 30 jun 2014
This is the 7th beta of Redis 3.0.0.
>> General changes
* [FIX] Scripting fixes backported from unstable, see Redis 2.8.12 changelog
for more info. (Salvatore Sanfilippo)
* [FIX] Cancel SHUTDOWN if initial AOF is being written. (Matt Stancliff)
* [NEW] New command: COMMAND, for commands introspection (Matt Stancliff &
Salvatore Sanfilippo)
* [NEW] hiredis: Update to latest version. (Matt Stancliff)
* [NEW] Jemalloc updated to 3.6.0. (Salvatore Sanfilippo)
>> Cluster changes
* [FIX] Cluster: clear NOADDR flag when updating node address.
(Salvatore Sanfilippo)
* [NEW] New CLUSTER SLOTS command to simplify Cluster clients operations.
(Matt Stancliff)
* [NEW] More Cluster tests. (Salvatore Sanfilippo)
* [NEW] Log when failover authorization are granted / denied.
(Salvatore Sanfilippo)
>> Sentinel changes
* [FIX] A few Sentinel bugs fixed and improvements, see Redis 2.8.12
changelog for more info. (Salvatore Sanfilippo & Matt Stancliff)
* [NEW] New Sentinel-Client handshake protocol, ROLE command, CLIENT KILL,
all backported to 3.0 branch. (Salvatore Sanfilippo)
--[ Redis 3.0.0 Beta 6 (version 2.9.55) ] Release date: 9 jun 2014
This is the 6th beta of Redis 3.0.0.
>> General changes
* [FIX] Fix software watchdog signal handler crash due to re-entering.
* [FIX] Better Lua number -> string conversion for Lua scripts.
* [FIX] Serious replication bug when min-slaves-* feature is used in slaves
configuration fixed.
* [FIX] Blocking pop on lists now works when the list is created by commands
other than *PUSH* (for example SORT STORE).
>> Cluster changes
* [FIX] CRITICAL: For a bug in the implementation of CLUSTER SET-CONFIG-EPOCH
introduced with beta-3 (especially beta-4 where the command
is actually used by redis-trib), a configEpoch for a node could
jump backward, breaking the eventual consistency property of the
slots -> nodes mapping in the cluster.
>> Sentinel changes
* No changes for Sentinel in this release.
--[ Redis 3.0.0 Beta 5 (version 2.9.54) ] Release date: 26 may 2014
This is the 5th beta of Redis 3.0.0. It does not include any real
worthwhile change (just three days passed since the previous beta), but
fixes two stupid bugs preventing cluster tests to pass.
--[ Redis 3.0.0 Beta 4 (version 2.9.53) ] Release date: 23 may 2014
This is the fourth beta of Redis 3.0.0.
>> General changes
* [NEW] Scripting engine performances improvements.
* [NEW] Log format slightly changed to report current node role.
* [FIX] Correct the HyperLogLog stale cache flag to prevent unnecessary
computation.
>> Cluster changes
* [NEW] redis-trib: ability to import data from standalone Redis instances.
* [NEW] redis-trib: "fix" subcommand much better at fixing errors.
* [NEW] CLUSTER FAILOVER FORCE implemented.
* [NEW] CLUSTER RESET implemented, it is now possible to completely reset
nodes and create a new cluster without restarting anything.
* [NEW] Slave validity factor (max estimated data age to still failover)
is now under the control of the user, and can be configured via
redis.conf or CONFIG SET. Option name cluster-slave-validity-factor.
* [NEW] Cluster test: failure detection and failover initial tests.
* [NEW] CLUSTER MEET: better error messages when address is invalid.
* [NEW] Bulk-accept new Cluster nodes in the Cluster bus instead of
performing just a single accept() per event fired.
* [FIX] Bypass data_age check for manual failovers.
* [FIX] Fixed data_age computation / check integer overflow.
* [FIX] Various fixes to Tcl client.tcl Redis Cluster client used in tests.
* [FIX] Better handling of stolen slots.
* [FIX] Don't accept cluster bus connections during startup.
>> Sentinel changes
* [NEW] Generate +config-update-from event when a new config is received.
* [NEW] Log when a failover will be re-attempted again.
* [FIX] Sentinel: Add "dir /tmp" directive in example sentinel.conf.
--[ Redis 3.0.0 Beta 3 (version 2.9.52) ] Release date: 5 may 2014
This is the third beta of Redis 3.0.0.
>> General changes
* [NEW] New data structure: the HyperLogLog (see 2.8 release notes).
* [NEW] Lexicograhical range queries in sorted sets (see 2.8 release notes).
* [NEW] LRU algorithm precision greatly improved.
* [FIX] Redis is now much more responsive to reply with LOADING / BUSY errors.
>> Cluster changes
* [NEW] Cluster testing framework and initial tests.
* [NEW] Cluster epochs collision resolution (make Redis Cluster more resilient
to admin and programming errors).
* [NEW] Persist / fsync some global state to ensure correct crash-recovery
semantics.
* [NEW] New command introduced: CLUSTER SET-CONFIG-EPOCH, still not used
by redis-trib. Will be used to speedup the assignment of unique
epochs to different nodes at cluster creation time. For now this is
handled as a side effect of the cluster epochs collision resolution.
* [FIX] Different fixes to redis-trib cluster creation.
* [FIX] Fix an error in the CLUSTER NODES output for nodes slots.
>> Sentinel changes
* [NEW] Sentinels are now able to send update messages in a peer-to-peer
fashion even if no Redis instances are available. Now the Sentinel
liveness property that the most updated configuration in a given
partition is propagated to all the Sentinels is extended to partitions
without reachable instances.
* [NEW] Sentinel safety properties are now ensured in a crash-recovery system
model since some state is persisted on disk before replying to other
nodes, and reloaded at startup.
* [NEW] Sentinel now uses CLIENT SETNAME so that it is easy to identify
Sentinels using CLIENT LIST among other clients.
* [NEW] Sentinel failure detection and reconnection code improved.
--[ Redis 3.0.0 Beta 2 (version 2.9.51) ] Release date: 11 mar 2014
This is the second beta of Redis 3.0.0.
>> General changes
* [FIX] Sometimes the absolute config file path was obtained in a wrong way.
This happened when there was a "dir" directive inside the config file
and at the same time the configuration file was given as a relative
path to redis-server or redis-sentinel executables.
* [FIX] redis-cli: Automatically enter --slave mode when SYNC or PSYNC are
called during an interactive session.
* [FIX] BITCOUNT: fixed unaligned access causing issues in sparc and other
archs not capable of dealing with unaligned accesses. This also makes
the code faster in archs where unaligned accesses are allowed.
* [FIX] Force INFO used_memory_peak to match peak memory. This generated some
confusion among users even if it was not an actual bug.
* [FIX] Fixed an critical EVALSHA script cache bug: scripts executed may not
propagate to AOF / Slaves correctly under certain conditions.
See issue #1549 at Github for more information.
* [FIX] Fixed multiple bugs resulting into closing the link with master or slave
during replication without good reasons. This will result in useless
resynchronizations, or infinite loops where the replication link can't
be established.
* [FIX] Don't count the time needed to populate the buffers of clients waiting
in MONITOR mode when populating the Slow Log entries.
* [NEW] Redis-cli updated to use SCAN instead of random sampling via
RANDOMKEY in order to implement --bigkeys feature. Moreover the
implementation now supports pipelining and reports more information
at the end of the scan. Much faster, much better. A special thank
you to Michael Grunder for this improvement.
* [NEW] redis-cli now supports a new --intrinsic-latency mode that is able
to meter the latency of a system due to kernel / hypervisor.
How to use it is explained at http://redis.io/topics/latency.
* [NEW] New command BITPOS: find first bit set or clear in a bitmap.
* [NEW] CONFIG REWRITE calls are now logged.
* [NEW] AOF write errors (like no space on device) no longer abort Redis if the
fsync policy is none or every second. The database enters a read-only
mode where every write is refused with an error. Normal operations are
restored as soon as Redis is able to append again data to the AOF file.
* [NEW] Sentinel now accepts SHUTDOWN command.
>> Cluster changes
* [FIX] Bind the first interface listed in the "bind" configuration directive
if any, in order to perform outgoing connections. This fixes Cluster
usage when an address is bound but there are multiple interfaces that
may be used to connect with other nodes.
* [FIX] When an "Importing" slot is closed via CLUSTER SETSLOT NODE ...
increment the configEpoch in the special case it is zero.
* [FIX] Current transaction is invalidated on redirection errors.
* [FIX] Abort if port does not allow for a valid cluster bus port that is
always at fixed +10000 offset.
* [FIX] Keys extraction algorithm fixed for ZUNIONSTORE/ZINTERSTORE and SORT.
* [FIX] Better failover timeout and retry times: failover should now work
reliabily when node-timeout is very small (a few milliseconds).
* [FIX] Don't allow SORT GET/BY options in Cluster mode.
* [FIX] Clear importing/migrating state when turning from master to slave role.
* [FIX] Set slot error if we receive an update for a busy slot.
* [FIX] Update node configEpoch on UPDATE messages.
* [NEW] Support multi-key operations as long as keys resolve to the same
hash slot, and the slot is not migrating, or it is migrating but all
the mentioned keys are available.
* [NEW] New DEBUG command CMDKEYS available to debug / test keys identification
in Redis commands.
* [NEW] redis-trib: create subcommand is now able to assign spare slaves.
* [NEW] redis-trib: new subcommand 'call'. Exec command in all nodes.
>> Sentinel changes
* [FIX] Sentinel "IDONTKNOW" error removed as it does not made sense with the
new Sentinel design. This error was actually a fix for a design error
in the first implementation of Sentinel.
* [FIX] Sentinel: added a missing exit() call to abort after config file
checks at startup. This error was introduced with an improvement in
a previous 2.8 release.
* [FIX] Sentinel: better nodes fail over start time desynchronization to avoid
split-brain during the voting process needed to get authorization to
fail over. This means the system is less likely to need to retry
and will fail over faster. No changes in behavior / correctness.
* [NEW] Sentinel unit tests and framework. More tests needed and units must
be improved in order to have less false positives, but it is a start
and features a debugging console that is useful to fix tests or to
inspect bugs causing tests failures.
* [NEW] New Sentinel events: +/-monitor and +set used to monitor when an
instance to monitor is added or removed, or when a configuration
is modified via SENTINEL SET.
--[ Redis 3.0.0 Beta 1 (version 2.9.50) ] Release date: 11 Feb 2014
This is the first beta of Redis 3.0.0.
Migrating from 2.8 to 3.0
=========================
Redis 2.8 is mostly a strict subset of 3.0, you should not have any problem
upgrading your application from 2.8 to 3.0. However this is a list of small
non-backward compatible changes introduced in the 3.0 release:
* The log format was modified. The prefix of each line included the pid
in the following format [1234]. Now instead it is 1234:? Where
'?' is actually the role of the instance. M for master, S for slave, C
if this process is a saving child (for RDB/AOF), and X for Sentinel.
* The default maxmemory policy in Redis 3.0 is no longer "volatile-lru" as
it used to be in 2.8, but "noeviction". The policies behavior is the same
(but LRU eviction is much more precise in 3.0), so only the default value
changed. Just make sure to specify in your redis.conf what you mean.
--------------------------------------------------------------------------------
Credits: Where not specified the implementation and design is done by
Salvatore Sanfilippo. Thanks to Pivotal for making all this possible.
Also many thanks to all the other contributors and the amazing community
we have.
See commit messages for more credits.
Cheers,
Salvatore
Happy hacking!
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (c) 2006-2015, Salvatore Sanfilippo
Copyright (c) 2006-2014, Salvatore Sanfilippo
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
-19
View File
@@ -26,25 +26,6 @@ After building Redis is a good idea to test it, using:
% make test
Fixing build problems with dependencies or cached build options
—--------
Redis has some dependencies which are included into the "deps" directory.
"make" does not rebuild dependencies automatically, even if something in the
source code of dependencies is changes.
When you update the source code with `git pull` or when code inside the
dependencies tree is modified in any other way, make sure to use the following
command in order to really clean everything and rebuild from scratch:
make distclean
This will clean: jemalloc, lua, hiredis, linenoise.
Also if you force certain build options like 32bit target, no C compiler
optimizations (for debugging purposes), and other similar build time options,
those options are cached indefinitely until you issue a "make distclean"
command.
Fixing problems building 32 bit binaries
---------
+1 -1
View File
@@ -58,7 +58,7 @@ ifeq ($(uname_S),SunOS)
LUA_CFLAGS= -D__C99FEATURES__=1
endif
LUA_CFLAGS+= -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL $(CFLAGS)
LUA_CFLAGS+= -O2 -Wall -DLUA_ANSI $(CFLAGS)
LUA_LDFLAGS+= $(LDFLAGS)
# lua's Makefile defines AR="ar rcu", which is unusual, and makes it more
# challenging to cross-compile lua (and redis). These defines make it easier
-1
View File
@@ -443,7 +443,6 @@ void redisProcessCallbacks(redisAsyncContext *ac) {
if (((redisReply*)reply)->type == REDIS_REPLY_ERROR) {
c->err = REDIS_ERR_OTHER;
snprintf(c->errstr,sizeof(c->errstr),"%s",((redisReply*)reply)->str);
c->reader->fn->freeObject(reply);
__redisAsyncDisconnect(ac);
return;
}
+2 -3
View File
@@ -25,10 +25,9 @@ PLATS= aix ansi bsd freebsd generic linux macosx mingw posix solaris
LUA_A= liblua.a
CORE_O= lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o \
lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o \
lundump.o lvm.o lzio.o strbuf.o fpconv.o
lundump.o lvm.o lzio.o strbuf.o
LIB_O= lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o \
lstrlib.o loadlib.o linit.o lua_cjson.o lua_struct.o lua_cmsgpack.o \
lua_bit.o
lstrlib.o loadlib.o linit.o lua_cjson.o lua_struct.o lua_cmsgpack.o
LUA_T= lua
LUA_O= lua.o
-205
View File
@@ -1,205 +0,0 @@
/* fpconv - Floating point conversion routines
*
* Copyright (c) 2011-2012 Mark Pulford <mark@kyne.com.au>
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
* "Software"), to deal in the Software without restriction, including
* without limitation the rights to use, copy, modify, merge, publish,
* distribute, sublicense, and/or sell copies of the Software, and to
* permit persons to whom the Software is furnished to do so, subject to
* the following conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
* IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
* CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
* TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
* SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
/* JSON uses a '.' decimal separator. strtod() / sprintf() under C libraries
* with locale support will break when the decimal separator is a comma.
*
* fpconv_* will around these issues with a translation buffer if required.
*/
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#include <string.h>
#include "fpconv.h"
/* Lua CJSON assumes the locale is the same for all threads within a
* process and doesn't change after initialisation.
*
* This avoids the need for per thread storage or expensive checks
* for call. */
static char locale_decimal_point = '.';
/* In theory multibyte decimal_points are possible, but
* Lua CJSON only supports UTF-8 and known locales only have
* single byte decimal points ([.,]).
*
* localconv() may not be thread safe (=>crash), and nl_langinfo() is
* not supported on some platforms. Use sprintf() instead - if the
* locale does change, at least Lua CJSON won't crash. */
static void fpconv_update_locale()
{
char buf[8];
snprintf(buf, sizeof(buf), "%g", 0.5);
/* Failing this test might imply the platform has a buggy dtoa
* implementation or wide characters */
if (buf[0] != '0' || buf[2] != '5' || buf[3] != 0) {
fprintf(stderr, "Error: wide characters found or printf() bug.");
abort();
}
locale_decimal_point = buf[1];
}
/* Check for a valid number character: [-+0-9a-yA-Y.]
* Eg: -0.6e+5, infinity, 0xF0.F0pF0
*
* Used to find the probable end of a number. It doesn't matter if
* invalid characters are counted - strtod() will find the valid
* number if it exists. The risk is that slightly more memory might
* be allocated before a parse error occurs. */
static inline int valid_number_character(char ch)
{
char lower_ch;
if ('0' <= ch && ch <= '9')
return 1;
if (ch == '-' || ch == '+' || ch == '.')
return 1;
/* Hex digits, exponent (e), base (p), "infinity",.. */
lower_ch = ch | 0x20;
if ('a' <= lower_ch && lower_ch <= 'y')
return 1;
return 0;
}
/* Calculate the size of the buffer required for a strtod locale
* conversion. */
static int strtod_buffer_size(const char *s)
{
const char *p = s;
while (valid_number_character(*p))
p++;
return p - s;
}
/* Similar to strtod(), but must be passed the current locale's decimal point
* character. Guaranteed to be called at the start of any valid number in a string */
double fpconv_strtod(const char *nptr, char **endptr)
{
char localbuf[FPCONV_G_FMT_BUFSIZE];
char *buf, *endbuf, *dp;
int buflen;
double value;
/* System strtod() is fine when decimal point is '.' */
if (locale_decimal_point == '.')
return strtod(nptr, endptr);
buflen = strtod_buffer_size(nptr);
if (!buflen) {
/* No valid characters found, standard strtod() return */
*endptr = (char *)nptr;
return 0;
}
/* Duplicate number into buffer */
if (buflen >= FPCONV_G_FMT_BUFSIZE) {
/* Handle unusually large numbers */
buf = malloc(buflen + 1);
if (!buf) {
fprintf(stderr, "Out of memory");
abort();
}
} else {
/* This is the common case.. */
buf = localbuf;
}
memcpy(buf, nptr, buflen);
buf[buflen] = 0;
/* Update decimal point character if found */
dp = strchr(buf, '.');
if (dp)
*dp = locale_decimal_point;
value = strtod(buf, &endbuf);
*endptr = (char *)&nptr[endbuf - buf];
if (buflen >= FPCONV_G_FMT_BUFSIZE)
free(buf);
return value;
}
/* "fmt" must point to a buffer of at least 6 characters */
static void set_number_format(char *fmt, int precision)
{
int d1, d2, i;
assert(1 <= precision && precision <= 14);
/* Create printf format (%.14g) from precision */
d1 = precision / 10;
d2 = precision % 10;
fmt[0] = '%';
fmt[1] = '.';
i = 2;
if (d1) {
fmt[i++] = '0' + d1;
}
fmt[i++] = '0' + d2;
fmt[i++] = 'g';
fmt[i] = 0;
}
/* Assumes there is always at least 32 characters available in the target buffer */
int fpconv_g_fmt(char *str, double num, int precision)
{
char buf[FPCONV_G_FMT_BUFSIZE];
char fmt[6];
int len;
char *b;
set_number_format(fmt, precision);
/* Pass through when decimal point character is dot. */
if (locale_decimal_point == '.')
return snprintf(str, FPCONV_G_FMT_BUFSIZE, fmt, num);
/* snprintf() to a buffer then translate for other decimal point characters */
len = snprintf(buf, FPCONV_G_FMT_BUFSIZE, fmt, num);
/* Copy into target location. Translate decimal point if required */
b = buf;
do {
*str++ = (*b == locale_decimal_point ? '.' : *b);
} while(*b++);
return len;
}
void fpconv_init()
{
fpconv_update_locale();
}
/* vi:ai et sw=4 ts=4:
*/
-22
View File
@@ -1,22 +0,0 @@
/* Lua CJSON floating point conversion routines */
/* Buffer required to store the largest string representation of a double.
*
* Longest double printed with %.14g is 21 characters long:
* -1.7976931348623e+308 */
# define FPCONV_G_FMT_BUFSIZE 32
#ifdef USE_INTERNAL_FPCONV
static inline void fpconv_init()
{
/* Do nothing - not required */
}
#else
extern void fpconv_init();
#endif
extern int fpconv_g_fmt(char*, double, int);
extern double fpconv_strtod(const char*, char**);
/* vi:ai et sw=4 ts=4:
*/
+1 -1
View File
@@ -495,7 +495,7 @@ static void f_parser (lua_State *L, void *ud) {
struct SParser *p = cast(struct SParser *, ud);
int c = luaZ_lookahead(p->z);
luaC_checkGC(L);
tf = (luaY_parser)(L, p->z,
tf = ((c == LUA_SIGNATURE[0]) ? luaU_undump : luaY_parser)(L, p->z,
&p->buff, p->name);
cl = luaF_newLclosure(L, tf->nups, hvalue(gt(L)));
cl->l.p = tf;
-189
View File
@@ -1,189 +0,0 @@
/*
** Lua BitOp -- a bit operations library for Lua 5.1/5.2.
** http://bitop.luajit.org/
**
** Copyright (C) 2008-2012 Mike Pall. All rights reserved.
**
** Permission is hereby granted, free of charge, to any person obtaining
** a copy of this software and associated documentation files (the
** "Software"), to deal in the Software without restriction, including
** without limitation the rights to use, copy, modify, merge, publish,
** distribute, sublicense, and/or sell copies of the Software, and to
** permit persons to whom the Software is furnished to do so, subject to
** the following conditions:
**
** The above copyright notice and this permission notice shall be
** included in all copies or substantial portions of the Software.
**
** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
** EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
** MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
** IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
** CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
** TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
** SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
**
** [ MIT license: http://www.opensource.org/licenses/mit-license.php ]
*/
#define LUA_BITOP_VERSION "1.0.2"
#define LUA_LIB
#include "lua.h"
#include "lauxlib.h"
#ifdef _MSC_VER
/* MSVC is stuck in the last century and doesn't have C99's stdint.h. */
typedef __int32 int32_t;
typedef unsigned __int32 uint32_t;
typedef unsigned __int64 uint64_t;
#else
#include <stdint.h>
#endif
typedef int32_t SBits;
typedef uint32_t UBits;
typedef union {
lua_Number n;
#ifdef LUA_NUMBER_DOUBLE
uint64_t b;
#else
UBits b;
#endif
} BitNum;
/* Convert argument to bit type. */
static UBits barg(lua_State *L, int idx)
{
BitNum bn;
UBits b;
#if LUA_VERSION_NUM < 502
bn.n = lua_tonumber(L, idx);
#else
bn.n = luaL_checknumber(L, idx);
#endif
#if defined(LUA_NUMBER_DOUBLE)
bn.n += 6755399441055744.0; /* 2^52+2^51 */
#ifdef SWAPPED_DOUBLE
b = (UBits)(bn.b >> 32);
#else
b = (UBits)bn.b;
#endif
#elif defined(LUA_NUMBER_INT) || defined(LUA_NUMBER_LONG) || \
defined(LUA_NUMBER_LONGLONG) || defined(LUA_NUMBER_LONG_LONG) || \
defined(LUA_NUMBER_LLONG)
if (sizeof(UBits) == sizeof(lua_Number))
b = bn.b;
else
b = (UBits)(SBits)bn.n;
#elif defined(LUA_NUMBER_FLOAT)
#error "A 'float' lua_Number type is incompatible with this library"
#else
#error "Unknown number type, check LUA_NUMBER_* in luaconf.h"
#endif
#if LUA_VERSION_NUM < 502
if (b == 0 && !lua_isnumber(L, idx)) {
luaL_typerror(L, idx, "number");
}
#endif
return b;
}
/* Return bit type. */
#define BRET(b) lua_pushnumber(L, (lua_Number)(SBits)(b)); return 1;
static int bit_tobit(lua_State *L) { BRET(barg(L, 1)) }
static int bit_bnot(lua_State *L) { BRET(~barg(L, 1)) }
#define BIT_OP(func, opr) \
static int func(lua_State *L) { int i; UBits b = barg(L, 1); \
for (i = lua_gettop(L); i > 1; i--) b opr barg(L, i); BRET(b) }
BIT_OP(bit_band, &=)
BIT_OP(bit_bor, |=)
BIT_OP(bit_bxor, ^=)
#define bshl(b, n) (b << n)
#define bshr(b, n) (b >> n)
#define bsar(b, n) ((SBits)b >> n)
#define brol(b, n) ((b << n) | (b >> (32-n)))
#define bror(b, n) ((b << (32-n)) | (b >> n))
#define BIT_SH(func, fn) \
static int func(lua_State *L) { \
UBits b = barg(L, 1); UBits n = barg(L, 2) & 31; BRET(fn(b, n)) }
BIT_SH(bit_lshift, bshl)
BIT_SH(bit_rshift, bshr)
BIT_SH(bit_arshift, bsar)
BIT_SH(bit_rol, brol)
BIT_SH(bit_ror, bror)
static int bit_bswap(lua_State *L)
{
UBits b = barg(L, 1);
b = (b >> 24) | ((b >> 8) & 0xff00) | ((b & 0xff00) << 8) | (b << 24);
BRET(b)
}
static int bit_tohex(lua_State *L)
{
UBits b = barg(L, 1);
SBits n = lua_isnone(L, 2) ? 8 : (SBits)barg(L, 2);
const char *hexdigits = "0123456789abcdef";
char buf[8];
int i;
if (n < 0) { n = -n; hexdigits = "0123456789ABCDEF"; }
if (n > 8) n = 8;
for (i = (int)n; --i >= 0; ) { buf[i] = hexdigits[b & 15]; b >>= 4; }
lua_pushlstring(L, buf, (size_t)n);
return 1;
}
static const struct luaL_Reg bit_funcs[] = {
{ "tobit", bit_tobit },
{ "bnot", bit_bnot },
{ "band", bit_band },
{ "bor", bit_bor },
{ "bxor", bit_bxor },
{ "lshift", bit_lshift },
{ "rshift", bit_rshift },
{ "arshift", bit_arshift },
{ "rol", bit_rol },
{ "ror", bit_ror },
{ "bswap", bit_bswap },
{ "tohex", bit_tohex },
{ NULL, NULL }
};
/* Signed right-shifts are implementation-defined per C89/C99.
** But the de facto standard are arithmetic right-shifts on two's
** complement CPUs. This behaviour is required here, so test for it.
*/
#define BAD_SAR (bsar(-8, 2) != (SBits)-2)
LUALIB_API int luaopen_bit(lua_State *L)
{
UBits b;
lua_pushnumber(L, (lua_Number)1437217655L);
b = barg(L, -1);
if (b != (UBits)1437217655L || BAD_SAR) { /* Perform a simple self-test. */
const char *msg = "compiled with incompatible luaconf.h";
#ifdef LUA_NUMBER_DOUBLE
#ifdef _WIN32
if (b == (UBits)1610612736L)
msg = "use D3DCREATE_FPU_PRESERVE with DirectX";
#endif
if (b == (UBits)1127743488L)
msg = "not compiled with SWAPPED_DOUBLE";
#endif
if (BAD_SAR)
msg = "arithmetic right-shift broken";
luaL_error(L, "bit library self-test failed (%s)", msg);
}
#if LUA_VERSION_NUM < 502
luaL_register(L, "bit", bit_funcs);
#else
luaL_newlib(L, bit_funcs);
#endif
return 1;
}
+310 -438
View File
File diff suppressed because it is too large Load Diff
+127 -368
View File
@@ -7,38 +7,14 @@
#include "lua.h"
#include "lauxlib.h"
#define LUACMSGPACK_NAME "cmsgpack"
#define LUACMSGPACK_SAFE_NAME "cmsgpack_safe"
#define LUACMSGPACK_VERSION "lua-cmsgpack 0.4.0"
#define LUACMSGPACK_VERSION "lua-cmsgpack 0.3.0"
#define LUACMSGPACK_COPYRIGHT "Copyright (C) 2012, Salvatore Sanfilippo"
#define LUACMSGPACK_DESCRIPTION "MessagePack C implementation for Lua"
/* Allows a preprocessor directive to override MAX_NESTING */
#ifndef LUACMSGPACK_MAX_NESTING
#define LUACMSGPACK_MAX_NESTING 16 /* Max tables nesting. */
#endif
#define LUACMSGPACK_MAX_NESTING 16 /* Max tables nesting. */
/* Check if float or double can be an integer without loss of precision */
#define IS_INT_TYPE_EQUIVALENT(x, T) (!isinf(x) && (T)(x) == (x))
#define IS_INT64_EQUIVALENT(x) IS_INT_TYPE_EQUIVALENT(x, int64_t)
#define IS_INT_EQUIVALENT(x) IS_INT_TYPE_EQUIVALENT(x, int)
/* If size of pointer is equal to a 4 byte integer, we're on 32 bits. */
#if UINTPTR_MAX == UINT_MAX
#define BITS_32 1
#else
#define BITS_32 0
#endif
#if BITS_32
#define lua_pushunsigned(L, n) lua_pushnumber(L, n)
#else
#define lua_pushunsigned(L, n) lua_pushinteger(L, n)
#endif
/* =============================================================================
* MessagePack implementation and bindings for Lua 5.1/5.2.
/* ==============================================================================
* MessagePack implementation and bindings for Lua 5.1.
* Copyright(C) 2012 Salvatore Sanfilippo <antirez@gmail.com>
*
* http://github.com/antirez/lua-cmsgpack
@@ -53,27 +29,23 @@
* 20-Feb-2012 (ver 0.2.0): Tables encoding improved.
* 20-Feb-2012 (ver 0.2.1): Minor bug fixing.
* 20-Feb-2012 (ver 0.3.0): Module renamed lua-cmsgpack (was lua-msgpack).
* 04-Apr-2014 (ver 0.3.1): Lua 5.2 support and minor bug fix.
* 07-Apr-2014 (ver 0.4.0): Multiple pack/unpack, lua allocator, efficiency.
* ========================================================================== */
* ============================================================================ */
/* -------------------------- Endian conversion --------------------------------
* We use it only for floats and doubles, all the other conversions performed
/* --------------------------- Endian conversion --------------------------------
* We use it only for floats and doubles, all the other conversions are performed
* in an endian independent fashion. So the only thing we need is a function
* that swaps a binary string if arch is little endian (and left it untouched
* that swaps a binary string if the arch is little endian (and left it untouched
* otherwise). */
/* Reverse memory bytes if arch is little endian. Given the conceptual
* simplicity of the Lua build system we prefer check for endianess at runtime.
* simplicity of the Lua build system we prefer to check for endianess at runtime.
* The performance difference should be acceptable. */
void memrevifle(void *ptr, size_t len) {
unsigned char *p = (unsigned char *)ptr,
*e = (unsigned char *)p+len-1,
aux;
static void memrevifle(void *ptr, size_t len) {
unsigned char *p = ptr, *e = p+len-1, aux;
int test = 1;
unsigned char *testp = (unsigned char*) &test;
if (testp[0] == 0) return; /* Big endian, nothing to do. */
if (testp[0] == 0) return; /* Big endian, nothign to do. */
len /= 2;
while(len--) {
aux = *p;
@@ -84,34 +56,20 @@ void memrevifle(void *ptr, size_t len) {
}
}
/* ---------------------------- String buffer ----------------------------------
* This is a simple implementation of string buffers. The only operation
/* ----------------------------- String buffer ----------------------------------
* This is a simple implementation of string buffers. The only opereation
* supported is creating empty buffers and appending bytes to it.
* The string buffer uses 2x preallocation on every realloc for O(N) append
* behavior. */
typedef struct mp_buf {
lua_State *L;
unsigned char *b;
size_t len, free;
} mp_buf;
void *mp_realloc(lua_State *L, void *target, size_t osize,size_t nsize) {
void *(*local_realloc) (void *, void *, size_t osize, size_t nsize) = NULL;
void *ud;
local_realloc = lua_getallocf(L, &ud);
return local_realloc(ud, target, osize, nsize);
}
mp_buf *mp_buf_new(lua_State *L) {
mp_buf *buf = NULL;
/* Old size = 0; new size = sizeof(*buf) */
buf = (mp_buf*)mp_realloc(L, NULL, 0, sizeof(*buf));
buf->L = L;
static mp_buf *mp_buf_new(void) {
mp_buf *buf = malloc(sizeof(*buf));
buf->b = NULL;
buf->len = buf->free = 0;
return buf;
@@ -121,7 +79,7 @@ void mp_buf_append(mp_buf *buf, const unsigned char *s, size_t len) {
if (buf->free < len) {
size_t newlen = buf->len+len;
buf->b = (unsigned char*)mp_realloc(buf->L, buf->b, buf->len, newlen*2);
buf->b = realloc(buf->b,newlen*2);
buf->free = newlen;
}
memcpy(buf->b+buf->len,s,len);
@@ -130,11 +88,11 @@ void mp_buf_append(mp_buf *buf, const unsigned char *s, size_t len) {
}
void mp_buf_free(mp_buf *buf) {
mp_realloc(buf->L, buf->b, buf->len, 0); /* realloc to 0 = free */
mp_realloc(buf->L, buf, sizeof(*buf), 0);
free(buf->b);
free(buf);
}
/* ---------------------------- String cursor ----------------------------------
/* ------------------------------ String cursor ----------------------------------
* This simple data structure is used for parsing. Basically you create a cursor
* using a string pointer and a length, then it is possible to access the
* current string position with cursor->p, check the remaining length
@@ -144,7 +102,7 @@ void mp_buf_free(mp_buf *buf) {
* be used to report errors. */
#define MP_CUR_ERROR_NONE 0
#define MP_CUR_ERROR_EOF 1 /* Not enough data to complete operation. */
#define MP_CUR_ERROR_EOF 1 /* Not enough data to complete the opereation. */
#define MP_CUR_ERROR_BADFMT 2 /* Bad data format */
typedef struct mp_cur {
@@ -153,15 +111,22 @@ typedef struct mp_cur {
int err;
} mp_cur;
void mp_cur_init(mp_cur *cursor, const unsigned char *s, size_t len) {
static mp_cur *mp_cur_new(const unsigned char *s, size_t len) {
mp_cur *cursor = malloc(sizeof(*cursor));
cursor->p = s;
cursor->left = len;
cursor->err = MP_CUR_ERROR_NONE;
return cursor;
}
static void mp_cur_free(mp_cur *cursor) {
free(cursor);
}
#define mp_cur_consume(_c,_len) do { _c->p += _len; _c->left -= _len; } while(0)
/* When there is not enough room we set an error in the cursor and return. This
/* When there is not enough room we set an error in the cursor and return, this
* is very common across the code so we have a macro to make the code look
* a bit simpler. */
#define mp_cur_need(_c,_len) do { \
@@ -171,19 +136,15 @@ void mp_cur_init(mp_cur *cursor, const unsigned char *s, size_t len) {
} \
} while(0)
/* ------------------------- Low level MP encoding -------------------------- */
/* --------------------------- Low level MP encoding -------------------------- */
void mp_encode_bytes(mp_buf *buf, const unsigned char *s, size_t len) {
static void mp_encode_bytes(mp_buf *buf, const unsigned char *s, size_t len) {
unsigned char hdr[5];
int hdrlen;
if (len < 32) {
hdr[0] = 0xa0 | (len&0xff); /* fix raw */
hdrlen = 1;
} else if (len <= 0xff) {
hdr[0] = 0xd9;
hdr[1] = len;
hdrlen = 2;
} else if (len <= 0xffff) {
hdr[0] = 0xda;
hdr[1] = (len&0xff00)>>8;
@@ -202,7 +163,7 @@ void mp_encode_bytes(mp_buf *buf, const unsigned char *s, size_t len) {
}
/* we assume IEEE 754 internal format for single and double precision floats. */
void mp_encode_double(mp_buf *buf, double d) {
static void mp_encode_double(mp_buf *buf, double d) {
unsigned char b[9];
float f = d;
@@ -220,7 +181,7 @@ void mp_encode_double(mp_buf *buf, double d) {
}
}
void mp_encode_int(mp_buf *buf, int64_t n) {
static void mp_encode_int(mp_buf *buf, int64_t n) {
unsigned char b[9];
int enclen;
@@ -258,7 +219,7 @@ void mp_encode_int(mp_buf *buf, int64_t n) {
}
} else {
if (n >= -32) {
b[0] = ((signed char)n); /* negative fixnum */
b[0] = ((char)n); /* negative fixnum */
enclen = 1;
} else if (n >= -128) {
b[0] = 0xd0; /* int 8 */
@@ -292,7 +253,7 @@ void mp_encode_int(mp_buf *buf, int64_t n) {
mp_buf_append(buf,b,enclen);
}
void mp_encode_array(mp_buf *buf, int64_t n) {
static void mp_encode_array(mp_buf *buf, int64_t n) {
unsigned char b[5];
int enclen;
@@ -315,7 +276,7 @@ void mp_encode_array(mp_buf *buf, int64_t n) {
mp_buf_append(buf,b,enclen);
}
void mp_encode_map(mp_buf *buf, int64_t n) {
static void mp_encode_map(mp_buf *buf, int64_t n) {
unsigned char b[5];
int enclen;
@@ -338,9 +299,9 @@ void mp_encode_map(mp_buf *buf, int64_t n) {
mp_buf_append(buf,b,enclen);
}
/* --------------------------- Lua types encoding --------------------------- */
/* ----------------------------- Lua types encoding --------------------------- */
void mp_encode_lua_string(lua_State *L, mp_buf *buf) {
static void mp_encode_lua_string(lua_State *L, mp_buf *buf) {
size_t len;
const char *s;
@@ -348,43 +309,26 @@ void mp_encode_lua_string(lua_State *L, mp_buf *buf) {
mp_encode_bytes(buf,(const unsigned char*)s,len);
}
void mp_encode_lua_bool(lua_State *L, mp_buf *buf) {
static void mp_encode_lua_bool(lua_State *L, mp_buf *buf) {
unsigned char b = lua_toboolean(L,-1) ? 0xc3 : 0xc2;
mp_buf_append(buf,&b,1);
}
/* Lua 5.3 has a built in 64-bit integer type */
void mp_encode_lua_integer(lua_State *L, mp_buf *buf) {
#if (LUA_VERSION_NUM < 503) && BITS_32
lua_Number i = lua_tonumber(L,-1);
#else
lua_Integer i = lua_tointeger(L,-1);
#endif
mp_encode_int(buf, (int64_t)i);
}
/* Lua 5.2 and lower only has 64-bit doubles, so we need to
* detect if the double may be representable as an int
* for Lua < 5.3 */
void mp_encode_lua_number(lua_State *L, mp_buf *buf) {
static void mp_encode_lua_number(lua_State *L, mp_buf *buf) {
lua_Number n = lua_tonumber(L,-1);
if (IS_INT64_EQUIVALENT(n)) {
mp_encode_lua_integer(L, buf);
} else {
if (floor(n) != n) {
mp_encode_double(buf,(double)n);
} else {
mp_encode_int(buf,(int64_t)n);
}
}
void mp_encode_lua_type(lua_State *L, mp_buf *buf, int level);
static void mp_encode_lua_type(lua_State *L, mp_buf *buf, int level);
/* Convert a lua table into a message pack list. */
void mp_encode_lua_table_as_array(lua_State *L, mp_buf *buf, int level) {
#if LUA_VERSION_NUM < 502
static void mp_encode_lua_table_as_array(lua_State *L, mp_buf *buf, int level) {
size_t len = lua_objlen(L,-1), j;
#else
size_t len = lua_rawlen(L,-1), j;
#endif
mp_encode_array(buf,len);
for (j = 1; j <= len; j++) {
@@ -395,13 +339,13 @@ void mp_encode_lua_table_as_array(lua_State *L, mp_buf *buf, int level) {
}
/* Convert a lua table into a message pack key-value map. */
void mp_encode_lua_table_as_map(lua_State *L, mp_buf *buf, int level) {
static void mp_encode_lua_table_as_map(lua_State *L, mp_buf *buf, int level) {
size_t len = 0;
/* First step: count keys into table. No other way to do it with the
* Lua API, we need to iterate a first time. Note that an alternative
* would be to do a single run, and then hack the buffer to insert the
* map opcodes for message pack. Too hackish for this lib. */
* map opcodes for message pack. Too hachish for this lib. */
lua_pushnil(L);
while(lua_next(L,-2)) {
lua_pop(L,1); /* remove value, keep key for next iteration. */
@@ -422,131 +366,80 @@ void mp_encode_lua_table_as_map(lua_State *L, mp_buf *buf, int level) {
/* Returns true if the Lua table on top of the stack is exclusively composed
* of keys from numerical keys from 1 up to N, with N being the total number
* of elements, without any hole in the middle. */
int table_is_an_array(lua_State *L) {
int count = 0, max = 0;
#if LUA_VERSION_NUM < 503
static int table_is_an_array(lua_State *L) {
long count = 0, max = 0, idx = 0;
lua_Number n;
#else
lua_Integer n;
#endif
/* Stack top on function entry */
int stacktop;
stacktop = lua_gettop(L);
lua_pushnil(L);
while(lua_next(L,-2)) {
/* Stack: ... key value */
lua_pop(L,1); /* Stack: ... key */
/* The <= 0 check is valid here because we're comparing indexes. */
#if LUA_VERSION_NUM < 503
if ((LUA_TNUMBER != lua_type(L,-1)) || (n = lua_tonumber(L, -1)) <= 0 ||
!IS_INT_EQUIVALENT(n))
#else
if (!lua_isinteger(L,-1) || (n = lua_tointeger(L, -1)) <= 0)
#endif
{
lua_settop(L, stacktop);
return 0;
}
max = (n > max ? n : max);
if (lua_type(L,-1) != LUA_TNUMBER) goto not_array;
n = lua_tonumber(L,-1);
idx = n;
if (idx != n || idx < 1) goto not_array;
count++;
max = idx;
}
/* We have the total number of elements in "count". Also we have
* the max index encountered in "max". We can't reach this code
* the max index encountered in "idx". We can't reach this code
* if there are indexes <= 0. If you also note that there can not be
* repeated keys into a table, you have that if max==count you are sure
* repeated keys into a table, you have that if idx==count you are sure
* that there are all the keys form 1 to count (both included). */
lua_settop(L, stacktop);
return max == count;
return idx == count;
not_array:
lua_pop(L,1);
return 0;
}
/* If the length operator returns non-zero, that is, there is at least
* an object at key '1', we serialize to message pack list. Otherwise
* we use a map. */
void mp_encode_lua_table(lua_State *L, mp_buf *buf, int level) {
static void mp_encode_lua_table(lua_State *L, mp_buf *buf, int level) {
if (table_is_an_array(L))
mp_encode_lua_table_as_array(L,buf,level);
else
mp_encode_lua_table_as_map(L,buf,level);
}
void mp_encode_lua_null(lua_State *L, mp_buf *buf) {
static void mp_encode_lua_null(lua_State *L, mp_buf *buf) {
unsigned char b[1];
(void)L;
b[0] = 0xc0;
mp_buf_append(buf,b,1);
}
void mp_encode_lua_type(lua_State *L, mp_buf *buf, int level) {
static void mp_encode_lua_type(lua_State *L, mp_buf *buf, int level) {
int t = lua_type(L,-1);
/* Limit the encoding of nested tables to a specified maximum depth, so that
/* Limit the encoding of nested tables to a specfiied maximum depth, so that
* we survive when called against circular references in tables. */
if (t == LUA_TTABLE && level == LUACMSGPACK_MAX_NESTING) t = LUA_TNIL;
switch(t) {
case LUA_TSTRING: mp_encode_lua_string(L,buf); break;
case LUA_TBOOLEAN: mp_encode_lua_bool(L,buf); break;
case LUA_TNUMBER:
#if LUA_VERSION_NUM < 503
mp_encode_lua_number(L,buf); break;
#else
if (lua_isinteger(L, -1)) {
mp_encode_lua_integer(L, buf);
} else {
mp_encode_lua_number(L, buf);
}
break;
#endif
case LUA_TNUMBER: mp_encode_lua_number(L,buf); break;
case LUA_TTABLE: mp_encode_lua_table(L,buf,level); break;
default: mp_encode_lua_null(L,buf); break;
}
lua_pop(L,1);
}
/*
* Packs all arguments as a stream for multiple upacking later.
* Returns error if no arguments provided.
*/
int mp_pack(lua_State *L) {
int nargs = lua_gettop(L);
int i;
mp_buf *buf;
static int mp_pack(lua_State *L) {
mp_buf *buf = mp_buf_new();
if (nargs == 0)
return luaL_argerror(L, 0, "MessagePack pack needs input.");
buf = mp_buf_new(L);
for(i = 1; i <= nargs; i++) {
/* Copy argument i to top of stack for _encode processing;
* the encode function pops it from the stack when complete. */
lua_pushvalue(L, i);
mp_encode_lua_type(L,buf,0);
lua_pushlstring(L,(char*)buf->b,buf->len);
/* Reuse the buffer for the next operation by
* setting its free count to the total buffer size
* and the current position to zero. */
buf->free += buf->len;
buf->len = 0;
}
mp_encode_lua_type(L,buf,0);
lua_pushlstring(L,(char*)buf->b,buf->len);
mp_buf_free(buf);
/* Concatenate all nargs buffers together */
lua_concat(L, nargs);
return 1;
}
/* ------------------------------- Decoding --------------------------------- */
/* --------------------------------- Decoding --------------------------------- */
void mp_decode_to_lua_type(lua_State *L, mp_cur *c);
void mp_decode_to_lua_array(lua_State *L, mp_cur *c, size_t len) {
assert(len <= UINT_MAX);
int index = 1;
lua_newtable(L);
@@ -559,7 +452,6 @@ void mp_decode_to_lua_array(lua_State *L, mp_cur *c, size_t len) {
}
void mp_decode_to_lua_hash(lua_State *L, mp_cur *c, size_t len) {
assert(len <= UINT_MAX);
lua_newtable(L);
while(len--) {
mp_decode_to_lua_type(L,c); /* key */
@@ -574,44 +466,34 @@ void mp_decode_to_lua_hash(lua_State *L, mp_cur *c, size_t len) {
* a Lua type, that is left as the only result on the stack. */
void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
mp_cur_need(c,1);
/* If we return more than 18 elements, we must resize the stack to
* fit all our return values. But, there is no way to
* determine how many objects a msgpack will unpack to up front, so
* we request a +1 larger stack on each iteration (noop if stack is
* big enough, and when stack does require resize it doubles in size) */
luaL_checkstack(L, 1,
"too many return values at once; "
"use unpack_one or unpack_limit instead.");
switch(c->p[0]) {
case 0xcc: /* uint 8 */
mp_cur_need(c,2);
lua_pushunsigned(L,c->p[1]);
lua_pushnumber(L,c->p[1]);
mp_cur_consume(c,2);
break;
case 0xd0: /* int 8 */
mp_cur_need(c,2);
lua_pushinteger(L,(signed char)c->p[1]);
lua_pushnumber(L,(char)c->p[1]);
mp_cur_consume(c,2);
break;
case 0xcd: /* uint 16 */
mp_cur_need(c,3);
lua_pushunsigned(L,
lua_pushnumber(L,
(c->p[1] << 8) |
c->p[2]);
mp_cur_consume(c,3);
break;
case 0xd1: /* int 16 */
mp_cur_need(c,3);
lua_pushinteger(L,(int16_t)
lua_pushnumber(L,(int16_t)
(c->p[1] << 8) |
c->p[2]);
mp_cur_consume(c,3);
break;
case 0xce: /* uint 32 */
mp_cur_need(c,5);
lua_pushunsigned(L,
lua_pushnumber(L,
((uint32_t)c->p[1] << 24) |
((uint32_t)c->p[2] << 16) |
((uint32_t)c->p[3] << 8) |
@@ -620,7 +502,7 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
break;
case 0xd2: /* int 32 */
mp_cur_need(c,5);
lua_pushinteger(L,
lua_pushnumber(L,
((int32_t)c->p[1] << 24) |
((int32_t)c->p[2] << 16) |
((int32_t)c->p[3] << 8) |
@@ -629,7 +511,7 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
break;
case 0xcf: /* uint 64 */
mp_cur_need(c,9);
lua_pushunsigned(L,
lua_pushnumber(L,
((uint64_t)c->p[1] << 56) |
((uint64_t)c->p[2] << 48) |
((uint64_t)c->p[3] << 40) |
@@ -642,11 +524,7 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
break;
case 0xd3: /* int 64 */
mp_cur_need(c,9);
#if LUA_VERSION_NUM < 503
lua_pushnumber(L,
#else
lua_pushinteger(L,
#endif
((int64_t)c->p[1] << 56) |
((int64_t)c->p[2] << 48) |
((int64_t)c->p[3] << 40) |
@@ -691,15 +569,6 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
mp_cur_consume(c,9);
}
break;
case 0xd9: /* raw 8 */
mp_cur_need(c,2);
{
size_t l = c->p[1];
mp_cur_need(c,2+l);
lua_pushlstring(L,(char*)c->p+2,l);
mp_cur_consume(c,2+l);
}
break;
case 0xda: /* raw 16 */
mp_cur_need(c,3);
{
@@ -712,14 +581,13 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
case 0xdb: /* raw 32 */
mp_cur_need(c,5);
{
size_t l = ((size_t)c->p[1] << 24) |
((size_t)c->p[2] << 16) |
((size_t)c->p[3] << 8) |
(size_t)c->p[4];
mp_cur_consume(c,5);
mp_cur_need(c,l);
lua_pushlstring(L,(char*)c->p,l);
mp_cur_consume(c,l);
size_t l = (c->p[1] << 24) |
(c->p[2] << 16) |
(c->p[3] << 8) |
c->p[4];
mp_cur_need(c,5+l);
lua_pushlstring(L,(char*)c->p+5,l);
mp_cur_consume(c,5+l);
}
break;
case 0xdc: /* array 16 */
@@ -733,10 +601,10 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
case 0xdd: /* array 32 */
mp_cur_need(c,5);
{
size_t l = ((size_t)c->p[1] << 24) |
((size_t)c->p[2] << 16) |
((size_t)c->p[3] << 8) |
(size_t)c->p[4];
size_t l = (c->p[1] << 24) |
(c->p[2] << 16) |
(c->p[3] << 8) |
c->p[4];
mp_cur_consume(c,5);
mp_decode_to_lua_array(L,c,l);
}
@@ -752,20 +620,20 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
case 0xdf: /* map 32 */
mp_cur_need(c,5);
{
size_t l = ((size_t)c->p[1] << 24) |
((size_t)c->p[2] << 16) |
((size_t)c->p[3] << 8) |
(size_t)c->p[4];
size_t l = (c->p[1] << 24) |
(c->p[2] << 16) |
(c->p[3] << 8) |
c->p[4];
mp_cur_consume(c,5);
mp_decode_to_lua_hash(L,c,l);
}
break;
default: /* types that can't be idenitified by first byte value. */
if ((c->p[0] & 0x80) == 0) { /* positive fixnum */
lua_pushunsigned(L,c->p[0]);
lua_pushnumber(L,c->p[0]);
mp_cur_consume(c,1);
} else if ((c->p[0] & 0xe0) == 0xe0) { /* negative fixnum */
lua_pushinteger(L,(signed char)c->p[0]);
lua_pushnumber(L,(signed char)c->p[0]);
mp_cur_consume(c,1);
} else if ((c->p[0] & 0xe0) == 0xa0) { /* fix raw */
size_t l = c->p[0] & 0x1f;
@@ -786,163 +654,54 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
}
}
int mp_unpack_full(lua_State *L, int limit, int offset) {
static int mp_unpack(lua_State *L) {
size_t len;
const char *s;
mp_cur c;
int cnt; /* Number of objects unpacked */
int decode_all = (!limit && !offset);
const unsigned char *s;
mp_cur *c;
s = luaL_checklstring(L,1,&len); /* if no match, exits */
if (offset < 0 || limit < 0) /* requesting negative off or lim is invalid */
return luaL_error(L,
"Invalid request to unpack with offset of %d and limit of %d.",
offset, len);
else if (offset > len)
return luaL_error(L,
"Start offset %d greater than input length %d.", offset, len);
if (decode_all) limit = INT_MAX;
mp_cur_init(&c,(const unsigned char *)s+offset,len-offset);
/* We loop over the decode because this could be a stream
* of multiple top-level values serialized together */
for(cnt = 0; c.left > 0 && cnt < limit; cnt++) {
mp_decode_to_lua_type(L,&c);
if (c.err == MP_CUR_ERROR_EOF) {
return luaL_error(L,"Missing bytes in input.");
} else if (c.err == MP_CUR_ERROR_BADFMT) {
return luaL_error(L,"Bad data format in input.");
}
if (!lua_isstring(L,-1)) {
lua_pushstring(L,"MessagePack decoding needs a string as input.");
lua_error(L);
}
if (!decode_all) {
/* c->left is the remaining size of the input buffer.
* subtract the entire buffer size from the unprocessed size
* to get our next start offset */
int offset = len - c.left;
/* Return offset -1 when we have have processed the entire buffer. */
lua_pushinteger(L, c.left == 0 ? -1 : offset);
/* Results are returned with the arg elements still
* in place. Lua takes care of only returning
* elements above the args for us.
* In this case, we have one arg on the stack
* for this function, so we insert our first return
* value at position 2. */
lua_insert(L, 2);
cnt += 1; /* increase return count by one to make room for offset */
s = (const unsigned char*) lua_tolstring(L,-1,&len);
c = mp_cur_new(s,len);
mp_decode_to_lua_type(L,c);
if (c->err == MP_CUR_ERROR_EOF) {
mp_cur_free(c);
lua_pushstring(L,"Missing bytes in input.");
lua_error(L);
} else if (c->err == MP_CUR_ERROR_BADFMT) {
mp_cur_free(c);
lua_pushstring(L,"Bad data format in input.");
lua_error(L);
} else if (c->left != 0) {
mp_cur_free(c);
lua_pushstring(L,"Extra bytes in input.");
lua_error(L);
}
return cnt;
mp_cur_free(c);
return 1;
}
int mp_unpack(lua_State *L) {
return mp_unpack_full(L, 0, 0);
}
/* ---------------------------------------------------------------------------- */
int mp_unpack_one(lua_State *L) {
int offset = luaL_optinteger(L, 2, 0);
/* Variable pop because offset may not exist */
lua_pop(L, lua_gettop(L)-1);
return mp_unpack_full(L, 1, offset);
}
int mp_unpack_limit(lua_State *L) {
int limit = luaL_checkinteger(L, 2);
int offset = luaL_optinteger(L, 3, 0);
/* Variable pop because offset may not exist */
lua_pop(L, lua_gettop(L)-1);
return mp_unpack_full(L, limit, offset);
}
int mp_safe(lua_State *L) {
int argc, err, total_results;
argc = lua_gettop(L);
/* This adds our function to the bottom of the stack
* (the "call this function" position) */
lua_pushvalue(L, lua_upvalueindex(1));
lua_insert(L, 1);
err = lua_pcall(L, argc, LUA_MULTRET, 0);
total_results = lua_gettop(L);
if (!err) {
return total_results;
} else {
lua_pushnil(L);
lua_insert(L,-2);
return 2;
}
}
/* -------------------------------------------------------------------------- */
const struct luaL_Reg cmds[] = {
static const struct luaL_reg thislib[] = {
{"pack", mp_pack},
{"unpack", mp_unpack},
{"unpack_one", mp_unpack_one},
{"unpack_limit", mp_unpack_limit},
{0}
{NULL, NULL}
};
int luaopen_create(lua_State *L) {
int i;
/* Manually construct our module table instead of
* relying on _register or _newlib */
lua_newtable(L);
LUALIB_API int luaopen_cmsgpack (lua_State *L) {
luaL_register(L, "cmsgpack", thislib);
for (i = 0; i < (sizeof(cmds)/sizeof(*cmds) - 1); i++) {
lua_pushcfunction(L, cmds[i].func);
lua_setfield(L, -2, cmds[i].name);
}
/* Add metadata */
lua_pushliteral(L, LUACMSGPACK_NAME);
lua_setfield(L, -2, "_NAME");
lua_pushliteral(L, LUACMSGPACK_VERSION);
lua_setfield(L, -2, "_VERSION");
lua_pushliteral(L, LUACMSGPACK_COPYRIGHT);
lua_setfield(L, -2, "_COPYRIGHT");
lua_pushliteral(L, LUACMSGPACK_DESCRIPTION);
lua_setfield(L, -2, "_DESCRIPTION");
return 1;
}
LUALIB_API int luaopen_cmsgpack(lua_State *L) {
luaopen_create(L);
#if LUA_VERSION_NUM < 502
/* Register name globally for 5.1 */
lua_pushvalue(L, -1);
lua_setglobal(L, LUACMSGPACK_NAME);
#endif
return 1;
}
LUALIB_API int luaopen_cmsgpack_safe(lua_State *L) {
int i;
luaopen_cmsgpack(L);
/* Wrap all functions in the safe handler */
for (i = 0; i < (sizeof(cmds)/sizeof(*cmds) - 1); i++) {
lua_getfield(L, -1, cmds[i].name);
lua_pushcclosure(L, mp_safe, 1);
lua_setfield(L, -2, cmds[i].name);
}
#if LUA_VERSION_NUM < 502
/* Register name globally for 5.1 */
lua_pushvalue(L, -1);
lua_setglobal(L, LUACMSGPACK_SAFE_NAME);
#endif
lua_setfield(L, -2, "_DESCRIPTION");
return 1;
}
+3 -3
View File
@@ -1,6 +1,6 @@
/* strbuf - String buffer routines
/* strbuf - string buffer routines
*
* Copyright (c) 2010-2012 Mark Pulford <mark@kyne.com.au>
* Copyright (c) 2010-2011 Mark Pulford <mark@kyne.com.au>
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
@@ -29,7 +29,7 @@
#include "strbuf.h"
static void die(const char *fmt, ...)
void die(const char *fmt, ...)
{
va_list arg;
+2 -14
View File
@@ -1,6 +1,6 @@
/* strbuf - String buffer routines
*
* Copyright (c) 2010-2012 Mark Pulford <mark@kyne.com.au>
* Copyright (c) 2010-2011 Mark Pulford <mark@kyne.com.au>
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
@@ -62,9 +62,7 @@ extern void strbuf_resize(strbuf_t *s, int len);
static int strbuf_empty_length(strbuf_t *s);
static int strbuf_length(strbuf_t *s);
static char *strbuf_string(strbuf_t *s, int *len);
static void strbuf_ensure_empty_length(strbuf_t *s, int len);
static char *strbuf_empty_ptr(strbuf_t *s);
static void strbuf_extend_length(strbuf_t *s, int len);
static void strbuf_ensure_empty_length(strbuf_t *s, int len);
/* Update */
extern void strbuf_append_fmt(strbuf_t *s, int len, const char *fmt, ...);
@@ -98,16 +96,6 @@ static inline void strbuf_ensure_empty_length(strbuf_t *s, int len)
strbuf_resize(s, s->length + len);
}
static inline char *strbuf_empty_ptr(strbuf_t *s)
{
return s->buf + s->length;
}
static inline void strbuf_extend_length(strbuf_t *s, int len)
{
s->length += len;
}
static inline int strbuf_length(strbuf_t *s)
{
return s->length;
+3 -13
View File
@@ -242,10 +242,6 @@ slave-read-only yes
# Replication SYNC strategy: disk or socket.
#
# -------------------------------------------------------
# WARNING: DISKLESS REPLICATION IS EXPERIMENTAL CURRENTLY
# -------------------------------------------------------
#
# New slaves and reconnecting slaves that are not able to continue the replication
# process just receiving differences, need to do what is called a "full
# synchronization". An RDB file is transmitted from the master to the slaves.
@@ -272,7 +268,7 @@ slave-read-only yes
repl-diskless-sync no
# When diskless replication is enabled, it is possible to configure the delay
# the server waits in order to spawn the child that transfers the RDB via socket
# the server waits in order to spawn the child that trnasfers the RDB via socket
# to the slaves.
#
# This is important since once the transfer starts, it is not possible to serve
@@ -619,12 +615,6 @@ lua-time-limit 5000
################################ REDIS CLUSTER ###############################
#
# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# WARNING EXPERIMENTAL: Redis Cluster is considered to be stable code, however
# in order to mark it as "mature" we need to wait for a non trivial percentage
# of users to deploy it in production.
# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#
# Normal Redis instances can't be part of a Redis Cluster; only nodes that are
# started as cluster nodes can. In order to start a Redis instance as a
# cluster node enable the cluster support uncommenting the following:
@@ -766,11 +756,11 @@ slowlog-max-len 128
# By default latency monitoring is disabled since it is mostly not needed
# if you don't have latency issues, and collecting data has a performance
# impact, that while very small, can be measured under big load. Latency
# monitoring can easily be enabled at runtime using the command
# monitoring can easily be enalbed at runtime using the command
# "CONFIG SET latency-monitor-threshold <milliseconds>" if needed.
latency-monitor-threshold 0
############################# EVENT NOTIFICATION ##############################
############################# Event notification ##############################
# Redis can notify Pub/Sub clients about events happening in the key space.
# This feature is documented at http://redis.io/topics/notifications
+1 -2
View File
@@ -58,7 +58,7 @@ ifeq ($(uname_S),SunOS)
# SunOS
INSTALL=cp -pf
FINAL_CFLAGS+= -D__EXTENSIONS__ -D_XPG6
FINAL_LIBS+= -ldl -lnsl -lsocket -lresolv -lpthread -lrt
FINAL_LIBS+= -ldl -lnsl -lsocket -lresolv -lpthread
else
ifeq ($(uname_S),Darwin)
# Darwin (nothing to do)
@@ -251,4 +251,3 @@ install: all
$(REDIS_INSTALL) $(REDIS_CLI_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_CHECK_DUMP_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_CHECK_AOF_NAME) $(INSTALL_BIN)
@ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_SENTINEL_NAME)
+5 -23
View File
@@ -264,7 +264,6 @@ static int anetCreateSocket(char *err, int domain) {
#define ANET_CONNECT_NONE 0
#define ANET_CONNECT_NONBLOCK 1
#define ANET_CONNECT_BE_BINDING 2 /* Best effort binding. */
static int anetTcpGenericConnect(char *err, char *addr, int port,
char *source_addr, int flags)
{
@@ -296,7 +295,7 @@ static int anetTcpGenericConnect(char *err, char *addr, int port,
if ((rv = getaddrinfo(source_addr, NULL, &hints, &bservinfo)) != 0)
{
anetSetError(err, "%s", gai_strerror(rv));
goto error;
goto end;
}
for (b = bservinfo; b != NULL; b = b->ai_next) {
if (bind(s,b->ai_addr,b->ai_addrlen) != -1) {
@@ -307,7 +306,7 @@ static int anetTcpGenericConnect(char *err, char *addr, int port,
freeaddrinfo(bservinfo);
if (!bound) {
anetSetError(err, "bind: %s", strerror(errno));
goto error;
goto end;
}
}
if (connect(s,p->ai_addr,p->ai_addrlen) == -1) {
@@ -332,17 +331,9 @@ error:
close(s);
s = ANET_ERR;
}
end:
freeaddrinfo(servinfo);
/* Handle best effort binding: if a binding address was used, but it is
* not possible to create a socket, try again without a binding address. */
if (s == ANET_ERR && source_addr && (flags & ANET_CONNECT_BE_BINDING)) {
return anetTcpGenericConnect(err,addr,port,NULL,flags);
} else {
return s;
}
return s;
}
int anetTcpConnect(char *err, char *addr, int port)
@@ -355,18 +346,9 @@ int anetTcpNonBlockConnect(char *err, char *addr, int port)
return anetTcpGenericConnect(err,addr,port,NULL,ANET_CONNECT_NONBLOCK);
}
int anetTcpNonBlockBindConnect(char *err, char *addr, int port,
char *source_addr)
int anetTcpNonBlockBindConnect(char *err, char *addr, int port, char *source_addr)
{
return anetTcpGenericConnect(err,addr,port,source_addr,
ANET_CONNECT_NONBLOCK);
}
int anetTcpNonBlockBestEffortBindConnect(char *err, char *addr, int port,
char *source_addr)
{
return anetTcpGenericConnect(err,addr,port,source_addr,
ANET_CONNECT_NONBLOCK|ANET_CONNECT_BE_BINDING);
return anetTcpGenericConnect(err,addr,port,source_addr,ANET_CONNECT_NONBLOCK);
}
int anetUnixGenericConnect(char *err, char *path, int flags)
-1
View File
@@ -50,7 +50,6 @@
int anetTcpConnect(char *err, char *addr, int port);
int anetTcpNonBlockConnect(char *err, char *addr, int port);
int anetTcpNonBlockBindConnect(char *err, char *addr, int port, char *source_addr);
int anetTcpNonBlockBestEffortBindConnect(char *err, char *addr, int port, char *source_addr);
int anetUnixConnect(char *err, char *path);
int anetUnixNonBlockConnect(char *err, char *path);
int anetRead(int fd, char *buf, int count);
+1 -2
View File
@@ -1105,7 +1105,6 @@ int rewriteAppendOnlyFile(char *filename) {
}
}
dictReleaseIterator(di);
di = NULL;
}
/* Do an initial slow fsync here while the parent is still sending
@@ -1169,9 +1168,9 @@ int rewriteAppendOnlyFile(char *filename) {
return REDIS_OK;
werr:
redisLog(REDIS_WARNING,"Write error writing append only file on disk: %s", strerror(errno));
fclose(fp);
unlink(tmpfile);
redisLog(REDIS_WARNING,"Write error writing append only file on disk: %s", strerror(errno));
if (di) dictReleaseIterator(di);
return REDIS_ERR;
}
+9 -21
View File
@@ -70,19 +70,16 @@ size_t redisPopcount(void *s, long count) {
count--;
}
/* Count bits 28 bytes at a time */
/* Count bits 16 bytes at a time */
p4 = (uint32_t*)p;
while(count>=28) {
uint32_t aux1, aux2, aux3, aux4, aux5, aux6, aux7;
while(count>=16) {
uint32_t aux1, aux2, aux3, aux4;
aux1 = *p4++;
aux2 = *p4++;
aux3 = *p4++;
aux4 = *p4++;
aux5 = *p4++;
aux6 = *p4++;
aux7 = *p4++;
count -= 28;
count -= 16;
aux1 = aux1 - ((aux1 >> 1) & 0x55555555);
aux1 = (aux1 & 0x33333333) + ((aux1 >> 2) & 0x33333333);
@@ -92,19 +89,10 @@ size_t redisPopcount(void *s, long count) {
aux3 = (aux3 & 0x33333333) + ((aux3 >> 2) & 0x33333333);
aux4 = aux4 - ((aux4 >> 1) & 0x55555555);
aux4 = (aux4 & 0x33333333) + ((aux4 >> 2) & 0x33333333);
aux5 = aux5 - ((aux5 >> 1) & 0x55555555);
aux5 = (aux5 & 0x33333333) + ((aux5 >> 2) & 0x33333333);
aux6 = aux6 - ((aux6 >> 1) & 0x55555555);
aux6 = (aux6 & 0x33333333) + ((aux6 >> 2) & 0x33333333);
aux7 = aux7 - ((aux7 >> 1) & 0x55555555);
aux7 = (aux7 & 0x33333333) + ((aux7 >> 2) & 0x33333333);
bits += ((((aux1 + (aux1 >> 4)) & 0x0F0F0F0F) +
((aux2 + (aux2 >> 4)) & 0x0F0F0F0F) +
((aux3 + (aux3 >> 4)) & 0x0F0F0F0F) +
((aux4 + (aux4 >> 4)) & 0x0F0F0F0F) +
((aux5 + (aux5 >> 4)) & 0x0F0F0F0F) +
((aux6 + (aux6 >> 4)) & 0x0F0F0F0F) +
((aux7 + (aux7 >> 4)) & 0x0F0F0F0F))* 0x01010101) >> 24;
bits += ((((aux1 + (aux1 >> 4)) & 0x0F0F0F0F) * 0x01010101) >> 24) +
((((aux2 + (aux2 >> 4)) & 0x0F0F0F0F) * 0x01010101) >> 24) +
((((aux3 + (aux3 >> 4)) & 0x0F0F0F0F) * 0x01010101) >> 24) +
((((aux4 + (aux4 >> 4)) & 0x0F0F0F0F) * 0x01010101) >> 24);
}
/* Count the remaining bytes. */
p = (unsigned char*)p4;
@@ -360,7 +348,7 @@ void bitopCommand(redisClient *c) {
* can take a fast path that performs much better than the
* vanilla algorithm. */
j = 0;
if (minlen >= sizeof(unsigned long)*4 && numkeys <= 16) {
if (minlen && numkeys <= 16) {
unsigned long *lp[16];
unsigned long *lres = (unsigned long*) res;
+1 -26
View File
@@ -59,8 +59,6 @@
* When implementing a new type of blocking opeation, the implementation
* should modify unblockClient() and replyToBlockedClientTimedOut() in order
* to handle the btype-specific behavior of this two functions.
* If the blocking operation waits for certain keys to change state, the
* clusterRedirectBlockedClientIfNeeded() function should also be updated.
*/
#include "redis.h"
@@ -116,6 +114,7 @@ void processUnblockedClients(void) {
c = ln->value;
listDelNode(server.unblocked_clients,ln);
c->flags &= ~REDIS_UNBLOCKED;
c->btype = REDIS_BLOCKED_NONE;
/* Process remaining data in the input buffer. */
if (c->querybuf && sdslen(c->querybuf) > 0) {
@@ -157,27 +156,3 @@ void replyToBlockedClientTimedOut(redisClient *c) {
}
}
/* Mass-unblock clients because something changed in the instance that makes
* blocking no longer safe. For example clients blocked in list operations
* in an instance which turns from master to slave is unsafe, so this function
* is called when a master turns into a slave.
*
* The semantics is to send an -UNBLOCKED error to the client, disconnecting
* it at the same time. */
void disconnectAllBlockedClients(void) {
listNode *ln;
listIter li;
listRewind(server.clients,&li);
while((ln = listNext(&li))) {
redisClient *c = listNodeValue(ln);
if (c->flags & REDIS_BLOCKED) {
addReplySds(c,sdsnew(
"-UNBLOCKED force unblock from blocking operation, "
"instance state changed (master -> slave?)\r\n"));
unblockClient(c);
c->flags |= REDIS_CLOSE_AFTER_REPLY;
}
}
}
+198 -492
View File
File diff suppressed because it is too large Load Diff
+7 -16
View File
@@ -26,12 +26,10 @@
/* Redirection errors returned by getNodeByQuery(). */
#define REDIS_CLUSTER_REDIR_NONE 0 /* Node can serve the request. */
#define REDIS_CLUSTER_REDIR_CROSS_SLOT 1 /* -CROSSSLOT request. */
#define REDIS_CLUSTER_REDIR_UNSTABLE 2 /* -TRYAGAIN redirection required */
#define REDIS_CLUSTER_REDIR_CROSS_SLOT 1 /* Keys in different slots. */
#define REDIS_CLUSTER_REDIR_UNSTABLE 2 /* Keys in slot resharding. */
#define REDIS_CLUSTER_REDIR_ASK 3 /* -ASK redirection required. */
#define REDIS_CLUSTER_REDIR_MOVED 4 /* -MOVED redirection required. */
#define REDIS_CLUSTER_REDIR_DOWN_STATE 5 /* -CLUSTERDOWN, global state. */
#define REDIS_CLUSTER_REDIR_DOWN_UNBOUND 6 /* -CLUSTERDOWN, unbound slot. */
struct clusterNode;
@@ -165,11 +163,10 @@ typedef struct {
char nodename[REDIS_CLUSTER_NAMELEN];
uint32_t ping_sent;
uint32_t pong_received;
char ip[REDIS_IP_STR_LEN]; /* IP address last time it was seen */
uint16_t port; /* port last time it was seen */
uint16_t flags; /* node->flags copy */
uint16_t notused1; /* Some room for future improvements. */
uint32_t notused2;
char ip[REDIS_IP_STR_LEN]; /* IP address last time it was seen */
uint16_t port; /* port last time it was seen */
uint16_t flags;
uint32_t notused; /* for 64 bit alignment */
} clusterMsgDataGossip;
typedef struct {
@@ -179,10 +176,7 @@ typedef struct {
typedef struct {
uint32_t channel_len;
uint32_t message_len;
/* We can't reclare bulk_data as bulk_data[] since this structure is
* nested. The 8 bytes are removed from the count during the message
* length computation. */
unsigned char bulk_data[8];
unsigned char bulk_data[8]; /* defined as 8 just for alignment concerns. */
} clusterMsgDataPublish;
typedef struct {
@@ -214,7 +208,6 @@ union clusterMsgData {
} update;
};
#define CLUSTER_PROTO_VER 0 /* Cluster bus protocol version. */
typedef struct {
char sig[4]; /* Siganture "RCmb" (Redis Cluster message bus). */
@@ -250,7 +243,5 @@ typedef struct {
/* ---------------------- API exported outside cluster.c -------------------- */
clusterNode *getNodeByQuery(redisClient *c, struct redisCommand *cmd, robj **argv, int argc, int *hashslot, int *ask);
int clusterRedirectBlockedClientIfNeeded(redisClient *c);
void clusterRedirectClient(redisClient *c, clusterNode *n, int hashslot, int error_code);
#endif /* __REDIS_CLUSTER_H */
+19 -12
View File
@@ -226,6 +226,11 @@ void loadServerConfigFromString(char *config) {
}
} else if (!strcasecmp(argv[0],"maxmemory") && argc == 2) {
server.maxmemory = memtoll(argv[1],NULL);
server.maxmemory_enforced = (double) server.maxmemory / server.maxmemory_frag_guess;
} else if (!strcasecmp(argv[0],"rss-aware-maxmemory") && argc==2) {
if ((server.rss_aware_maxmemory = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"maxmemory-policy") && argc == 2) {
if (!strcasecmp(argv[1],"volatile-lru")) {
server.maxmemory_policy = REDIS_MAXMEMORY_VOLATILE_LRU;
@@ -609,7 +614,6 @@ void loadServerConfig(char *filename, char *options) {
void configSetCommand(redisClient *c) {
robj *o;
long long ll;
int err;
redisAssertWithInfo(c,c->argv[2],sdsEncodedObject(c->argv[2]));
redisAssertWithInfo(c,c->argv[3],sdsEncodedObject(c->argv[3]));
o = c->argv[3];
@@ -629,15 +633,21 @@ void configSetCommand(redisClient *c) {
zfree(server.masterauth);
server.masterauth = ((char*)o->ptr)[0] ? zstrdup(o->ptr) : NULL;
} else if (!strcasecmp(c->argv[2]->ptr,"maxmemory")) {
ll = memtoll(o->ptr,&err);
if (err || ll < 0) goto badfmt;
if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
ll < 0) goto badfmt;
server.maxmemory = ll;
server.maxmemory_enforced = (double) server.maxmemory / server.maxmemory_frag_guess;
if (server.maxmemory) {
if (server.maxmemory < zmalloc_used_memory()) {
redisLog(REDIS_WARNING,"WARNING: the new maxmemory value set via CONFIG SET is smaller than the current memory usage. This will result in keys eviction and/or inability to accept new write commands depending on the maxmemory-policy.");
}
freeMemoryIfNeeded();
}
} else if (!strcasecmp(c->argv[2]->ptr,"rss-aware-maxmemory")) {
int yn = yesnotoi(o->ptr);
if (yn == -1) goto badfmt;
server.rss_aware_maxmemory = yn;
} else if (!strcasecmp(c->argv[2]->ptr,"maxclients")) {
int orig_value = server.maxclients;
@@ -785,11 +795,6 @@ void configSetCommand(redisClient *c) {
if (yn == -1) goto badfmt;
server.repl_slave_ro = yn;
} else if (!strcasecmp(c->argv[2]->ptr,"activerehashing")) {
int yn = yesnotoi(o->ptr);
if (yn == -1) goto badfmt;
server.activerehashing = yn;
} else if (!strcasecmp(c->argv[2]->ptr,"dir")) {
if (chdir((char*)o->ptr) == -1) {
addReplyErrorFormat(c,"Changing directory: %s", strerror(errno));
@@ -857,6 +862,7 @@ void configSetCommand(redisClient *c) {
* whole configuration string or accept it all, even if a single
* error in a single client class is present. */
for (j = 0; j < vlen; j++) {
char *eptr;
long val;
if ((j % 4) == 0) {
@@ -865,8 +871,8 @@ void configSetCommand(redisClient *c) {
goto badfmt;
}
} else {
val = memtoll(v[j], &err);
if (err || val < 0) {
val = strtoll(v[j], &eptr, 10);
if (eptr[0] != '\0' || val < 0) {
sdsfreesplitres(v,vlen);
goto badfmt;
}
@@ -900,8 +906,7 @@ void configSetCommand(redisClient *c) {
if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll <= 0) goto badfmt;
server.repl_timeout = ll;
} else if (!strcasecmp(c->argv[2]->ptr,"repl-backlog-size")) {
ll = memtoll(o->ptr,&err);
if (err || ll < 0) goto badfmt;
if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll <= 0) goto badfmt;
resizeReplicationBacklog(ll);
} else if (!strcasecmp(c->argv[2]->ptr,"repl-backlog-ttl")) {
if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
@@ -1099,6 +1104,8 @@ void configGetCommand(redisClient *c) {
server.aof_rewrite_incremental_fsync);
config_get_bool_field("aof-load-truncated",
server.aof_load_truncated);
config_get_bool_field("rss-aware-maxmemory",
server.rss_aware_maxmemory);
/* Everything we can't handle with macros follows. */
+6 -8
View File
@@ -34,11 +34,6 @@
#include <AvailabilityMacros.h>
#endif
#ifdef __linux__
#include <linux/version.h>
#include <features.h>
#endif
/* Define redis_fstat to fstat or fstat64() */
#if defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
#define redis_fstat fstat64
@@ -53,16 +48,17 @@
#define HAVE_PROC_STAT 1
#define HAVE_PROC_MAPS 1
#define HAVE_PROC_SMAPS 1
#define HAVE_PROC_SOMAXCONN 1
#define HAVE_RSS_REPORTING 1
#endif
/* Test for task_info() */
#if defined(__APPLE__)
#define HAVE_TASKINFO 1
#define HAVE_RSS_REPORTING 1
#endif
/* Test for backtrace() */
#if defined(__APPLE__) || (defined(__linux__) && defined(__GLIBC__))
#if defined(__APPLE__) || defined(__linux__)
#define HAVE_BACKTRACE 1
#endif
@@ -92,6 +88,8 @@
/* Define rdb_fsync_range to sync_file_range() on Linux, otherwise we use
* the plain fsync() call. */
#ifdef __linux__
#include <linux/version.h>
#include <features.h>
#if defined(__GLIBC__) && defined(__GLIBC_PREREQ)
#if (LINUX_VERSION_CODE >= 0x020611 && __GLIBC_PREREQ(2, 6))
#define HAVE_SYNC_FILE_RANGE 1
@@ -116,7 +114,7 @@
#define USE_SETPROCTITLE
#endif
#if ((defined __linux && defined(__GLIBC__)) || defined __APPLE__)
#if (defined __linux || defined __APPLE__)
#define USE_SETPROCTITLE
#define INIT_SETPROCTITLE_REPLACEMENT
void spt_init(int argc, char *argv[]);
+6 -10
View File
@@ -293,17 +293,13 @@ void delCommand(redisClient *c) {
addReplyLongLong(c,deleted);
}
/* EXISTS key1 key2 ... key_N.
* Return value is the number of keys existing. */
void existsCommand(redisClient *c) {
long long count = 0;
int j;
for (j = 1; j < c->argc; j++) {
expireIfNeeded(c->db,c->argv[j]);
if (dbExists(c->db,c->argv[j])) count++;
expireIfNeeded(c->db,c->argv[1]);
if (dbExists(c->db,c->argv[1])) {
addReply(c, shared.cone);
} else {
addReply(c, shared.czero);
}
addReplyLongLong(c,count);
}
void selectCommand(redisClient *c) {
@@ -385,7 +381,7 @@ void scanCallback(void *privdata, const dictEntry *de) {
} else if (o->type == REDIS_ZSET) {
key = dictGetKey(de);
incrRefCount(key);
val = createStringObjectFromLongDouble(*(double*)dictGetVal(de),0);
val = createStringObjectFromLongDouble(*(double*)dictGetVal(de));
} else {
redisPanic("Type not handled in SCAN callback.");
}
+1 -1
View File
@@ -301,7 +301,7 @@ void debugCommand(redisClient *c) {
"lru:%d lru_seconds_idle:%llu",
(void*)val, val->refcount,
strenc, (long long) rdbSavedObjectLen(val),
val->lru, estimateObjectIdleTime(val)/1000);
val->lru, estimateObjectIdleTime(val));
} else if (!strcasecmp(c->argv[1]->ptr,"sdslen") && c->argc == 3) {
dictEntry *de;
robj *val;
+35 -90
View File
@@ -211,9 +211,6 @@ int dictExpand(dict *d, unsigned long size)
if (dictIsRehashing(d) || d->ht[0].used > size)
return DICT_ERR;
/* Rehashing to the same table size is not useful. */
if (realsize == d->ht[0].size) return DICT_ERR;
/* Allocate the new hash table and initialize all pointers to NULL */
n.size = realsize;
n.sizemask = realsize-1;
@@ -235,27 +232,27 @@ int dictExpand(dict *d, unsigned long size)
/* Performs N steps of incremental rehashing. Returns 1 if there are still
* keys to move from the old to the new hash table, otherwise 0 is returned.
*
* Note that a rehashing step consists in moving a bucket (that may have more
* than one key as we use chaining) from the old to the new hash table, however
* since part of the hash table may be composed of empty spaces, it is not
* guaranteed that this function will rehash even a single bucket, since it
* will visit at max N*10 empty buckets in total, otherwise the amount of
* work it does would be unbound and the function may block for a long time. */
* than one key as we use chaining) from the old to the new hash table. */
int dictRehash(dict *d, int n) {
int empty_visits = n*10; /* Max number of empty buckets to visit. */
if (!dictIsRehashing(d)) return 0;
while(n-- && d->ht[0].used != 0) {
while(n--) {
dictEntry *de, *nextde;
/* Check if we already rehashed the whole table... */
if (d->ht[0].used == 0) {
zfree(d->ht[0].table);
d->ht[0] = d->ht[1];
_dictReset(&d->ht[1]);
d->rehashidx = -1;
return 0;
}
/* Note that rehashidx can't overflow as we are sure there are more
* elements because ht[0].used != 0 */
assert(d->ht[0].size > (unsigned long)d->rehashidx);
while(d->ht[0].table[d->rehashidx] == NULL) {
d->rehashidx++;
if (--empty_visits == 0) return 1;
}
while(d->ht[0].table[d->rehashidx] == NULL) d->rehashidx++;
de = d->ht[0].table[d->rehashidx];
/* Move all the keys in this bucket from the old to the new hash HT */
while(de) {
@@ -273,17 +270,6 @@ int dictRehash(dict *d, int n) {
d->ht[0].table[d->rehashidx] = NULL;
d->rehashidx++;
}
/* Check if we already rehashed the whole table... */
if (d->ht[0].used == 0) {
zfree(d->ht[0].table);
d->ht[0] = d->ht[1];
_dictReset(&d->ht[1]);
d->rehashidx = -1;
return 0;
}
/* More to rehash... */
return 1;
}
@@ -630,11 +616,7 @@ dictEntry *dictGetRandomKey(dict *d)
if (dictIsRehashing(d)) _dictRehashStep(d);
if (dictIsRehashing(d)) {
do {
/* We are sure there are no elements in indexes from 0
* to rehashidx-1 */
h = d->rehashidx + (random() % (d->ht[0].size +
d->ht[1].size -
d->rehashidx));
h = random() % (d->ht[0].size+d->ht[1].size);
he = (h >= d->ht[0].size) ? d->ht[1].table[h - d->ht[0].size] :
d->ht[0].table[h];
} while(he == NULL);
@@ -661,12 +643,9 @@ dictEntry *dictGetRandomKey(dict *d)
return he;
}
/* This function samples the dictionary to return a few keys from random
* locations.
*
* It does not guarantee to return all the keys specified in 'count', nor
* it does guarantee to return non-duplicated elements, however it will make
* some effort to do both things.
/* This is a version of dictGetRandomKey() that is modified in order to
* return multiple entries by jumping at a random place of the hash table
* and scanning linearly for entries.
*
* Returned pointers to hash table entries are stored into 'des' that
* points to an array of dictEntry pointers. The array must have room for
@@ -675,65 +654,28 @@ dictEntry *dictGetRandomKey(dict *d)
*
* The function returns the number of items stored into 'des', that may
* be less than 'count' if the hash table has less than 'count' elements
* inside, or if not enough elements were found in a reasonable amount of
* steps.
* inside.
*
* Note that this function is not suitable when you need a good distribution
* of the returned items, but only when you need to "sample" a given number
* of continuous elements to run some kind of algorithm or to produce
* statistics. However the function is much faster than dictGetRandomKey()
* at producing N elements. */
unsigned int dictGetSomeKeys(dict *d, dictEntry **des, unsigned int count) {
unsigned int j; /* internal hash table id, 0 or 1. */
unsigned int tables; /* 1 or 2 tables? */
unsigned int stored = 0, maxsizemask;
unsigned int maxsteps;
* at producing N elements, and the elements are guaranteed to be non
* repeating. */
unsigned int dictGetRandomKeys(dict *d, dictEntry **des, unsigned int count) {
int j; /* internal hash table id, 0 or 1. */
unsigned int stored = 0;
if (dictSize(d) < count) count = dictSize(d);
maxsteps = count*10;
while(stored < count) {
for (j = 0; j < 2; j++) {
/* Pick a random point inside the hash table 0 or 1. */
unsigned int i = random() & d->ht[j].sizemask;
int size = d->ht[j].size;
/* Try to do a rehashing work proportional to 'count'. */
for (j = 0; j < count; j++) {
if (dictIsRehashing(d))
_dictRehashStep(d);
else
break;
}
tables = dictIsRehashing(d) ? 2 : 1;
maxsizemask = d->ht[0].sizemask;
if (tables > 1 && maxsizemask < d->ht[1].sizemask)
maxsizemask = d->ht[1].sizemask;
/* Pick a random point inside the larger table. */
unsigned int i = random() & maxsizemask;
unsigned int emptylen = 0; /* Continuous empty entries so far. */
while(stored < count && maxsteps--) {
for (j = 0; j < tables; j++) {
/* Invariant of the dict.c rehashing: up to the indexes already
* visited in ht[0] during the rehashing, there are no populated
* buckets, so we can skip ht[0] for indexes between 0 and idx-1. */
if (tables == 2 && j == 0 && i < (unsigned int) d->rehashidx) {
/* Moreover, if we are currently out of range in the second
* table, there will be no elements in both tables up to
* the current rehashing index, so we jump if possible.
* (this happens when going from big to small table). */
if (i >= d->ht[1].size) i = d->rehashidx;
continue;
}
if (i >= d->ht[j].size) continue; /* Out of range for this table. */
dictEntry *he = d->ht[j].table[i];
/* Count contiguous empty buckets, and jump to other
* locations if they reach 'count' (with a minimum of 5). */
if (he == NULL) {
emptylen++;
if (emptylen >= 5 && emptylen > count) {
i = random() & maxsizemask;
emptylen = 0;
}
} else {
emptylen = 0;
/* Make sure to visit every bucket by iterating 'size' times. */
while(size--) {
dictEntry *he = d->ht[j].table[i];
while (he) {
/* Collect all the elements of the buckets found non
* empty while iterating. */
@@ -743,11 +685,14 @@ unsigned int dictGetSomeKeys(dict *d, dictEntry **des, unsigned int count) {
stored++;
if (stored == count) return stored;
}
i = (i+1) & d->ht[j].sizemask;
}
/* If there is only one table and we iterated it all, we should
* already have 'count' elements. Assert this condition. */
assert(dictIsRehashing(d) != 0);
}
i = (i+1) & maxsizemask;
}
return stored;
return stored; /* Never reached. */
}
/* Function to reverse bits. Algorithm from:
+1 -1
View File
@@ -164,7 +164,7 @@ dictIterator *dictGetSafeIterator(dict *d);
dictEntry *dictNext(dictIterator *iter);
void dictReleaseIterator(dictIterator *iter);
dictEntry *dictGetRandomKey(dict *d);
unsigned int dictGetSomeKeys(dict *d, dictEntry **des, unsigned int count);
unsigned int dictGetRandomKeys(dict *d, dictEntry **des, unsigned int count);
void dictPrintStats(dict *d);
unsigned int dictGenHashFunction(const void *key, int len);
unsigned int dictGenCaseHashFunction(const unsigned char *buf, int len);
-1
View File
@@ -34,7 +34,6 @@
#if defined(__linux__)
#define _GNU_SOURCE
#define _DEFAULT_SOURCE
#endif
#if defined(_AIX)
+1 -1
View File
@@ -1213,7 +1213,7 @@ void pfcountCommand(redisClient *c) {
for (j = 1; j < c->argc; j++) {
/* Check type and size. */
robj *o = lookupKeyRead(c->db,c->argv[j]);
if (o == NULL) continue; /* Assume empty HLL for non existing var.*/
if (o == NULL) continue; /* Assume empty HLL for non existing var. */
if (isHLLObjectOrReply(c,o) != REDIS_OK) return;
/* Merge with this HLL with our 'max' HHL by setting max[i]
+6 -14
View File
@@ -73,7 +73,6 @@ int THPIsEnabled(void) {
fclose(fp);
return (strstr(buf,"[never]") == NULL) ? 1 : 0;
}
#endif
/* Report the amount of AnonHugePages in smap, in bytes. If the return
* value of the function is non-zero, the process is being targeted by
@@ -81,6 +80,7 @@ int THPIsEnabled(void) {
int THPGetAnonHugePagesSize(void) {
return zmalloc_get_smap_bytes_by_field("AnonHugePages:");
}
#endif
/* ---------------------------- Latency API --------------------------------- */
@@ -228,7 +228,6 @@ sds createLatencyReport(void) {
int advise_write_load_info = 0; /* Print info about AOF and write load. */
int advise_hz = 0; /* Use higher HZ. */
int advise_large_objects = 0; /* Deletion of large objects. */
int advise_mass_eviction = 0; /* Avoid mass eviction of keys. */
int advise_relax_fsync_policy = 0; /* appendfsync always is slow. */
int advise_disable_thp = 0; /* AnonHugePages detected. */
int advices = 0;
@@ -248,7 +247,7 @@ sds createLatencyReport(void) {
dictEntry *de;
int eventnum = 0;
di = dictGetSafeIterator(server.latency_events);
di = dictGetIterator(server.latency_events);
while((de = dictNext(di)) != NULL) {
char *event = dictGetKey(de);
struct latencyTimeSeries *ts = dictGetVal(de);
@@ -365,13 +364,8 @@ sds createLatencyReport(void) {
}
/* Eviction cycle. */
if (!strcasecmp(event,"eviction-del")) {
advise_large_objects = 1;
advices++;
}
if (!strcasecmp(event,"eviction-cycle")) {
advise_mass_eviction = 1;
advise_large_objects = 1;
advices++;
}
@@ -458,10 +452,6 @@ sds createLatencyReport(void) {
report = sdscat(report,"- Deleting, expiring or evicting (because of maxmemory policy) large objects is a blocking operation. If you have very large objects that are often deleted, expired, or evicted, try to fragment those objects into multiple smaller objects.\n");
}
if (advise_mass_eviction) {
report = sdscat(report,"- Sudden changes to the 'maxmemory' setting via 'CONFIG SET', or allocation of large objects via sets or sorted sets intersections, STORE option of SORT, Redis Cluster large keys migrations (RESTORE command), may create sudden memory pressure forcing the server to block trying to evict keys. \n");
}
if (advise_disable_thp) {
report = sdscat(report,"- I detected a non zero amount of anonymous huge pages used by your process. This creates very serious latency events in different conditions, especially when Redis is persisting on disk. To disable THP support use the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled', make sure to also add it into /etc/rc.local so that the command will be executed again after a reboot. Note that even if you have already disabled THP, you still need to restart the Redis process to get rid of the huge pages already created.\n");
}
@@ -522,6 +512,7 @@ sds latencyCommandGenSparkeline(char *event, struct latencyTimeSeries *ts) {
for (j = 0; j < LATENCY_TS_LEN; j++) {
int i = (ts->idx + j) % LATENCY_TS_LEN;
int elapsed;
char *label;
char buf[64];
if (ts->samples[i].time == 0) continue;
@@ -543,7 +534,8 @@ sds latencyCommandGenSparkeline(char *event, struct latencyTimeSeries *ts) {
snprintf(buf,sizeof(buf),"%dh",elapsed/3600);
else
snprintf(buf,sizeof(buf),"%dd",elapsed/(3600*24));
sparklineSequenceAddSample(seq,ts->samples[i].latency,buf);
label = zstrdup(buf);
sparklineSequenceAddSample(seq,ts->samples[i].latency,label);
}
graph = sdscatprintf(graph,
-4
View File
@@ -86,8 +86,4 @@ int THPIsEnabled(void);
(var) >= server.latency_monitor_threshold) \
latencyAddSample((event),(var));
/* Remove time from a nested event. */
#define latencyRemoveNestedEvent(event_var,nested_var) \
event_var += nested_var;
#endif /* __LATENCY_H */
-3
View File
@@ -35,9 +35,6 @@
#include <errno.h>
#include <termios.h>
#include <sys/ioctl.h>
#if defined(__sun)
#include <stropts.h>
#endif
#include "config.h"
#if (ULONG_MAX == 4294967295UL)
+13 -48
View File
@@ -135,49 +135,23 @@ redisClient *createClient(int fd) {
* returns REDIS_OK, and make sure to install the write handler in our event
* loop so that when the socket is writable new data gets written.
*
* If the client should not receive new data, because it is a fake client
* (used to load AOF in memory), a master or because the setup of the write
* handler failed, the function returns REDIS_ERR.
*
* The function may return REDIS_OK without actually installing the write
* event handler in the following cases:
*
* 1) The event handler should already be installed since the output buffer
* already contained something.
* 2) The client is a slave but not yet online, so we want to just accumulate
* writes in the buffer but not actually sending them yet.
* If the client should not receive new data, because it is a fake client,
* a master, a slave not yet online, or because the setup of the write handler
* failed, the function returns REDIS_ERR.
*
* Typically gets called every time a reply is built, before adding more
* data to the clients output buffers. If the function returns REDIS_ERR no
* data should be appended to the output buffers. */
int prepareClientToWrite(redisClient *c) {
/* If it's the Lua client we always return ok without installing any
* handler since there is no socket at all. */
if (c->flags & REDIS_LUA_CLIENT) return REDIS_OK;
/* Masters don't receive replies, unless REDIS_MASTER_FORCE_REPLY flag
* is set. */
if ((c->flags & REDIS_MASTER) &&
!(c->flags & REDIS_MASTER_FORCE_REPLY)) return REDIS_ERR;
if (c->fd <= 0) return REDIS_ERR; /* Fake client for AOF loading. */
/* Only install the handler if not already installed and, in case of
* slaves, if the client can actually receive writes. */
if (c->fd <= 0) return REDIS_ERR; /* Fake client */
if (c->bufpos == 0 && listLength(c->reply) == 0 &&
(c->replstate == REDIS_REPL_NONE ||
(c->replstate == REDIS_REPL_ONLINE && !c->repl_put_online_on_ack)))
{
/* Try to install the write handler. */
if (aeCreateFileEvent(server.el, c->fd, AE_WRITABLE,
sendReplyToClient, c) == AE_ERR)
{
freeClientAsync(c);
return REDIS_ERR;
}
}
/* Authorize the caller to queue in the output buffer of this client. */
c->replstate == REDIS_REPL_ONLINE) &&
aeCreateFileEvent(server.el, c->fd, AE_WRITABLE,
sendReplyToClient, c) == AE_ERR) return REDIS_ERR;
return REDIS_OK;
}
@@ -797,7 +771,7 @@ void freeClient(redisClient *c) {
* a context where calling freeClient() is not possible, because the client
* should be valid for the continuation of the flow of the program. */
void freeClientAsync(redisClient *c) {
if (c->flags & REDIS_CLOSE_ASAP || c->flags & REDIS_LUA_CLIENT) return;
if (c->flags & REDIS_CLOSE_ASAP) return;
c->flags |= REDIS_CLOSE_ASAP;
listAddNodeTail(server.clients_to_close,c);
}
@@ -865,7 +839,6 @@ void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask) {
*
* However if we are over the maxmemory limit we ignore that and
* just deliver as much data as it is possible to deliver. */
server.stat_net_output_bytes += totwritten;
if (totwritten > REDIS_MAX_WRITE_PER_EVENT &&
(server.maxmemory == 0 ||
zmalloc_used_memory() < server.maxmemory)) break;
@@ -953,10 +926,8 @@ int processInlineBuffer(redisClient *c) {
sdsrange(c->querybuf,querylen+2,-1);
/* Setup argv array on client structure */
if (argc) {
if (c->argv) zfree(c->argv);
c->argv = zmalloc(sizeof(robj*)*argc);
}
if (c->argv) zfree(c->argv);
c->argv = zmalloc(sizeof(robj*)*argc);
/* Create redis objects for all arguments. */
for (c->argc = 0, j = 0; j < argc; j++) {
@@ -974,7 +945,7 @@ int processInlineBuffer(redisClient *c) {
/* Helper function. Trims query buffer to make the function that processes
* multi bulk requests idempotent. */
static void setProtocolError(redisClient *c, int pos) {
if (server.verbosity <= REDIS_VERBOSE) {
if (server.verbosity >= REDIS_VERBOSE) {
sds client = catClientInfoString(sdsempty(),c);
redisLog(REDIS_VERBOSE,
"Protocol error from client: %s", client);
@@ -1208,7 +1179,6 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
sdsIncrLen(c->querybuf,nread);
c->lastinteraction = server.unixtime;
if (c->flags & REDIS_MASTER) c->reploff += nread;
server.stat_net_input_bytes += nread;
} else {
server.current_client = NULL;
return;
@@ -1715,9 +1685,7 @@ void pauseClients(mstime_t end) {
/* Return non-zero if clients are currently paused. As a side effect the
* function checks if the pause time was reached and clear it. */
int clientsArePaused(void) {
if (server.clients_paused &&
server.clients_pause_end_time < server.mstime)
{
if (server.clients_paused && server.clients_pause_end_time < server.mstime) {
listNode *ln;
listIter li;
redisClient *c;
@@ -1730,10 +1698,7 @@ int clientsArePaused(void) {
while ((ln = listNext(&li)) != NULL) {
c = listNodeValue(ln);
/* Don't touch slaves and blocked clients. The latter pending
* requests be processed when unblocked. */
if (c->flags & (REDIS_SLAVE|REDIS_BLOCKED)) continue;
c->flags |= REDIS_UNBLOCKED;
if (c->flags & REDIS_SLAVE) continue;
listAddNodeTail(server.unblocked_clients,c);
}
}
+16 -34
View File
@@ -109,44 +109,26 @@ robj *createStringObjectFromLongLong(long long value) {
return o;
}
/* Create a string object from a long double. If humanfriendly is non-zero
* it does not use exponential format and trims trailing zeroes at the end,
* however this results in loss of precision. Otherwise exp format is used
* and the output of snprintf() is not modified.
*
* The 'humanfriendly' option is used for INCRBYFLOAT and HINCRBYFLOAT. */
robj *createStringObjectFromLongDouble(long double value, int humanfriendly) {
/* Note: this function is defined into object.c since here it is where it
* belongs but it is actually designed to be used just for INCRBYFLOAT */
robj *createStringObjectFromLongDouble(long double value) {
char buf[256];
int len;
if (isinf(value)) {
/* Libc in odd systems (Hi Solaris!) will format infinite in a
* different way, so better to handle it in an explicit way. */
if (value > 0) {
memcpy(buf,"inf",3);
len = 3;
} else {
memcpy(buf,"-inf",4);
len = 4;
/* We use 17 digits precision since with 128 bit floats that precision
* after rounding is able to represent most small decimal numbers in a way
* that is "non surprising" for the user (that is, most small decimal
* numbers will be represented in a way that when converted back into
* a string are exactly the same as what the user typed.) */
len = snprintf(buf,sizeof(buf),"%.17Lf", value);
/* Now remove trailing zeroes after the '.' */
if (strchr(buf,'.') != NULL) {
char *p = buf+len-1;
while(*p == '0') {
p--;
len--;
}
} else if (humanfriendly) {
/* We use 17 digits precision since with 128 bit floats that precision
* after rounding is able to represent most small decimal numbers in a
* way that is "non surprising" for the user (that is, most small
* decimal numbers will be represented in a way that when converted
* back into a string are exactly the same as what the user typed.) */
len = snprintf(buf,sizeof(buf),"%.17Lf", value);
/* Now remove trailing zeroes after the '.' */
if (strchr(buf,'.') != NULL) {
char *p = buf+len-1;
while(*p == '0') {
p--;
len--;
}
if (*p == '.') len--;
}
} else {
len = snprintf(buf,sizeof(buf),"%.17Lg", value);
if (*p == '.') len--;
}
return createStringObject(buf,len);
}
+2 -5
View File
@@ -755,9 +755,9 @@ int rdbSave(char *filename) {
return REDIS_OK;
werr:
redisLog(REDIS_WARNING,"Write error saving DB on disk: %s", strerror(errno));
fclose(fp);
unlink(tmpfile);
redisLog(REDIS_WARNING,"Write error saving DB on disk: %s", strerror(errno));
return REDIS_ERR;
}
@@ -1087,9 +1087,8 @@ void startLoading(FILE *fp) {
/* Load the DB */
server.loading = 1;
server.loading_start_time = time(NULL);
server.loading_loaded_bytes = 0;
if (fstat(fileno(fp), &sb) == -1) {
server.loading_total_bytes = 0;
server.loading_total_bytes = 1; /* just to avoid division by zero */
} else {
server.loading_total_bytes = sb.st_size;
}
@@ -1492,9 +1491,7 @@ int rdbSaveToSlavesSockets(void) {
{
retval = REDIS_ERR;
}
zfree(msg);
}
zfree(clientids);
exitFromChild((retval == REDIS_OK) ? 0 : 1);
} else {
/* Parent */
+34 -41
View File
@@ -90,10 +90,9 @@ typedef struct _client {
long long start; /* Start time of a request */
long long latency; /* Request latency */
int pending; /* Number of pending requests (replies to consume) */
int prefix_pending; /* If non-zero, number of pending prefix commands. Commands
such as auth and select are prefixed to the pipeline of
benchmark commands and discarded after the first send. */
int prefixlen; /* Size in bytes of the pending prefix commands */
int selectlen; /* If non-zero, a SELECT of 'selectlen' bytes is currently
used as a prefix of the pipline of commands. This gets
discarded the first time it's sent. */
} *client;
/* Prototypes */
@@ -213,21 +212,20 @@ static void readHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
}
freeReplyObject(reply);
/* This is an OK for prefix commands such as auth and select.*/
if (c->prefix_pending > 0) {
c->prefix_pending--;
if (c->selectlen) {
size_t j;
/* This is the OK from SELECT. Just discard the SELECT
* from the buffer. */
c->pending--;
/* Discard prefix commands on first response.*/
if (c->prefixlen > 0) {
size_t j;
sdsrange(c->obuf, c->prefixlen, -1);
/* We also need to fix the pointers to the strings
* we need to randomize. */
for (j = 0; j < c->randlen; j++)
c->randptr[j] -= c->prefixlen;
c->prefixlen = 0;
}
continue;
sdsrange(c->obuf,c->selectlen,-1);
/* We also need to fix the pointers to the strings
* we need to randomize. */
for (j = 0; j < c->randlen; j++)
c->randptr[j] -= c->selectlen;
c->selectlen = 0;
continue;
}
if (config.requests_finished < config.requests)
@@ -301,7 +299,8 @@ static void writeHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
* 2) The offsets of the __rand_int__ elements inside the command line, used
* for arguments randomization.
*
* Even when cloning another client, prefix commands are applied if needed.*/
* Even when cloning another client, the SELECT command is automatically prefixed
* if needed. */
static client createClient(char *cmd, size_t len, client from) {
int j;
client c = zmalloc(sizeof(struct _client));
@@ -326,16 +325,12 @@ static client createClient(char *cmd, size_t len, client from) {
* Queue N requests accordingly to the pipeline size, or simply clone
* the example client buffer. */
c->obuf = sdsempty();
/* Prefix the request buffer with AUTH and/or SELECT commands, if applicable.
* These commands are discarded after the first response, so if the client is
* reused the commands will not be used again. */
c->prefix_pending = 0;
if (config.auth) {
char *buf = NULL;
int len = redisFormatCommand(&buf, "AUTH %s", config.auth);
c->obuf = sdscatlen(c->obuf, buf, len);
free(buf);
c->prefix_pending++;
}
/* If a DB number different than zero is selected, prefix our request
@@ -345,23 +340,26 @@ static client createClient(char *cmd, size_t len, client from) {
if (config.dbnum != 0) {
c->obuf = sdscatprintf(c->obuf,"*2\r\n$6\r\nSELECT\r\n$%d\r\n%s\r\n",
(int)sdslen(config.dbnumstr),config.dbnumstr);
c->prefix_pending++;
c->selectlen = sdslen(c->obuf);
} else {
c->selectlen = 0;
}
c->prefixlen = sdslen(c->obuf);
/* Append the request itself. */
if (from) {
c->obuf = sdscatlen(c->obuf,
from->obuf+from->prefixlen,
sdslen(from->obuf)-from->prefixlen);
from->obuf+from->selectlen,
sdslen(from->obuf)-from->selectlen);
} else {
for (j = 0; j < config.pipeline; j++)
c->obuf = sdscatlen(c->obuf,cmd,len);
}
c->written = 0;
c->pending = config.pipeline+c->prefix_pending;
c->pending = config.pipeline;
c->randptr = NULL;
c->randlen = 0;
if (c->selectlen) c->pending++;
/* Find substrings in the output buffer that need to be randomized. */
if (config.randomkeys) {
@@ -373,7 +371,7 @@ static client createClient(char *cmd, size_t len, client from) {
for (j = 0; j < (int)c->randlen; j++) {
c->randptr[j] = c->obuf + (from->randptr[j]-from->obuf);
/* Adjust for the different select prefix length. */
c->randptr[j] += c->prefixlen - from->prefixlen;
c->randptr[j] += c->selectlen - from->selectlen;
}
} else {
char *p = c->obuf;
@@ -392,8 +390,7 @@ static client createClient(char *cmd, size_t len, client from) {
}
}
}
if (config.idlemode == 0)
aeCreateFileEvent(config.el,c->context->fd,AE_WRITABLE,writeHandler,c);
aeCreateFileEvent(config.el,c->context->fd,AE_WRITABLE,writeHandler,c);
listAddNodeTail(config.clients,c);
config.liveclients++;
return c;
@@ -558,7 +555,7 @@ usage:
" -s <socket> Server socket (overrides host and port)\n"
" -a <password> Password for Redis Auth\n"
" -c <clients> Number of parallel connections (default 50)\n"
" -n <requests> Total number of requests (default 100000)\n"
" -n <requests> Total number of requests (default 10000)\n"
" -d <size> Data size of SET/GET value in bytes (default 2)\n"
" -dbnum <db> SELECT the specified db number (default 0)\n"
" -k <boolean> 1=keep alive 0=reconnect (default 1)\n"
@@ -602,13 +599,9 @@ int showThroughput(struct aeEventLoop *eventLoop, long long id, void *clientData
if (config.liveclients == 0) {
fprintf(stderr,"All clients disconnected... aborting.\n");
exit(1);
}
if (config.csv) return 250;
if (config.idlemode == 1) {
printf("clients: %d\r", config.liveclients);
fflush(stdout);
return 250;
}
if (config.csv) return 250;
float dt = (float)(mstime()-config.start)/1000.0;
float rps = (float)config.requests_finished/dt;
printf("%s: %.2f\r", config.title, rps);
@@ -642,7 +635,7 @@ int main(int argc, const char **argv) {
signal(SIGPIPE, SIG_IGN);
config.numclients = 50;
config.requests = 100000;
config.requests = 10000;
config.liveclients = 0;
config.el = aeCreateEventLoop(1024*10);
aeCreateTimeEvent(config.el,1,showThroughput,NULL,NULL);
@@ -700,8 +693,8 @@ int main(int argc, const char **argv) {
}
/* Run default benchmark suite. */
data = zmalloc(config.datasize+1);
do {
data = zmalloc(config.datasize+1);
memset(data,'x',config.datasize);
data[config.datasize] = '\0';
+40 -357
View File
@@ -44,7 +44,6 @@
#include <assert.h>
#include <fcntl.h>
#include <limits.h>
#include <math.h>
#include "hiredis.h"
#include "sds.h"
@@ -61,19 +60,6 @@
#define OUTPUT_CSV 2
#define REDIS_CLI_KEEPALIVE_INTERVAL 15 /* seconds */
#define REDIS_CLI_DEFAULT_PIPE_TIMEOUT 30 /* seconds */
#define REDIS_CLI_HISTFILE_ENV "REDISCLI_HISTFILE"
#define REDIS_CLI_HISTFILE_DEFAULT ".rediscli_history"
/* --latency-dist palettes. */
int spectrum_palette_color_size = 19;
int spectrum_palette_color[] = {0,233,234,235,237,239,241,243,245,247,144,143,142,184,226,214,208,202,196};
int spectrum_palette_mono_size = 13;
int spectrum_palette_mono[] = {0,233,234,235,237,239,241,243,245,247,249,251,253};
/* The actual palette in use. */
int *spectrum_palette;
int spectrum_palette_size;
static redisContext *context;
static struct config {
@@ -88,10 +74,7 @@ static struct config {
int monitor_mode;
int pubsub_mode;
int latency_mode;
int latency_dist_mode;
int latency_history;
int lru_test_mode;
long long lru_test_sample_size;
int cluster_mode;
int cluster_reissue_command;
int slave_mode;
@@ -155,30 +138,6 @@ static void cliRefreshPrompt(void) {
snprintf(config.prompt+len,sizeof(config.prompt)-len,"> ");
}
static sds getHistoryPath() {
char *path = NULL;
sds historyPath = NULL;
/* check the env for a histfile override */
path = getenv(REDIS_CLI_HISTFILE_ENV);
if (path != NULL && *path != '\0') {
if (!strcmp("/dev/null", path)) {
return NULL;
}
/* if the env is set, return it */
historyPath = sdscatprintf(sdsempty(), "%s", path);
} else {
char *home = getenv("HOME");
if (home != NULL && *home != '\0') {
/* otherwise, return the default */
historyPath = sdscatprintf(sdsempty(), "%s/%s", home, REDIS_CLI_HISTFILE_DEFAULT);
}
}
return historyPath;
}
/*------------------------------------------------------------------------------
* Help functions
*--------------------------------------------------------------------------- */
@@ -343,7 +302,7 @@ static void completionCallback(const char *buf, linenoiseCompletions *lc) {
*--------------------------------------------------------------------------- */
/* Send AUTH command to the server */
static int cliAuth(void) {
static int cliAuth() {
redisReply *reply;
if (config.auth == NULL) return REDIS_OK;
@@ -356,7 +315,7 @@ static int cliAuth(void) {
}
/* Send SELECT dbnum to the server */
static int cliSelect(void) {
static int cliSelect() {
redisReply *reply;
if (config.dbnum == 0) return REDIS_OK;
@@ -534,7 +493,7 @@ static sds cliFormatReplyCSV(redisReply *r) {
out = sdscatrepr(out,r->str,r->len);
break;
case REDIS_REPLY_NIL:
out = sdscat(out,"NIL");
out = sdscat(out,"NIL\n");
break;
case REDIS_REPLY_ARRAY:
for (i = 0; i < r->elements; i++) {
@@ -713,17 +672,16 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
return REDIS_OK;
}
/* Send a command reconnecting the link if needed. */
static redisReply *reconnectingRedisCommand(redisContext *c, const char *fmt, ...) {
/* Send the INFO command, reconnecting the link if needed. */
static redisReply *reconnectingInfo(void) {
redisContext *c = context;
redisReply *reply = NULL;
int tries = 0;
va_list ap;
assert(!c->err);
while(reply == NULL) {
while (c->err & (REDIS_ERR_IO | REDIS_ERR_EOF)) {
printf("\r\x1b[0K"); /* Cursor to left edge + clear line. */
printf("Reconnecting... %d\r", ++tries);
printf("Reconnecting (%d)...\r", ++tries);
fflush(stdout);
redisFree(c);
@@ -731,15 +689,12 @@ static redisReply *reconnectingRedisCommand(redisContext *c, const char *fmt, ..
usleep(1000000);
}
va_start(ap,fmt);
reply = redisvCommand(c,fmt,ap);
va_end(ap);
reply = redisCommand(c,"INFO");
if (c->err && !(c->err & (REDIS_ERR_IO | REDIS_ERR_EOF))) {
fprintf(stderr, "Error: %s\n", c->errstr);
exit(1);
} else if (tries > 0) {
printf("\r\x1b[0K"); /* Cursor to left edge + clear line. */
printf("\n");
}
}
@@ -787,17 +742,9 @@ static int parseOptions(int argc, char **argv) {
config.output = OUTPUT_CSV;
} else if (!strcmp(argv[i],"--latency")) {
config.latency_mode = 1;
} else if (!strcmp(argv[i],"--latency-dist")) {
config.latency_dist_mode = 1;
} else if (!strcmp(argv[i],"--mono")) {
spectrum_palette = spectrum_palette_mono;
spectrum_palette_size = spectrum_palette_mono_size;
} else if (!strcmp(argv[i],"--latency-history")) {
config.latency_mode = 1;
config.latency_history = 1;
} else if (!strcmp(argv[i],"--lru-test") && !lastarg) {
config.lru_test_mode = 1;
config.lru_test_sample_size = strtoll(argv[++i],NULL,10);
} else if (!strcmp(argv[i],"--slave")) {
config.slave_mode = 1;
} else if (!strcmp(argv[i],"--stat")) {
@@ -883,13 +830,9 @@ static void usage(void) {
" not a tty).\n"
" --no-raw Force formatted output even when STDOUT is not a tty.\n"
" --csv Output in CSV format.\n"
" --stat Print rolling stats about server: mem, clients, ...\n"
" --latency Enter a special mode continuously sampling latency.\n"
" --latency-history Like --latency but tracking latency changes over time.\n"
" Default time interval is 15 sec. Change it using -i.\n"
" --latency-dist Shows latency as a spectrum, requires xterm 256 colors.\n"
" Default time interval is 1 sec. Change it using -i.\n"
" --lru-test <keys> Simulate a cache workload with an 80-20 distribution.\n"
" --slave Simulate a slave showing commands received from the master.\n"
" --rdb <filename> Transfer an RDB dump from remote server to local file.\n"
" --pipe Transfer raw Redis protocol from stdin to server.\n"
@@ -934,33 +877,6 @@ static char **convertToSds(int count, char** args) {
return sds;
}
static int issueCommandRepeat(int argc, char **argv, long repeat) {
while (1) {
config.cluster_reissue_command = 0;
if (cliSendCommand(argc,argv,repeat) != REDIS_OK) {
cliConnect(1);
/* If we still cannot send the command print error.
* We'll try to reconnect the next time. */
if (cliSendCommand(argc,argv,repeat) != REDIS_OK) {
cliPrintContextError();
return REDIS_ERR;
}
}
/* Issue the command again if we got redirected in cluster mode */
if (config.cluster_mode && config.cluster_reissue_command) {
cliConnect(1);
} else {
break;
}
}
return REDIS_OK;
}
static int issueCommand(int argc, char **argv) {
return issueCommandRepeat(argc, argv, config.repeat);
}
static void repl(void) {
sds historyfile = NULL;
int history = 0;
@@ -974,9 +890,10 @@ static void repl(void) {
/* Only use history when stdin is a tty. */
if (isatty(fileno(stdin))) {
historyfile = getHistoryPath();
if (historyfile != NULL) {
history = 1;
history = 1;
if (getenv("HOME") != NULL) {
historyfile = sdscatprintf(sdsempty(),"%s/.rediscli_history",getenv("HOME"));
linenoiseHistoryLoad(historyfile);
}
}
@@ -1016,8 +933,26 @@ static void repl(void) {
repeat = 1;
}
issueCommandRepeat(argc-skipargs, argv+skipargs, repeat);
while (1) {
config.cluster_reissue_command = 0;
if (cliSendCommand(argc-skipargs,argv+skipargs,repeat)
!= REDIS_OK)
{
cliConnect(1);
/* If we still cannot send the command print error.
* We'll try to reconnect the next time. */
if (cliSendCommand(argc-skipargs,argv+skipargs,repeat)
!= REDIS_OK)
cliPrintContextError();
}
/* Issue the command again if we got redirected in cluster mode */
if (config.cluster_mode && config.cluster_reissue_command) {
cliConnect(1);
} else {
break;
}
}
elapsed = mstime()-start_time;
if (elapsed >= 500) {
printf("(%.2fs)\n",(double)elapsed/1000);
@@ -1038,9 +973,10 @@ static int noninteractive(int argc, char **argv) {
if (config.stdinarg) {
argv = zrealloc(argv, (argc+1)*sizeof(char*));
argv[argc] = readArgFromStdin();
retval = issueCommand(argc+1, argv);
retval = cliSendCommand(argc+1, argv, config.repeat);
} else {
retval = issueCommand(argc, argv);
/* stdin is probably a tty, can be tested with S_ISCHR(s.st_mode) */
retval = cliSendCommand(argc, argv, config.repeat);
}
return retval;
}
@@ -1084,7 +1020,7 @@ static int evalMode(int argc, char **argv) {
argv2[2] = sdscatprintf(sdsempty(),"%d",keys);
/* Call it */
return issueCommand(argc+3-got_comma, argv2);
return cliSendCommand(argc+3-got_comma, argv2, config.repeat);
}
/*------------------------------------------------------------------------------
@@ -1105,7 +1041,7 @@ static void latencyMode(void) {
if (!context) exit(1);
while(1) {
start = mstime();
reply = reconnectingRedisCommand(context,"PING");
reply = redisCommand(context,"PING");
if (reply == NULL) {
fprintf(stderr,"\nI/O error\n");
exit(1);
@@ -1135,148 +1071,6 @@ static void latencyMode(void) {
}
}
/*------------------------------------------------------------------------------
* Latency distribution mode -- requires 256 colors xterm
*--------------------------------------------------------------------------- */
#define LATENCY_DIST_DEFAULT_INTERVAL 1000 /* milliseconds. */
/* Structure to store samples distribution. */
struct distsamples {
long long max; /* Max latency to fit into this interval (usec). */
long long count; /* Number of samples in this interval. */
int character; /* Associated character in visualization. */
};
/* Helper function for latencyDistMode(). Performs the spectrum visualization
* of the collected samples targeting an xterm 256 terminal.
*
* Takes an array of distsamples structures, ordered from smaller to bigger
* 'max' value. Last sample max must be 0, to mean that it olds all the
* samples greater than the previous one, and is also the stop sentinel.
*
* "tot' is the total number of samples in the different buckets, so it
* is the SUM(samples[i].conut) for i to 0 up to the max sample.
*
* As a side effect the function sets all the buckets count to 0. */
void showLatencyDistSamples(struct distsamples *samples, long long tot) {
int j;
/* We convert samples into a index inside the palette
* proportional to the percentage a given bucket represents.
* This way intensity of the different parts of the spectrum
* don't change relative to the number of requests, which avoids to
* pollute the visualization with non-latency related info. */
printf("\033[38;5;0m"); /* Set foreground color to black. */
for (j = 0; ; j++) {
int coloridx =
ceil((float) samples[j].count / tot * (spectrum_palette_size-1));
int color = spectrum_palette[coloridx];
printf("\033[48;5;%dm%c", (int)color, samples[j].character);
samples[j].count = 0;
if (samples[j].max == 0) break; /* Last sample. */
}
printf("\033[0m\n");
fflush(stdout);
}
/* Show the legend: different buckets values and colors meaning, so
* that the spectrum is more easily readable. */
void showLatencyDistLegend(void) {
int j;
printf("---------------------------------------------\n");
printf(". - * # .01 .125 .25 .5 milliseconds\n");
printf("1,2,3,...,9 from 1 to 9 milliseconds\n");
printf("A,B,C,D,E 10,20,30,40,50 milliseconds\n");
printf("F,G,H,I,J .1,.2,.3,.4,.5 seconds\n");
printf("K,L,M,N,O,P,Q,? 1,2,4,8,16,30,60,>60 seconds\n");
printf("From 0 to 100%%: ");
for (j = 0; j < spectrum_palette_size; j++) {
printf("\033[48;5;%dm ", spectrum_palette[j]);
}
printf("\033[0m\n");
printf("---------------------------------------------\n");
}
static void latencyDistMode(void) {
redisReply *reply;
long long start, latency, count = 0;
long long history_interval =
config.interval ? config.interval/1000 :
LATENCY_DIST_DEFAULT_INTERVAL;
long long history_start = ustime();
int j, outputs = 0;
struct distsamples samples[] = {
/* We use a mostly logarithmic scale, with certain linear intervals
* which are more interesting than others, like 1-10 milliseconds
* range. */
{10,0,'.'}, /* 0.01 ms */
{125,0,'-'}, /* 0.125 ms */
{250,0,'*'}, /* 0.25 ms */
{500,0,'#'}, /* 0.5 ms */
{1000,0,'1'}, /* 1 ms */
{2000,0,'2'}, /* 2 ms */
{3000,0,'3'}, /* 3 ms */
{4000,0,'4'}, /* 4 ms */
{5000,0,'5'}, /* 5 ms */
{6000,0,'6'}, /* 6 ms */
{7000,0,'7'}, /* 7 ms */
{8000,0,'8'}, /* 8 ms */
{9000,0,'9'}, /* 9 ms */
{10000,0,'A'}, /* 10 ms */
{20000,0,'B'}, /* 20 ms */
{30000,0,'C'}, /* 30 ms */
{40000,0,'D'}, /* 40 ms */
{50000,0,'E'}, /* 50 ms */
{100000,0,'F'}, /* 0.1 s */
{200000,0,'G'}, /* 0.2 s */
{300000,0,'H'}, /* 0.3 s */
{400000,0,'I'}, /* 0.4 s */
{500000,0,'J'}, /* 0.5 s */
{1000000,0,'K'}, /* 1 s */
{2000000,0,'L'}, /* 2 s */
{4000000,0,'M'}, /* 4 s */
{8000000,0,'N'}, /* 8 s */
{16000000,0,'O'}, /* 16 s */
{30000000,0,'P'}, /* 30 s */
{60000000,0,'Q'}, /* 1 minute */
{0,0,'?'}, /* > 1 minute */
};
if (!context) exit(1);
while(1) {
start = ustime();
reply = reconnectingRedisCommand(context,"PING");
if (reply == NULL) {
fprintf(stderr,"\nI/O error\n");
exit(1);
}
latency = ustime()-start;
freeReplyObject(reply);
count++;
/* Populate the relevant bucket. */
for (j = 0; ; j++) {
if (samples[j].max == 0 || latency <= samples[j].max) {
samples[j].count++;
break;
}
}
/* From time to time show the spectrum. */
if (count && (ustime()-history_start)/1000 > history_interval) {
if ((outputs++ % 20) == 0)
showLatencyDistLegend();
showLatencyDistSamples(samples,count);
history_start = ustime();
count = 0;
}
usleep(LATENCY_SAMPLE_RATE * 1000);
}
}
/*------------------------------------------------------------------------------
* Slave mode
*--------------------------------------------------------------------------- */
@@ -1896,7 +1690,7 @@ static void statMode(void) {
char buf[64];
int j;
reply = reconnectingRedisCommand(context,"INFO");
reply = reconnectingInfo();
if (reply->type == REDIS_REPLY_ERROR) {
printf("ERROR: %s\n", reply->str);
exit(1);
@@ -1950,7 +1744,6 @@ static void statMode(void) {
/* Children */
aux = getLongInfoField(reply->str,"bgsave_in_progress");
aux |= getLongInfoField(reply->str,"aof_rewrite_in_progress") << 1;
aux |= getLongInfoField(reply->str,"loading") << 2;
switch(aux) {
case 0: break;
case 1:
@@ -1962,9 +1755,6 @@ static void statMode(void) {
case 3:
printf("SAVE+AOF");
break;
case 4:
printf("LOAD");
break;
}
printf("\n");
@@ -2006,94 +1796,6 @@ static void scanMode(void) {
exit(0);
}
/*------------------------------------------------------------------------------
* LRU test mode
*--------------------------------------------------------------------------- */
/* Return an integer from min to max (both inclusive) using a power-law
* distribution, depending on the value of alpha: the greater the alpha
* the more bias towards lower values.
*
* With alpha = 6.2 the output follows the 80-20 rule where 20% of
* the returned numbers will account for 80% of the frequency. */
long long powerLawRand(long long min, long long max, double alpha) {
double pl, r;
max += 1;
r = ((double)rand()) / RAND_MAX;
pl = pow(
((pow(max,alpha+1) - pow(min,alpha+1))*r + pow(min,alpha+1)),
(1.0/(alpha+1)));
return (max-1-(long long)pl)+min;
}
/* Generates a key name among a set of lru_test_sample_size keys, using
* an 80-20 distribution. */
void LRUTestGenKey(char *buf, size_t buflen) {
snprintf(buf, buflen, "lru:%lld\n",
powerLawRand(1, config.lru_test_sample_size, 6.2));
}
#define LRU_CYCLE_PERIOD 1000 /* 1000 milliseconds. */
#define LRU_CYCLE_PIPELINE_SIZE 250
static void LRUTestMode(void) {
redisReply *reply;
char key[128];
long long start_cycle;
int j;
srand(time(NULL)^getpid());
while(1) {
/* Perform cycles of 1 second with 50% writes and 50% reads.
* We use pipelining batching writes / reads N times per cycle in order
* to fill the target instance easily. */
start_cycle = mstime();
long long hits = 0, misses = 0;
while(mstime() - start_cycle < 1000) {
/* Write cycle. */
for (j = 0; j < LRU_CYCLE_PIPELINE_SIZE; j++) {
LRUTestGenKey(key,sizeof(key));
redisAppendCommand(context, "SET %s val",key);
}
for (j = 0; j < LRU_CYCLE_PIPELINE_SIZE; j++)
redisGetReply(context, (void**)&reply);
/* Read cycle. */
for (j = 0; j < LRU_CYCLE_PIPELINE_SIZE; j++) {
LRUTestGenKey(key,sizeof(key));
redisAppendCommand(context, "GET %s",key);
}
for (j = 0; j < LRU_CYCLE_PIPELINE_SIZE; j++) {
if (redisGetReply(context, (void**)&reply) == REDIS_OK) {
switch(reply->type) {
case REDIS_REPLY_ERROR:
printf("%s\n", reply->str);
break;
case REDIS_REPLY_NIL:
misses++;
break;
default:
hits++;
break;
}
}
}
if (context->err) {
fprintf(stderr,"I/O error during LRU test\n");
exit(1);
}
}
/* Print stats. */
printf(
"%lld Gets/sec | Hits: %lld (%.2f%%) | Misses: %lld (%.2f%%)\n",
hits+misses,
hits, (double)hits/(hits+misses)*100,
misses, (double)misses/(hits+misses)*100);
}
exit(0);
}
/*------------------------------------------------------------------------------
* Intrisic latency mode.
*
@@ -2185,10 +1887,7 @@ int main(int argc, char **argv) {
config.monitor_mode = 0;
config.pubsub_mode = 0;
config.latency_mode = 0;
config.latency_dist_mode = 0;
config.latency_history = 0;
config.lru_test_mode = 0;
config.lru_test_sample_size = 0;
config.cluster_mode = 0;
config.slave_mode = 0;
config.getrdb_mode = 0;
@@ -2205,9 +1904,6 @@ int main(int argc, char **argv) {
config.eval = NULL;
config.last_cmd_type = -1;
spectrum_palette = spectrum_palette_color;
spectrum_palette_size = spectrum_palette_color_size;
if (!isatty(fileno(stdout)) && (getenv("FAKETTY") == NULL))
config.output = OUTPUT_RAW;
else
@@ -2219,18 +1915,14 @@ int main(int argc, char **argv) {
argc -= firstarg;
argv += firstarg;
signal(SIGPIPE, SIG_IGN);
/* Latency mode */
if (config.latency_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
latencyMode();
}
/* Latency distribution mode */
if (config.latency_dist_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
latencyDistMode();
}
/* Slave mode */
if (config.slave_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
@@ -2268,20 +1960,11 @@ int main(int argc, char **argv) {
scanMode();
}
/* LRU test mode */
if (config.lru_test_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
LRUTestMode();
}
/* Intrinsic latency mode */
if (config.intrinsic_latency_mode) intrinsicLatencyMode();
/* Start interactive mode when no command is provided */
if (argc == 0 && !config.eval) {
/* Ignore SIGPIPE in interactive mode to force a reconnect */
signal(SIGPIPE, SIG_IGN);
/* Note that in repl mode we don't abort on connection error.
* A new attempt will be performed for every command send. */
cliConnect(0);
+32 -58
View File
@@ -72,7 +72,7 @@ class ClusterNode
@friends
end
def slots
def slots
@info[:slots]
end
@@ -154,7 +154,7 @@ class ClusterNode
end
} if slots
@dirty = false
@r.cluster("info").split("\n").each{|e|
@r.cluster("info").split("\n").each{|e|
k,v=e.split(":")
k = k.to_sym
v.chop!
@@ -213,7 +213,7 @@ class ClusterNode
#
# Note: this could be easily written without side effects,
# we use 'slots' just to split the computation into steps.
# First step: we want an increasing array of integers
# for instance: [1,2,3,4,5,8,9,20,21,22,23,24,25,30]
slots = @info[:slots].keys.sort
@@ -273,7 +273,7 @@ class ClusterNode
def info
@info
end
def is_dirty?
@dirty
end
@@ -540,6 +540,7 @@ class RedisTrib
nodes_count = @nodes.length
masters_count = @nodes.length / (@replicas+1)
masters = []
slaves = []
# The first step is to split instances by IP. This is useful as
# we'll try to allocate master nodes in different physical machines
@@ -557,31 +558,16 @@ class RedisTrib
# Select master instances
puts "Using #{masters_count} masters:"
interleaved = []
stop = false
while not stop do
# Take one node from each IP until we run out of nodes
# across every IP.
ips.each do |ip,nodes|
if nodes.empty?
# if this IP has no remaining nodes, check for termination
if interleaved.length == nodes_count
# stop when 'interleaved' has accumulated all nodes
stop = true
next
end
else
# else, move one node from this IP to 'interleaved'
interleaved.push nodes.shift
end
end
while masters.length < masters_count
ips.each{|ip,nodes_list|
next if nodes_list.length == 0
masters << nodes_list.shift
puts masters[-1]
nodes_count -= 1
break if masters.length == masters_count
}
end
masters = interleaved.slice!(0, masters_count)
nodes_count -= masters.length
masters.each{|m| puts m}
# Alloc slots on masters
slots_per_node = ClusterHashSlots.to_f / masters_count
first = 0
@@ -608,8 +594,8 @@ class RedisTrib
# all nodes will be used.
assignment_verbose = false
[:requested,:unused].each do |assign|
masters.each do |m|
[:requested,:unused].each{|assign|
masters.each{|m|
assigned_replicas = 0
while assigned_replicas < @replicas
break if nodes_count == 0
@@ -623,33 +609,21 @@ class RedisTrib
"role too (#{nodes_count} remaining)."
end
end
# Return the first node not matching our current master
node = interleaved.find{|n| n.info[:host] != m.info[:host]}
# If we found a node, use it as a best-first match.
# Otherwise, we didn't find a node on a different IP, so we
# go ahead and use a same-IP replica.
if node
slave = node
interleaved.delete node
else
slave = interleaved.shift
end
slave.set_as_replica(m.info[:name])
nodes_count -= 1
assigned_replicas += 1
puts "Adding replica #{slave} to #{m}"
# If we are in the "assign extra nodes" loop,
# we want to assign one extra replica to each
# master before repeating masters.
# This break lets us assign extra replicas to masters
# in a round-robin way.
break if assign == :unused
ips.each{|ip,nodes_list|
next if nodes_list.length == 0
# Skip instances with the same IP as the master if we
# have some more IPs available.
next if ip == m.info[:host] && nodes_count > nodes_list.length
slave = nodes_list.shift
slave.set_as_replica(m.info[:name])
nodes_count -= 1
assigned_replicas += 1
puts "Adding replica #{slave} to #{m}"
break
}
end
end
end
}
}
end
def flush_nodes_config
@@ -789,7 +763,7 @@ class RedisTrib
# Move slots between source and target nodes using MIGRATE.
#
# Options:
# Options:
# :verbose -- Print a dot for every moved key.
# :fix -- We are moving in the context of a fix. Use REPLACE.
# :cold -- Move keys without opening / reconfiguring the nodes.
@@ -1165,7 +1139,7 @@ class RedisTrib
# right node as needed.
cursor = nil
while cursor != 0
cursor,keys = source.scan(cursor, :count => 1000)
cursor,keys = source.scan(cursor,:count,1000)
cursor = cursor.to_i
keys.each{|k|
# Migrate keys using the MIGRATE command.
@@ -1232,7 +1206,7 @@ end
#################################################################################
# Libraries
#
#
# We try to don't depend on external libs since this is a critical part
# of Redis Cluster.
#################################################################################
+159 -154
View File
@@ -129,7 +129,7 @@ struct redisCommand redisCommandTable[] = {
{"append",appendCommand,3,"wm",0,NULL,1,1,1,0,0},
{"strlen",strlenCommand,2,"rF",0,NULL,1,1,1,0,0},
{"del",delCommand,-2,"w",0,NULL,1,-1,1,0,0},
{"exists",existsCommand,-2,"rF",0,NULL,1,-1,1,0,0},
{"exists",existsCommand,2,"rF",0,NULL,1,1,1,0,0},
{"setbit",setbitCommand,4,"wm",0,NULL,1,1,1,0,0},
{"getbit",getbitCommand,3,"rF",0,NULL,1,1,1,0,0},
{"setrange",setrangeCommand,4,"wm",0,NULL,1,1,1,0,0},
@@ -247,7 +247,7 @@ struct redisCommand redisCommandTable[] = {
{"pttl",pttlCommand,2,"rF",0,NULL,1,1,1,0,0},
{"persist",persistCommand,2,"wF",0,NULL,1,1,1,0,0},
{"slaveof",slaveofCommand,3,"ast",0,NULL,0,0,0,0,0},
{"role",roleCommand,1,"lst",0,NULL,0,0,0,0,0},
{"role",roleCommand,1,"last",0,NULL,0,0,0,0,0},
{"debug",debugCommand,-2,"as",0,NULL,0,0,0,0,0},
{"config",configCommand,-2,"art",0,NULL,0,0,0,0,0},
{"subscribe",subscribeCommand,-2,"rpslt",0,NULL,0,0,0,0,0},
@@ -259,19 +259,19 @@ struct redisCommand redisCommandTable[] = {
{"watch",watchCommand,-2,"rsF",0,NULL,1,-1,1,0,0},
{"unwatch",unwatchCommand,1,"rsF",0,NULL,0,0,0,0,0},
{"cluster",clusterCommand,-2,"ar",0,NULL,0,0,0,0,0},
{"restore",restoreCommand,-4,"wm",0,NULL,1,1,1,0,0},
{"restore-asking",restoreCommand,-4,"wmk",0,NULL,1,1,1,0,0},
{"migrate",migrateCommand,-6,"w",0,NULL,0,0,0,0,0},
{"restore",restoreCommand,-4,"awm",0,NULL,1,1,1,0,0},
{"restore-asking",restoreCommand,-4,"awmk",0,NULL,1,1,1,0,0},
{"migrate",migrateCommand,-6,"aw",0,NULL,0,0,0,0,0},
{"asking",askingCommand,1,"r",0,NULL,0,0,0,0,0},
{"readonly",readonlyCommand,1,"rF",0,NULL,0,0,0,0,0},
{"readwrite",readwriteCommand,1,"rF",0,NULL,0,0,0,0,0},
{"dump",dumpCommand,2,"r",0,NULL,1,1,1,0,0},
{"dump",dumpCommand,2,"ar",0,NULL,1,1,1,0,0},
{"object",objectCommand,3,"r",0,NULL,2,2,2,0,0},
{"client",clientCommand,-2,"rs",0,NULL,0,0,0,0,0},
{"client",clientCommand,-2,"ars",0,NULL,0,0,0,0,0},
{"eval",evalCommand,-3,"s",0,evalGetKeys,0,0,0,0,0},
{"evalsha",evalShaCommand,-3,"s",0,evalGetKeys,0,0,0,0,0},
{"slowlog",slowlogCommand,-2,"r",0,NULL,0,0,0,0,0},
{"script",scriptCommand,-2,"rs",0,NULL,0,0,0,0,0},
{"script",scriptCommand,-2,"ras",0,NULL,0,0,0,0,0},
{"time",timeCommand,1,"rRF",0,NULL,0,0,0,0,0},
{"bitop",bitopCommand,-4,"wm",0,NULL,2,-1,1,0,0},
{"bitcount",bitcountCommand,-2,"r",0,NULL,1,1,1,0,0},
@@ -280,7 +280,7 @@ struct redisCommand redisCommandTable[] = {
{"command",commandCommand,0,"rlt",0,NULL,0,0,0,0,0},
{"pfselftest",pfselftestCommand,1,"r",0,NULL,0,0,0,0,0},
{"pfadd",pfaddCommand,-2,"wmF",0,NULL,1,1,1,0,0},
{"pfcount",pfcountCommand,-2,"r",0,NULL,1,-1,1,0,0},
{"pfcount",pfcountCommand,-2,"w",0,NULL,1,1,1,0,0},
{"pfmerge",pfmergeCommand,-2,"wm",0,NULL,1,-1,1,0,0},
{"pfdebug",pfdebugCommand,-3,"w",0,NULL,0,0,0,0,0},
{"latency",latencyCommand,-2,"arslt",0,NULL,0,0,0,0,0}
@@ -876,38 +876,32 @@ unsigned int getLRUClock(void) {
}
/* Add a sample to the operations per second array of samples. */
void trackInstantaneousMetric(int metric, long long current_reading) {
long long t = mstime() - server.inst_metric[metric].last_sample_time;
long long ops = current_reading -
server.inst_metric[metric].last_sample_count;
void trackOperationsPerSecond(void) {
long long t = mstime() - server.ops_sec_last_sample_time;
long long ops = server.stat_numcommands - server.ops_sec_last_sample_ops;
long long ops_sec;
ops_sec = t > 0 ? (ops*1000/t) : 0;
server.inst_metric[metric].samples[server.inst_metric[metric].idx] =
ops_sec;
server.inst_metric[metric].idx++;
server.inst_metric[metric].idx %= REDIS_METRIC_SAMPLES;
server.inst_metric[metric].last_sample_time = mstime();
server.inst_metric[metric].last_sample_count = current_reading;
server.ops_sec_samples[server.ops_sec_idx] = ops_sec;
server.ops_sec_idx = (server.ops_sec_idx+1) % REDIS_OPS_SEC_SAMPLES;
server.ops_sec_last_sample_time = mstime();
server.ops_sec_last_sample_ops = server.stat_numcommands;
}
/* Return the mean of all the samples. */
long long getInstantaneousMetric(int metric) {
long long getOperationsPerSecond(void) {
int j;
long long sum = 0;
for (j = 0; j < REDIS_METRIC_SAMPLES; j++)
sum += server.inst_metric[metric].samples[j];
return sum / REDIS_METRIC_SAMPLES;
for (j = 0; j < REDIS_OPS_SEC_SAMPLES; j++)
sum += server.ops_sec_samples[j];
return sum / REDIS_OPS_SEC_SAMPLES;
}
/* Check for timeouts. Returns non-zero if the client was terminated.
* The function gets the current time in milliseconds as argument since
* it gets called multiple times in a loop, so calling gettimeofday() for
* each iteration would be costly without any actual gain. */
int clientsCronHandleTimeout(redisClient *c, mstime_t now_ms) {
time_t now = now_ms/1000;
/* Check for timeouts. Returns non-zero if the client was terminated */
int clientsCronHandleTimeout(redisClient *c) {
time_t now = server.unixtime;
if (server.maxidletime &&
!(c->flags & REDIS_SLAVE) && /* no timeout for slaves */
@@ -923,16 +917,11 @@ int clientsCronHandleTimeout(redisClient *c, mstime_t now_ms) {
/* Blocked OPS timeout is handled with milliseconds resolution.
* However note that the actual resolution is limited by
* server.hz. */
mstime_t now_ms = mstime();
if (c->bpop.timeout != 0 && c->bpop.timeout < now_ms) {
/* Handle blocking operation specific timeout. */
replyToBlockedClientTimedOut(c);
unblockClient(c);
} else if (server.cluster_enabled) {
/* Cluster: handle unblock & redirect of clients blocked
* into keys no longer served by this server. */
if (clusterRedirectBlockedClientIfNeeded(c))
unblockClient(c);
}
}
return 0;
@@ -964,23 +953,17 @@ int clientsCronResizeQueryBuffer(redisClient *c) {
return 0;
}
#define CLIENTS_CRON_MIN_ITERATIONS 5
void clientsCron(void) {
/* Make sure to process at least numclients/server.hz of clients
* per call. Since this function is called server.hz times per second
* we are sure that in the worst case we process all the clients in 1
* second. */
/* Make sure to process at least 1/(server.hz*10) of clients per call.
* Since this function is called server.hz times per second we are sure that
* in the worst case we process all the clients in 10 seconds.
* In normal conditions (a reasonable number of clients) we process
* all the clients in a shorter time. */
int numclients = listLength(server.clients);
int iterations = numclients/server.hz;
mstime_t now = mstime();
/* Process at least a few clients while we are at it, even if we need
* to process less than CLIENTS_CRON_MIN_ITERATIONS to meet our contract
* of processing each client once per second. */
if (iterations < CLIENTS_CRON_MIN_ITERATIONS)
iterations = (numclients < CLIENTS_CRON_MIN_ITERATIONS) ?
numclients : CLIENTS_CRON_MIN_ITERATIONS;
int iterations = numclients/(server.hz*10);
if (iterations < 50)
iterations = (numclients < 50) ? numclients : 50;
while(listLength(server.clients) && iterations--) {
redisClient *c;
listNode *head;
@@ -994,7 +977,7 @@ void clientsCron(void) {
/* The following functions do different service checks on the client.
* The protocol is that they return non-zero if the client was
* terminated. */
if (clientsCronHandleTimeout(c,now)) continue;
if (clientsCronHandleTimeout(c)) continue;
if (clientsCronResizeQueryBuffer(c)) continue;
}
}
@@ -1085,13 +1068,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
/* Update the time cache. */
updateCachedTime();
run_with_period(100) {
trackInstantaneousMetric(REDIS_METRIC_COMMAND,server.stat_numcommands);
trackInstantaneousMetric(REDIS_METRIC_NET_INPUT,
server.stat_net_input_bytes);
trackInstantaneousMetric(REDIS_METRIC_NET_OUTPUT,
server.stat_net_output_bytes);
}
run_with_period(100) trackOperationsPerSecond();
/* We have just REDIS_LRU_BITS bits per object for LRU information.
* So we use an (eventually wrapping) LRU clock.
@@ -1271,12 +1248,6 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
void beforeSleep(struct aeEventLoop *eventLoop) {
REDIS_NOTUSED(eventLoop);
/* Call the Redis Cluster before sleep function. Note that this function
* may change the state of Redis Cluster (from ok to fail or vice versa),
* so it's a good idea to call it before serving the unblocked clients
* later in this function. */
if (server.cluster_enabled) clusterBeforeSleep();
/* Run a fast expire cycle (the called function will return
* ASAP if a fast cycle is not needed). */
if (server.active_expire_enabled && server.masterhost == NULL)
@@ -1308,6 +1279,9 @@ void beforeSleep(struct aeEventLoop *eventLoop) {
/* Write the AOF buffer on disk */
flushAppendOnlyFile(0);
/* Call the Redis Cluster before sleep function. */
if (server.cluster_enabled) clusterBeforeSleep();
}
/* =========================== Server initialization ======================== */
@@ -1459,8 +1433,11 @@ void initServerConfig(void) {
server.maxclients = REDIS_MAX_CLIENTS;
server.bpop_blocked_clients = 0;
server.maxmemory = REDIS_DEFAULT_MAXMEMORY;
server.maxmemory_frag_guess = REDIS_DEFAULT_MAXMEMORY_FRAG_GUESS;
server.maxmemory_enforced = (double) REDIS_DEFAULT_MAXMEMORY / server.maxmemory_frag_guess;
server.maxmemory_policy = REDIS_DEFAULT_MAXMEMORY_POLICY;
server.maxmemory_samples = REDIS_DEFAULT_MAXMEMORY_SAMPLES;
server.rss_aware_maxmemory = REDIS_DEFAULT_RSS_AWARE_MAXMEMORY;
server.hash_max_ziplist_entries = REDIS_HASH_MAX_ZIPLIST_ENTRIES;
server.hash_max_ziplist_value = REDIS_HASH_MAX_ZIPLIST_VALUE;
server.list_max_ziplist_entries = REDIS_LIST_MAX_ZIPLIST_ENTRIES;
@@ -1580,33 +1557,33 @@ void adjustOpenFilesLimit(void) {
/* Set the max number of files if the current limit is not enough
* for our needs. */
if (oldlimit < maxfiles) {
rlim_t bestlimit;
rlim_t f;
int setrlimit_error = 0;
/* Try to set the file limit to match 'maxfiles' or at least
* to the higher value supported less than maxfiles. */
bestlimit = maxfiles;
while(bestlimit > oldlimit) {
f = maxfiles;
while(f > oldlimit) {
rlim_t decr_step = 16;
limit.rlim_cur = bestlimit;
limit.rlim_max = bestlimit;
limit.rlim_cur = f;
limit.rlim_max = f;
if (setrlimit(RLIMIT_NOFILE,&limit) != -1) break;
setrlimit_error = errno;
/* We failed to set file limit to 'bestlimit'. Try with a
/* We failed to set file limit to 'f'. Try with a
* smaller limit decrementing by a few FDs per iteration. */
if (bestlimit < decr_step) break;
bestlimit -= decr_step;
if (f < decr_step) break;
f -= decr_step;
}
/* Assume that the limit we get initially is still valid if
* our last try was even lower. */
if (bestlimit < oldlimit) bestlimit = oldlimit;
if (f < oldlimit) f = oldlimit;
if (bestlimit < maxfiles) {
if (f != maxfiles) {
int old_maxclients = server.maxclients;
server.maxclients = bestlimit-REDIS_MIN_RESERVED_FDS;
server.maxclients = f-REDIS_MIN_RESERVED_FDS;
if (server.maxclients < 1) {
redisLog(REDIS_WARNING,"Your current 'ulimit -n' "
"of %llu is not enough for Redis to start. "
@@ -1627,7 +1604,7 @@ void adjustOpenFilesLimit(void) {
"maxclients has been reduced to %d to compensate for "
"low ulimit. "
"If you need higher maxclients increase 'ulimit -n'.",
(unsigned long long) bestlimit, server.maxclients);
(unsigned long long) oldlimit, server.maxclients);
} else {
redisLog(REDIS_NOTICE,"Increased maximum number of open files "
"to %llu (it was originally set to %llu).",
@@ -1638,23 +1615,6 @@ void adjustOpenFilesLimit(void) {
}
}
/* Check that server.tcp_backlog can be actually enforced in Linux according
* to the value of /proc/sys/net/core/somaxconn, or warn about it. */
void checkTcpBacklogSettings(void) {
#ifdef HAVE_PROC_SOMAXCONN
FILE *fp = fopen("/proc/sys/net/core/somaxconn","r");
char buf[1024];
if (!fp) return;
if (fgets(buf,sizeof(buf),fp) != NULL) {
int somaxconn = atoi(buf);
if (somaxconn > 0 && somaxconn < server.tcp_backlog) {
redisLog(REDIS_WARNING,"WARNING: The TCP backlog setting of %d cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of %d.", server.tcp_backlog, somaxconn);
}
}
fclose(fp);
#endif
}
/* Initialize a set of file descriptors to listen to the specified 'port'
* binding the addresses specified in the Redis server configuration.
*
@@ -1725,8 +1685,6 @@ int listenToPort(int port, int *fds, int *count) {
* to reset via CONFIG RESETSTAT. The function is also used in order to
* initialize these fields in initServer() at server startup. */
void resetServerStats(void) {
int j;
server.stat_numcommands = 0;
server.stat_numconnections = 0;
server.stat_expiredkeys = 0;
@@ -1739,16 +1697,10 @@ void resetServerStats(void) {
server.stat_sync_full = 0;
server.stat_sync_partial_ok = 0;
server.stat_sync_partial_err = 0;
for (j = 0; j < REDIS_METRIC_COUNT; j++) {
server.inst_metric[j].idx = 0;
server.inst_metric[j].last_sample_time = mstime();
server.inst_metric[j].last_sample_count = 0;
memset(server.inst_metric[j].samples,0,
sizeof(server.inst_metric[j].samples));
}
server.stat_net_input_bytes = 0;
server.stat_net_output_bytes = 0;
server.aof_delayed_fsync = 0;
memset(server.ops_sec_samples,0,sizeof(server.ops_sec_samples));
server.ops_sec_idx = 0;
server.ops_sec_last_sample_time = mstime();
server.ops_sec_last_sample_ops = 0;
}
void initServer(void) {
@@ -1792,7 +1744,7 @@ void initServer(void) {
server.sofd = anetUnixServer(server.neterr,server.unixsocket,
server.unixsocketperm, server.tcp_backlog);
if (server.sofd == ANET_ERR) {
redisLog(REDIS_WARNING, "Opening Unix socket: %s", server.neterr);
redisLog(REDIS_WARNING, "Opening socket: %s", server.neterr);
exit(1);
}
anetNonBlock(NULL,server.sofd);
@@ -2049,7 +2001,7 @@ void call(redisClient *c, int flags) {
* not generated from reading an AOF. */
if (listLength(server.monitors) &&
!server.loading &&
!(c->cmd->flags & (REDIS_CMD_SKIP_MONITOR|REDIS_CMD_ADMIN)))
!(c->cmd->flags & REDIS_CMD_SKIP_MONITOR))
{
replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
}
@@ -2173,22 +2125,36 @@ int processCommand(redisClient *c) {
* 2) The command has no key arguments. */
if (server.cluster_enabled &&
!(c->flags & REDIS_MASTER) &&
!(c->flags & REDIS_LUA_CLIENT &&
server.lua_caller->flags & REDIS_MASTER) &&
!(c->cmd->getkeys_proc == NULL && c->cmd->firstkey == 0))
{
int hashslot;
if (server.cluster->state != REDIS_CLUSTER_OK) {
flagTransaction(c);
clusterRedirectClient(c,NULL,0,REDIS_CLUSTER_REDIR_DOWN_STATE);
addReplySds(c,sdsnew("-CLUSTERDOWN The cluster is down. Use CLUSTER INFO for more information\r\n"));
return REDIS_OK;
} else {
int error_code;
clusterNode *n = getNodeByQuery(c,c->cmd,c->argv,c->argc,&hashslot,&error_code);
if (n == NULL || n != server.cluster->myself) {
if (n == NULL) {
flagTransaction(c);
clusterRedirectClient(c,n,hashslot,error_code);
if (error_code == REDIS_CLUSTER_REDIR_CROSS_SLOT) {
addReplySds(c,sdsnew("-CROSSSLOT Keys in request don't hash to the same slot\r\n"));
} else if (error_code == REDIS_CLUSTER_REDIR_UNSTABLE) {
/* The request spawns mutliple keys in the same slot,
* but the slot is not "stable" currently as there is
* a migration or import in progress. */
addReplySds(c,sdsnew("-TRYAGAIN Multiple keys request during rehashing of slot\r\n"));
} else {
redisPanic("getNodeByQuery() unknown error.");
}
return REDIS_OK;
} else if (n != server.cluster->myself) {
flagTransaction(c);
addReplySds(c,sdscatprintf(sdsempty(),
"-%s %d %s:%d\r\n",
(error_code == REDIS_CLUSTER_REDIR_ASK) ? "ASK" : "MOVED",
hashslot,n->ip,n->port));
return REDIS_OK;
}
}
@@ -2794,14 +2760,14 @@ sds genRedisInfoString(char *section) {
server.loading_loaded_bytes;
perc = ((double)server.loading_loaded_bytes /
(server.loading_total_bytes+1)) * 100;
server.loading_total_bytes) * 100;
elapsed = time(NULL)-server.loading_start_time;
elapsed = server.unixtime-server.loading_start_time;
if (elapsed == 0) {
eta = 1; /* A fake 1 second figure if we don't have
enough info */
} else {
eta = (elapsed*remaining_bytes)/(server.loading_loaded_bytes+1);
eta = (elapsed*remaining_bytes)/server.loading_loaded_bytes;
}
info = sdscatprintf(info,
@@ -2827,10 +2793,6 @@ sds genRedisInfoString(char *section) {
"total_connections_received:%lld\r\n"
"total_commands_processed:%lld\r\n"
"instantaneous_ops_per_sec:%lld\r\n"
"total_net_input_bytes:%lld\r\n"
"total_net_output_bytes:%lld\r\n"
"instantaneous_input_kbps:%.2f\r\n"
"instantaneous_output_kbps:%.2f\r\n"
"rejected_connections:%lld\r\n"
"sync_full:%lld\r\n"
"sync_partial_ok:%lld\r\n"
@@ -2845,11 +2807,7 @@ sds genRedisInfoString(char *section) {
"migrate_cached_sockets:%ld\r\n",
server.stat_numconnections,
server.stat_numcommands,
getInstantaneousMetric(REDIS_METRIC_COMMAND),
server.stat_net_input_bytes,
server.stat_net_output_bytes,
(float)getInstantaneousMetric(REDIS_METRIC_NET_INPUT)/1024,
(float)getInstantaneousMetric(REDIS_METRIC_NET_OUTPUT)/1024,
getOperationsPerSecond(),
server.stat_rejected_conn,
server.stat_sync_full,
server.stat_sync_partial_ok,
@@ -3140,7 +3098,13 @@ void evictionPoolPopulate(dict *sampledict, dict *keydict, struct evictionPoolEn
samples = zmalloc(sizeof(samples[0])*server.maxmemory_samples);
}
count = dictGetSomeKeys(sampledict,samples,server.maxmemory_samples);
#if 1 /* Use bulk get by default. */
count = dictGetRandomKeys(sampledict,samples,server.maxmemory_samples);
#else
count = server.maxmemory_samples;
for (j = 0; j < count; j++) samples[j] = dictGetRandomKey(sampledict);
#endif
for (j = 0; j < count; j++) {
unsigned long long idle;
sds key;
@@ -3193,9 +3157,9 @@ void evictionPoolPopulate(dict *sampledict, dict *keydict, struct evictionPoolEn
}
int freeMemoryIfNeeded(void) {
size_t mem_used, mem_tofree, mem_freed;
size_t mem_used, mem_tofree, mem_freed, mem_target = server.maxmemory;
int slaves = listLength(server.slaves);
mstime_t latency, eviction_latency;
mstime_t latency;
/* Remove the size of slaves output buffers and AOF buffer from the
* count of used memory. */
@@ -3219,14 +3183,72 @@ int freeMemoryIfNeeded(void) {
mem_used -= aofRewriteBufferSize();
}
/* If we use RSS aware maxmemory, update the target memory using
* the current fragmentation figure. */
#ifdef HAVE_RSS_REPORTING
if (server.rss_aware_maxmemory &&
server.maxmemory_policy != REDIS_MAXMEMORY_NO_EVICTION)
{
static unsigned long iterations = 0;
static unsigned long sampling_stage = 1;
static float last_observed_frag = 0;
/* For some time, we analyze what happens during memory pressure, when
* objects are evicted and reallocated. */
if (mem_used > server.maxmemory_enforced) {
unsigned long sample_cycles = 1000000;
/* Every sample_cycle cycles we sample the fragmentation, and
* compare it with the previos one. If it is no longer raising,
* we take it as a guess of the fragmentation with this workload. */
if (sampling_stage && iterations < sample_cycles) {
iterations++;
if (iterations == sample_cycles) {
float current_frag = zmalloc_get_fragmentation_ratio(server.resident_set_size);
if (last_observed_frag == 0) {
/* First sample we get. */
last_observed_frag = current_frag;
} else {
if (current_frag <= last_observed_frag) {
size_t enforced_new;
sampling_stage = 0;
server.maxmemory_frag_guess = current_frag;
/* Update the global fragmentation guess and use
* it (also used it for successive
* "CONFIG SET maxmemory" commands). */
if (server.maxmemory_frag_guess < 1)
server.maxmemory_frag_guess = 1;
else if (server.maxmemory_frag_guess > 2)
server.maxmemory_frag_guess = 2;
/* Only set the new limit if it is higher than our
* initial guess, otherwise it is futile: RSS will
* not go backward anyway. */
enforced_new = (double) server.maxmemory /
server.maxmemory_frag_guess;
if (enforced_new > server.maxmemory_enforced)
server.maxmemory_enforced = enforced_new;
redisLog(REDIS_NOTICE,"RSS aware maxmemory, fragmentation looks stable at: %f", server.maxmemory_frag_guess);
}
last_observed_frag = current_frag;
}
iterations = 0;
}
}
}
mem_target = server.maxmemory_enforced;
}
#endif
/* Check if we are over the memory limit. */
if (mem_used <= server.maxmemory) return REDIS_OK;
if (mem_used <= mem_target) return REDIS_OK;
if (server.maxmemory_policy == REDIS_MAXMEMORY_NO_EVICTION)
return REDIS_ERR; /* We need to free memory, but policy forbids. */
/* Compute how much memory we need to free. */
mem_tofree = mem_used - server.maxmemory;
mem_tofree = mem_used - mem_target;
mem_freed = 0;
latencyStartMonitor(latency);
while (mem_freed < mem_tofree) {
@@ -3326,11 +3348,7 @@ int freeMemoryIfNeeded(void) {
* AOF and Output buffer memory will be freed eventually so
* we only care about memory used by the key space. */
delta = (long long) zmalloc_used_memory();
latencyStartMonitor(eviction_latency);
dbDelete(db,keyobj);
latencyEndMonitor(eviction_latency);
latencyAddSampleIfNeeded("eviction-del",eviction_latency);
latencyRemoveNestedEvent(latency,eviction_latency);
delta -= (long long) zmalloc_used_memory();
mem_freed += delta;
server.stat_evictedkeys++;
@@ -3447,27 +3465,15 @@ void redisAsciiArt(void) {
else if (server.sentinel_mode) mode = "sentinel";
else mode = "standalone";
if (server.syslog_enabled) {
redisLog(REDIS_NOTICE,
"Redis %s (%s/%d) %s bit, %s mode, port %d, pid %ld ready to start.",
REDIS_VERSION,
redisGitSHA1(),
strtol(redisGitDirty(),NULL,10) > 0,
(sizeof(long) == 8) ? "64" : "32",
mode, server.port,
(long) getpid()
);
} else {
snprintf(buf,1024*16,ascii_logo,
REDIS_VERSION,
redisGitSHA1(),
strtol(redisGitDirty(),NULL,10) > 0,
(sizeof(long) == 8) ? "64" : "32",
mode, server.port,
(long) getpid()
);
redisLogRaw(REDIS_NOTICE|REDIS_LOG_RAW,buf);
}
snprintf(buf,1024*16,ascii_logo,
REDIS_VERSION,
redisGitSHA1(),
strtol(redisGitDirty(),NULL,10) > 0,
(sizeof(long) == 8) ? "64" : "32",
mode, server.port,
(long) getpid()
);
redisLogRaw(REDIS_NOTICE|REDIS_LOG_RAW,buf);
zfree(buf);
}
@@ -3667,7 +3673,6 @@ int main(int argc, char **argv) {
#ifdef __linux__
linuxMemoryWarnings();
#endif
checkTcpBacklogSettings();
loadDataFromDisk();
if (server.cluster_enabled) {
if (verifyClusterConfigWithData() == REDIS_ERR) {
+18 -20
View File
@@ -32,7 +32,10 @@
#include "fmacros.h"
#include "config.h"
#if defined(__sun)
#include "solarisfixes.h"
#endif
#include <stdio.h>
#include <stdlib.h>
@@ -94,6 +97,7 @@ typedef long long mstime_t; /* millisecond time type. */
#define REDIS_REPL_PING_SLAVE_PERIOD 10
#define REDIS_RUN_ID_SIZE 40
#define REDIS_EOF_MARK_SIZE 40
#define REDIS_OPS_SEC_SAMPLES 16
#define REDIS_DEFAULT_REPL_BACKLOG_SIZE (1024*1024) /* 1mb */
#define REDIS_DEFAULT_REPL_BACKLOG_TIME_LIMIT (60*60) /* 1 hour */
#define REDIS_REPL_BACKLOG_MIN_SIZE (1024*16) /* 16k */
@@ -117,6 +121,8 @@ typedef long long mstime_t; /* millisecond time type. */
#define REDIS_DEFAULT_REPL_DISABLE_TCP_NODELAY 0
#define REDIS_DEFAULT_MAXMEMORY 0
#define REDIS_DEFAULT_MAXMEMORY_SAMPLES 5
#define REDIS_DEFAULT_MAXMEMORY_FRAG_GUESS 1.4
#define REDIS_DEFAULT_RSS_AWARE_MAXMEMORY 0
#define REDIS_DEFAULT_AOF_FILENAME "appendonly.aof"
#define REDIS_DEFAULT_AOF_NO_FSYNC_ON_REWRITE 0
#define REDIS_DEFAULT_AOF_LOAD_TRUNCATED 1
@@ -124,7 +130,7 @@ typedef long long mstime_t; /* millisecond time type. */
#define REDIS_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC 1
#define REDIS_DEFAULT_MIN_SLAVES_TO_WRITE 0
#define REDIS_DEFAULT_MIN_SLAVES_MAX_LAG 10
#define REDIS_IP_STR_LEN 46 /* INET6_ADDRSTRLEN is 46, but we need to be sure */
#define REDIS_IP_STR_LEN INET6_ADDRSTRLEN
#define REDIS_PEER_ID_LEN (REDIS_IP_STR_LEN+32) /* Must be enough for ip:port */
#define REDIS_BINDADDR_MAX 16
#define REDIS_MIN_RESERVED_FDS 32
@@ -136,13 +142,6 @@ typedef long long mstime_t; /* millisecond time type. */
#define ACTIVE_EXPIRE_CYCLE_SLOW 0
#define ACTIVE_EXPIRE_CYCLE_FAST 1
/* Instantaneous metrics tracking. */
#define REDIS_METRIC_SAMPLES 16 /* Number of samples per metric. */
#define REDIS_METRIC_COMMAND 0 /* Number of commands executed. */
#define REDIS_METRIC_NET_INPUT 1 /* Bytes read to network .*/
#define REDIS_METRIC_NET_OUTPUT 2 /* Bytes written to network. */
#define REDIS_METRIC_COUNT 3
/* Protocol and I/O related defines */
#define REDIS_MAX_QUERYBUF_LEN (1024*1024*1024) /* 1GB max query buffer. */
#define REDIS_IOBUF_LEN (1024*16) /* Generic I/O buffer size */
@@ -713,16 +712,12 @@ struct redisServer {
long long slowlog_log_slower_than; /* SLOWLOG time limit (to get logged) */
unsigned long slowlog_max_len; /* SLOWLOG max number of items logged */
size_t resident_set_size; /* RSS sampled in serverCron(). */
long long stat_net_input_bytes; /* Bytes read from network. */
long long stat_net_output_bytes; /* Bytes written to network. */
/* The following two are used to track instantaneous metrics, like
* number of operations per second, network traffic. */
struct {
long long last_sample_time; /* Timestamp of last sample in ms */
long long last_sample_count;/* Count in last sample */
long long samples[REDIS_METRIC_SAMPLES];
int idx;
} inst_metric[REDIS_METRIC_COUNT];
/* The following two are used to track instantaneous "load" in terms
* of operations per second. */
long long ops_sec_last_sample_time; /* Timestamp of last sample (in ms) */
long long ops_sec_last_sample_ops; /* numcommands in last sample */
long long ops_sec_samples[REDIS_OPS_SEC_SAMPLES];
int ops_sec_idx;
/* Configuration */
int verbosity; /* Loglevel in redis.conf */
int maxidletime; /* Client timeout in seconds */
@@ -846,6 +841,10 @@ struct redisServer {
unsigned long long maxmemory; /* Max number of memory bytes to use */
int maxmemory_policy; /* Policy for key eviction */
int maxmemory_samples; /* Pricision of random sampling */
/* RSS aware maxmemory additional state. */
int rss_aware_maxmemory; /* Non zero if enabled. */
unsigned long long maxmemory_enforced; /* Currently enforced maxmemory. */
float maxmemory_frag_guess; /* Guessed fragmentation. */
/* Blocked clients */
unsigned int bpop_blocked_clients; /* Number of clients blocked by lists */
list *unblocked_clients; /* list of clients to unblock before next loop */
@@ -1123,7 +1122,7 @@ robj *tryObjectEncoding(robj *o);
robj *getDecodedObject(robj *o);
size_t stringObjectLen(robj *o);
robj *createStringObjectFromLongLong(long long value);
robj *createStringObjectFromLongDouble(long double value, int humanfriendly);
robj *createStringObjectFromLongDouble(long double value);
robj *createListObject(void);
robj *createZiplistObject(void);
robj *createSetObject(void);
@@ -1366,7 +1365,6 @@ void blockClient(redisClient *c, int btype);
void unblockClient(redisClient *c);
void replyToBlockedClientTimedOut(redisClient *c);
int getTimeoutFromObjectOrReply(redisClient *c, robj *object, mstime_t *timeout, int unit);
void disconnectAllBlockedClients(void);
/* Git SHA1 */
char *redisGitSHA1(void);
+3 -9
View File
@@ -652,8 +652,7 @@ void replconfCommand(redisClient *c) {
*
* It does a few things:
*
* 1) Put the slave in ONLINE state (useless when the function is called
* because state is already ONLINE but repl_put_online_on_ack is true).
* 1) Put the slave in ONLINE state.
* 2) Make sure the writable event is re-installed, since calling the SYNC
* command disables it, so that we can accumulate output buffer without
* sending it to the slave.
@@ -661,7 +660,7 @@ void replconfCommand(redisClient *c) {
void putSlaveOnline(redisClient *slave) {
slave->replstate = REDIS_REPL_ONLINE;
slave->repl_put_online_on_ack = 0;
slave->repl_ack_time = server.unixtime; /* Prevent false timeout. */
slave->repl_ack_time = server.unixtime;
if (aeCreateFileEvent(server.el, slave->fd, AE_WRITABLE,
sendReplyToClient, slave) == AE_ERR) {
redisLog(REDIS_WARNING,"Unable to register writable event for slave bulk transfer: %s", strerror(errno));
@@ -691,7 +690,6 @@ void sendBulkToSlave(aeEventLoop *el, int fd, void *privdata, int mask) {
freeClient(slave);
return;
}
server.stat_net_output_bytes += nwritten;
sdsrange(slave->replpreamble,nwritten,-1);
if (sdslen(slave->replpreamble) == 0) {
sdsfree(slave->replpreamble);
@@ -720,7 +718,6 @@ void sendBulkToSlave(aeEventLoop *el, int fd, void *privdata, int mask) {
return;
}
slave->repldboff += nwritten;
server.stat_net_output_bytes += nwritten;
if (slave->repldboff == slave->repldbsize) {
close(slave->repldbfd);
slave->repldbfd = -1;
@@ -774,7 +771,6 @@ void updateSlavesWaitingBgsave(int bgsaveerr, int type) {
* is technically online now. */
slave->replstate = REDIS_REPL_ONLINE;
slave->repl_put_online_on_ack = 1;
slave->repl_ack_time = server.unixtime; /* Timeout otherwise. */
} else {
if (bgsaveerr != REDIS_OK) {
freeClient(slave);
@@ -942,7 +938,6 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
replicationAbortSyncTransfer();
return;
}
server.stat_net_input_bytes += nread;
/* When a mark is used, we want to detect EOF asap in order to avoid
* writing the EOF mark into the file... */
@@ -1377,7 +1372,7 @@ error:
int connectWithMaster(void) {
int fd;
fd = anetTcpNonBlockBestEffortBindConnect(NULL,
fd = anetTcpNonBlockBindConnect(NULL,
server.masterhost,server.masterport,REDIS_BIND_ADDR);
if (fd == -1) {
redisLog(REDIS_WARNING,"Unable to connect to MASTER: %s",
@@ -1439,7 +1434,6 @@ void replicationSetMaster(char *ip, int port) {
server.masterhost = sdsnew(ip);
server.masterport = port;
if (server.master) freeClient(server.master);
disconnectAllBlockedClients(); /* Clients blocked in master, now slave. */
disconnectSlaves(); /* Force our slaves to resync with us as well. */
replicationDiscardCachedMaster(); /* Don't try a PSYNC. */
freeReplicationBacklog(); /* Don't allow our chained slaves to PSYNC. */
+7 -49
View File
@@ -30,7 +30,6 @@
#include "redis.h"
#include "sha1.h"
#include "rand.h"
#include "cluster.h"
#include <lua.h>
#include <lauxlib.h>
@@ -214,27 +213,11 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
static int argv_size = 0;
static robj *cached_objects[LUA_CMD_OBJCACHE_SIZE];
static size_t cached_objects_len[LUA_CMD_OBJCACHE_SIZE];
static int inuse = 0; /* Recursive calls detection. */
/* By using Lua debug hooks it is possible to trigger a recursive call
* to luaRedisGenericCommand(), which normally should never happen.
* To make this function reentrant is futile and makes it slower, but
* we should at least detect such a misuse, and abort. */
if (inuse) {
char *recursion_warning =
"luaRedisGenericCommand() recursive call detected. "
"Are you doing funny stuff with Lua debug hooks?";
redisLog(REDIS_WARNING,"%s",recursion_warning);
luaPushError(lua,recursion_warning);
return 1;
}
inuse++;
/* Require at least one argument */
if (argc == 0) {
luaPushError(lua,
"Please specify at least one argument for redis.call()");
inuse--;
return 1;
}
@@ -289,7 +272,6 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
}
luaPushError(lua,
"Lua redis() command arguments must be strings or integers");
inuse--;
return 1;
}
@@ -309,7 +291,6 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
luaPushError(lua,"Unknown Redis command called from Lua script");
goto cleanup;
}
c->cmd = cmd;
/* There are commands that are not allowed inside scripts. */
if (cmd->flags & REDIS_CMD_NOSCRIPT) {
@@ -356,24 +337,8 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
if (cmd->flags & REDIS_CMD_RANDOM) server.lua_random_dirty = 1;
if (cmd->flags & REDIS_CMD_WRITE) server.lua_write_dirty = 1;
/* If this is a Redis Cluster node, we need to make sure Lua is not
* trying to access non-local keys, with the exception of commands
* received from our master. */
if (server.cluster_enabled && !(server.lua_caller->flags & REDIS_MASTER)) {
/* Duplicate relevant flags in the lua client. */
c->flags &= ~(REDIS_READONLY|REDIS_ASKING);
c->flags |= server.lua_caller->flags & (REDIS_READONLY|REDIS_ASKING);
if (getNodeByQuery(c,c->cmd,c->argv,c->argc,NULL,NULL) !=
server.cluster->myself)
{
luaPushError(lua,
"Lua script attempted to access a non local key in a "
"cluster node");
goto cleanup;
}
}
/* Run the command */
c->cmd = cmd;
call(c,REDIS_CALL_SLOWLOG | REDIS_CALL_STATS);
/* Convert the result of the Redis command into a suitable Lua type.
@@ -444,10 +409,8 @@ cleanup:
* return the plain error. */
lua_pushstring(lua,"err");
lua_gettable(lua,-2);
inuse--;
return lua_error(lua);
}
inuse--;
return 1;
}
@@ -574,7 +537,6 @@ void luaLoadLib(lua_State *lua, const char *libname, lua_CFunction luafunc) {
LUALIB_API int (luaopen_cjson) (lua_State *L);
LUALIB_API int (luaopen_struct) (lua_State *L);
LUALIB_API int (luaopen_cmsgpack) (lua_State *L);
LUALIB_API int (luaopen_bit) (lua_State *L);
void luaLoadLibraries(lua_State *lua) {
luaLoadLib(lua, "", luaopen_base);
@@ -585,7 +547,6 @@ void luaLoadLibraries(lua_State *lua) {
luaLoadLib(lua, "cjson", luaopen_cjson);
luaLoadLib(lua, "struct", luaopen_struct);
luaLoadLib(lua, "cmsgpack", luaopen_cmsgpack);
luaLoadLib(lua, "bit", luaopen_bit);
#if 0 /* Stuff that we don't load currently, for sandboxing concerns. */
luaLoadLib(lua, LUA_LOADLIBNAME, luaopen_package);
@@ -612,12 +573,11 @@ void scriptingEnableGlobalsProtection(lua_State *lua) {
/* strict.lua from: http://metalua.luaforge.net/src/lib/strict.lua.html.
* Modified to be adapted to Redis. */
s[j++]="local dbg=debug\n";
s[j++]="local mt = {}\n";
s[j++]="setmetatable(_G, mt)\n";
s[j++]="mt.__newindex = function (t, n, v)\n";
s[j++]=" if dbg.getinfo(2) then\n";
s[j++]=" local w = dbg.getinfo(2, \"S\").what\n";
s[j++]=" if debug.getinfo(2) then\n";
s[j++]=" local w = debug.getinfo(2, \"S\").what\n";
s[j++]=" if w ~= \"main\" and w ~= \"C\" then\n";
s[j++]=" error(\"Script attempted to create global variable '\"..tostring(n)..\"'\", 2)\n";
s[j++]=" end\n";
@@ -625,12 +585,11 @@ void scriptingEnableGlobalsProtection(lua_State *lua) {
s[j++]=" rawset(t, n, v)\n";
s[j++]="end\n";
s[j++]="mt.__index = function (t, n)\n";
s[j++]=" if dbg.getinfo(2) and dbg.getinfo(2, \"S\").what ~= \"C\" then\n";
s[j++]=" if debug.getinfo(2) and debug.getinfo(2, \"S\").what ~= \"C\" then\n";
s[j++]=" error(\"Script attempted to access unexisting global variable '\"..tostring(n)..\"'\", 2)\n";
s[j++]=" end\n";
s[j++]=" return rawget(t, n)\n";
s[j++]="end\n";
s[j++]="debug = nil\n";
s[j++]=NULL;
for (j = 0; s[j] != NULL; j++) code = sdscatlen(code,s[j],strlen(s[j]));
@@ -734,11 +693,10 @@ void scriptingInit(void) {
* information about the caller, that's what makes sense from the point
* of view of the user debugging a script. */
{
char *errh_func = "local dbg = debug\n"
"function __redis__err__handler(err)\n"
" local i = dbg.getinfo(2,'nSl')\n"
char *errh_func = "function __redis__err__handler(err)\n"
" local i = debug.getinfo(2,'nSl')\n"
" if i and i.what == 'C' then\n"
" i = dbg.getinfo(3,'nSl')\n"
" i = debug.getinfo(3,'nSl')\n"
" end\n"
" if i then\n"
" return i.source .. ':' .. i.currentline .. ': ' .. err\n"
+8 -11
View File
@@ -71,7 +71,7 @@ sds sdsempty(void) {
return sdsnewlen("",0);
}
/* Create a new sds string starting from a null terminated C string. */
/* Create a new sds string starting from a null termined C string. */
sds sdsnew(const char *init) {
size_t initlen = (init == NULL) ? 0 : strlen(init);
return sdsnewlen(init, initlen);
@@ -295,7 +295,7 @@ sds sdscpy(sds s, const char *t) {
* conversion. 's' must point to a string with room for at least
* SDS_LLSTR_SIZE bytes.
*
* The function returns the length of the null-terminated string
* The function returns the lenght of the null-terminated string
* representation stored at 's'. */
#define SDS_LLSTR_SIZE 21
int sdsll2str(char *s, long long value) {
@@ -369,7 +369,7 @@ sds sdsfromlonglong(long long value) {
return sdsnewlen(buf,len);
}
/* Like sdscatprintf() but gets va_list instead of being variadic. */
/* Like sdscatpritf() but gets va_list instead of being variadic. */
sds sdscatvprintf(sds s, const char *fmt, va_list ap) {
va_list cpy;
char staticbuf[1024], *buf = staticbuf, *t;
@@ -390,7 +390,7 @@ sds sdscatvprintf(sds s, const char *fmt, va_list ap) {
buf[buflen-2] = '\0';
va_copy(cpy,ap);
vsnprintf(buf, buflen, fmt, cpy);
va_end(cpy);
va_end(ap);
if (buf[buflen-2] != '\0') {
if (buf != staticbuf) zfree(buf);
buflen *= 2;
@@ -415,7 +415,7 @@ sds sdscatvprintf(sds s, const char *fmt, va_list ap) {
*
* Example:
*
* s = sdsnew("Sum is: ");
* s = sdsempty("Sum is: ");
* s = sdscatprintf(s,"%d+%d = %d",a,b,a+b).
*
* Often you need to create a string from scratch with the printf-alike
@@ -557,7 +557,7 @@ sds sdscatfmt(sds s, char const *fmt, ...) {
* Example:
*
* s = sdsnew("AA...AA.a.aa.aHelloWorld :::");
* s = sdstrim(s,"Aa. :");
* s = sdstrim(s,"A. :");
* printf("%s\n", s);
*
* Output will be just "Hello World".
@@ -643,8 +643,8 @@ void sdstoupper(sds s) {
*
* Return value:
*
* positive if s1 > s2.
* negative if s1 < s2.
* 1 if s1 > s2.
* -1 if s1 < s2.
* 0 if s1 and s2 are exactly the same binary string.
*
* If two strings share exactly the same prefix, but one of the two has
@@ -1083,7 +1083,6 @@ int main(void) {
int oldfree;
sdsfree(x);
sdsfree(y);
x = sdsnew("0");
sh = (void*) (x-(sizeof(struct sdshdr)));
test_cond("sdsnew() free/len buffers", sh->len == 1 && sh->free == 0);
@@ -1096,8 +1095,6 @@ int main(void) {
test_cond("sdsIncrLen() -- content", x[0] == '0' && x[1] == '1');
test_cond("sdsIncrLen() -- len", sh->len == 2);
test_cond("sdsIncrLen() -- free", sh->free == oldfree-1);
sdsfree(x);
}
}
test_report()
+4 -59
View File
@@ -397,7 +397,6 @@ struct redisCommand sentinelcmds[] = {
{"publish",sentinelPublishCommand,3,"",0,NULL,0,0,0,0,0},
{"info",sentinelInfoCommand,-1,"",0,NULL,0,0,0,0,0},
{"role",sentinelRoleCommand,1,"l",0,NULL,0,0,0,0,0},
{"client",clientCommand,-2,"rs",0,NULL,0,0,0,0,0},
{"shutdown",shutdownCommand,-1,"",0,NULL,0,0,0,0,0}
};
@@ -923,7 +922,6 @@ sentinelRedisInstance *createSentinelRedisInstance(char *name, int flags, char *
else if (flags & SRI_SENTINEL) table = master->sentinels;
sdsname = sdsnew(name);
if (dictFind(table,sdsname)) {
releaseSentinelAddr(addr);
sdsfree(sdsname);
errno = EBUSY;
return NULL;
@@ -1270,7 +1268,10 @@ int sentinelResetMasterAndChangeAddress(sentinelRedisInstance *master, char *ip,
slave = createSentinelRedisInstance(NULL,SRI_SLAVE,slaves[j]->ip,
slaves[j]->port, master->quorum, master);
releaseSentinelAddr(slaves[j]);
if (slave) sentinelEvent(REDIS_NOTICE,"+slave",slave,"%@");
if (slave) {
sentinelEvent(REDIS_NOTICE,"+slave",slave,"%@");
sentinelFlushConfig();
}
}
zfree(slaves);
@@ -1842,7 +1843,6 @@ void sentinelRefreshInstanceInfo(sentinelRedisInstance *ri, const char *info) {
atoi(port), ri->quorum, ri)) != NULL)
{
sentinelEvent(REDIS_NOTICE,"+slave",slave,"%@");
sentinelFlushConfig();
}
}
}
@@ -2614,31 +2614,6 @@ sentinelRedisInstance *sentinelGetMasterByNameOrReplyError(redisClient *c,
return ri;
}
#define SENTINEL_ISQR_OK 0
#define SENTINEL_ISQR_NOQUORUM (1<<0)
#define SENTINEL_ISQR_NOAUTH (1<<1)
int sentinelIsQuorumReachable(sentinelRedisInstance *master, int *usableptr) {
dictIterator *di;
dictEntry *de;
int usable = 1; /* Number of usable Sentinels. Init to 1 to count myself. */
int result = SENTINEL_ISQR_OK;
int voters = dictSize(master->sentinels)+1; /* Known Sentinels + myself. */
di = dictGetIterator(master->sentinels);
while((de = dictNext(di)) != NULL) {
sentinelRedisInstance *ri = dictGetVal(de);
if (ri->flags & (SRI_S_DOWN|SRI_O_DOWN)) continue;
usable++;
}
dictReleaseIterator(di);
if (usable < (int)master->quorum) result |= SENTINEL_ISQR_NOQUORUM;
if (usable < voters/2+1) result |= SENTINEL_ISQR_NOAUTH;
if (usableptr) *usableptr = usable;
return result;
}
void sentinelCommand(redisClient *c) {
if (!strcasecmp(c->argv[1]->ptr,"masters")) {
/* SENTINEL MASTERS */
@@ -2789,10 +2764,6 @@ void sentinelCommand(redisClient *c) {
sentinelEvent(REDIS_WARNING,"+monitor",ri,"%@ quorum %d",ri->quorum);
addReply(c,shared.ok);
}
} else if (!strcasecmp(c->argv[1]->ptr,"flushconfig")) {
sentinelFlushConfig();
addReply(c,shared.ok);
return;
} else if (!strcasecmp(c->argv[1]->ptr,"remove")) {
/* SENTINEL REMOVE <name> */
sentinelRedisInstance *ri;
@@ -2803,32 +2774,6 @@ void sentinelCommand(redisClient *c) {
dictDelete(sentinel.masters,c->argv[2]->ptr);
sentinelFlushConfig();
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"ckquorum")) {
/* SENTINEL CKQUORUM <name> */
sentinelRedisInstance *ri;
int usable;
if ((ri = sentinelGetMasterByNameOrReplyError(c,c->argv[2]))
== NULL) return;
int result = sentinelIsQuorumReachable(ri,&usable);
if (result == SENTINEL_ISQR_OK) {
addReplySds(c, sdscatfmt(sdsempty(),
"+OK %i usable Sentinels. Quorum and failover authorization "
"can be reached\r\n",usable));
} else {
sds e = sdscatfmt(sdsempty(),
"-NOQUORUM %i usable Sentinels. ",usable);
if (result & SENTINEL_ISQR_NOQUORUM)
e = sdscat(e,"Not enough available Sentinels to reach the"
" specified quorum for this master");
if (result & SENTINEL_ISQR_NOAUTH) {
if (result & SENTINEL_ISQR_NOQUORUM) e = sdscat(e,". ");
e = sdscat(e, "Not enough available Sentinels to reach the"
" majority and authorize a failover");
}
e = sdscat(e,"\r\n");
addReplySds(c,e);
}
} else if (!strcasecmp(c->argv[1]->ptr,"set")) {
if (c->argc < 3 || c->argc % 2 == 0) goto numargserr;
sentinelSetCommand(c);
+9 -7
View File
@@ -23,8 +23,10 @@ A million repetitions of "a"
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <sys/types.h> /* for u_int*_t */
#if defined(__sun)
#include "solarisfixes.h"
#endif
#include "sha1.h"
#include "config.h"
@@ -53,12 +55,12 @@ A million repetitions of "a"
/* Hash a single 512-bit block. This is the core of the algorithm. */
void SHA1Transform(uint32_t state[5], const unsigned char buffer[64])
void SHA1Transform(u_int32_t state[5], const unsigned char buffer[64])
{
uint32_t a, b, c, d, e;
u_int32_t a, b, c, d, e;
typedef union {
unsigned char c[64];
uint32_t l[16];
u_int32_t l[16];
} CHAR64LONG16;
#ifdef SHA1HANDSOFF
CHAR64LONG16 block[1]; /* use array to appear as a pointer */
@@ -128,9 +130,9 @@ void SHA1Init(SHA1_CTX* context)
/* Run your data through this. */
void SHA1Update(SHA1_CTX* context, const unsigned char* data, uint32_t len)
void SHA1Update(SHA1_CTX* context, const unsigned char* data, u_int32_t len)
{
uint32_t i, j;
u_int32_t i, j;
j = context->count[0];
if ((context->count[0] += len << 3) < j)
@@ -168,7 +170,7 @@ void SHA1Final(unsigned char digest[20], SHA1_CTX* context)
for (i = 0; i < 2; i++)
{
uint32_t t = context->count[i];
u_int32_t t = context->count[i];
int j;
for (j = 0; j < 4; t >>= 8, j++)
+4 -4
View File
@@ -6,12 +6,12 @@ By Steve Reid <steve@edmweb.com>
*/
typedef struct {
uint32_t state[5];
uint32_t count[2];
u_int32_t state[5];
u_int32_t count[2];
unsigned char buffer[64];
} SHA1_CTX;
void SHA1Transform(uint32_t state[5], const unsigned char buffer[64]);
void SHA1Transform(u_int32_t state[5], const unsigned char buffer[64]);
void SHA1Init(SHA1_CTX* context);
void SHA1Update(SHA1_CTX* context, const unsigned char* data, uint32_t len);
void SHA1Update(SHA1_CTX* context, const unsigned char* data, u_int32_t len);
void SHA1Final(unsigned char digest[20], SHA1_CTX* context);
-4
View File
@@ -28,8 +28,6 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
#if defined(__sun)
#if defined(__GNUC__)
#include <math.h>
#undef isnan
@@ -50,5 +48,3 @@
#define u_int uint
#define u_int32_t uint32_t
#endif /* __GNUC__ */
#endif /* __sun */
+10 -9
View File
@@ -209,7 +209,7 @@ void sortCommand(redisClient *c) {
}
/* Create a list of operations to perform for every sorted element.
* Operations can be GET */
* Operations can be GET/DEL/INCR/DECR */
operations = listCreate();
listSetFreeMethod(operations,zfree);
j = 2; /* options start at argv[2] */
@@ -285,15 +285,16 @@ void sortCommand(redisClient *c) {
return;
}
/* When sorting a set with no sort specified, we must sort the output
* so the result is consistent across scripting and replication.
/* For the STORE option, or when SORT is called from a Lua script,
* we want to force a specific ordering even when no explicit ordering
* was asked (SORT BY nosort). This guarantees that replication / AOF
* is deterministic.
*
* The other types (list, sorted set) will retain their native order
* even if no sort order is requested, so they remain stable across
* scripting and replication. */
if (dontsort &&
sortval->type == REDIS_SET &&
(storekey || c->flags & REDIS_LUA_CLIENT))
* However in the case 'dontsort' is true, but the type to sort is a
* sorted set, we don't need to do anything as ordering is guaranteed
* in this special case. */
if ((storekey || c->flags & REDIS_LUA_CLIENT) &&
(dontsort && sortval->type != REDIS_ZSET))
{
/* Force ALPHA sorting */
dontsort = 0;
+1 -2
View File
@@ -49,7 +49,7 @@ static int label_margin_top = 1;
* sparklineSequenceAddSample(seq, 10, NULL);
* sparklineSequenceAddSample(seq, 20, NULL);
* sparklineSequenceAddSample(seq, 30, "last sample label");
* sds output = sparklineRender(sdsempty(), seq, 80, 4, SPARKLINE_FILL);
* sds output = sparklineRender(seq, 80, 4);
* freeSparklineSequence(seq);
* ------------------------------------------------------------------------- */
@@ -63,7 +63,6 @@ struct sequence *createSparklineSequence(void) {
/* Add a new sample into a sequence. */
void sparklineSequenceAddSample(struct sequence *seq, double value, char *label) {
label = (label == NULL || label[0] == '\0') ? NULL : zstrdup(label);
if (seq->length == 0) {
seq->min = seq->max = value;
} else {
+1 -1
View File
@@ -565,7 +565,7 @@ void hincrbyfloatCommand(redisClient *c) {
}
value += incr;
new = createStringObjectFromLongDouble(value,1);
new = createStringObjectFromLongDouble(value);
hashTypeTryObjectEncoding(o,&c->argv[2],NULL);
hashTypeSet(o,c->argv[2],new);
addReplyBulk(c,new);
+1 -1
View File
@@ -321,7 +321,7 @@ void smoveCommand(redisClient *c) {
/* If srcset and dstset are equal, SMOVE is a no-op */
if (srcset == dstset) {
addReply(c,setTypeIsMember(srcset,ele) ? shared.cone : shared.czero);
addReply(c,shared.cone);
return;
}
+1 -1
View File
@@ -397,7 +397,7 @@ void incrbyfloatCommand(redisClient *c) {
addReplyError(c,"increment would produce NaN or Infinity");
return;
}
new = createStringObjectFromLongDouble(value,1);
new = createStringObjectFromLongDouble(value);
if (o)
dbOverwrite(c->db,c->argv[1],new);
else
+16 -78
View File
@@ -1171,84 +1171,35 @@ void zsetConvert(robj *zobj, int encoding) {
*----------------------------------------------------------------------------*/
/* This generic command implements both ZADD and ZINCRBY. */
#define ZADD_NONE 0
#define ZADD_INCR (1<<0) /* Increment the score instead of setting it. */
#define ZADD_NX (1<<1) /* Don't touch elements not already existing. */
#define ZADD_XX (1<<2) /* Only touch elements already exisitng. */
#define ZADD_CH (1<<3) /* Return num of elements added or updated. */
void zaddGenericCommand(redisClient *c, int flags) {
void zaddGenericCommand(redisClient *c, int incr) {
static char *nanerr = "resulting score is not a number (NaN)";
robj *key = c->argv[1];
robj *ele;
robj *zobj;
robj *curobj;
double score = 0, *scores = NULL, curscore = 0.0;
int j, elements;
int scoreidx = 0;
/* The following vars are used in order to track what the command actually
* did during the execution, to reply to the client and to trigger the
* notification of keyspace change. */
int added = 0; /* Number of new elements added. */
int updated = 0; /* Number of elements with updated score. */
int processed = 0; /* Number of elements processed, may remain zero with
options like XX. */
int j, elements = (c->argc-2)/2;
int added = 0, updated = 0;
/* Parse options. At the end 'scoreidx' is set to the argument position
* of the score of the first score-element pair. */
scoreidx = 2;
while(scoreidx < c->argc) {
char *opt = c->argv[scoreidx]->ptr;
if (!strcasecmp(opt,"nx")) flags |= ZADD_NX;
else if (!strcasecmp(opt,"xx")) flags |= ZADD_XX;
else if (!strcasecmp(opt,"ch")) flags |= ZADD_CH;
else if (!strcasecmp(opt,"incr")) flags |= ZADD_INCR;
else break;
scoreidx++;
}
/* Turn options into simple to check vars. */
int incr = (flags & ZADD_INCR) != 0;
int nx = (flags & ZADD_NX) != 0;
int xx = (flags & ZADD_XX) != 0;
int ch = (flags & ZADD_CH) != 0;
/* After the options, we expect to have an even number of args, since
* we expect any number of score-element pairs. */
elements = c->argc-scoreidx;
if (elements % 2) {
if (c->argc % 2) {
addReply(c,shared.syntaxerr);
return;
}
elements /= 2; /* Now this holds the number of score-element pairs. */
/* Check for incompatible options. */
if (nx && xx) {
addReplyError(c,
"XX and NX options at the same time are not compatible");
return;
}
if (incr && elements > 1) {
addReplyError(c,
"INCR option supports a single increment-element pair");
return;
}
/* Start parsing all the scores, we need to emit any syntax error
* before executing additions to the sorted set, as the command should
* either execute fully or nothing at all. */
scores = zmalloc(sizeof(double)*elements);
for (j = 0; j < elements; j++) {
if (getDoubleFromObjectOrReply(c,c->argv[scoreidx+j*2],&scores[j],NULL)
if (getDoubleFromObjectOrReply(c,c->argv[2+j*2],&scores[j],NULL)
!= REDIS_OK) goto cleanup;
}
/* Lookup the key and create the sorted set if does not exist. */
zobj = lookupKeyWrite(c->db,key);
if (zobj == NULL) {
if (xx) goto reply_to_client; /* No key + XX option: nothing to do. */
if (server.zset_max_ziplist_entries == 0 ||
server.zset_max_ziplist_value < sdslen(c->argv[scoreidx+1]->ptr))
server.zset_max_ziplist_value < sdslen(c->argv[3]->ptr))
{
zobj = createZsetObject();
} else {
@@ -1269,9 +1220,8 @@ void zaddGenericCommand(redisClient *c, int flags) {
unsigned char *eptr;
/* Prefer non-encoded element when dealing with ziplists. */
ele = c->argv[scoreidx+1+j*2];
ele = c->argv[3+j*2];
if ((eptr = zzlFind(zobj->ptr,ele,&curscore)) != NULL) {
if (nx) continue;
if (incr) {
score += curscore;
if (isnan(score)) {
@@ -1287,8 +1237,7 @@ void zaddGenericCommand(redisClient *c, int flags) {
server.dirty++;
updated++;
}
processed++;
} else if (!xx) {
} else {
/* Optimize: check if the element is too large or the list
* becomes too long *before* executing zzlInsert. */
zobj->ptr = zzlInsert(zobj->ptr,ele,score);
@@ -1298,18 +1247,15 @@ void zaddGenericCommand(redisClient *c, int flags) {
zsetConvert(zobj,REDIS_ENCODING_SKIPLIST);
server.dirty++;
added++;
processed++;
}
} else if (zobj->encoding == REDIS_ENCODING_SKIPLIST) {
zset *zs = zobj->ptr;
zskiplistNode *znode;
dictEntry *de;
ele = c->argv[scoreidx+1+j*2] =
tryObjectEncoding(c->argv[scoreidx+1+j*2]);
ele = c->argv[3+j*2] = tryObjectEncoding(c->argv[3+j*2]);
de = dictFind(zs->dict,ele);
if (de != NULL) {
if (nx) continue;
curobj = dictGetKey(de);
curscore = *(double*)dictGetVal(de);
@@ -1334,30 +1280,22 @@ void zaddGenericCommand(redisClient *c, int flags) {
server.dirty++;
updated++;
}
processed++;
} else if (!xx) {
} else {
znode = zslInsert(zs->zsl,score,ele);
incrRefCount(ele); /* Inserted in skiplist. */
redisAssertWithInfo(c,NULL,dictAdd(zs->dict,ele,&znode->score) == DICT_OK);
incrRefCount(ele); /* Added to dictionary. */
server.dirty++;
added++;
processed++;
}
} else {
redisPanic("Unknown sorted set encoding");
}
}
reply_to_client:
if (incr) { /* ZINCRBY or INCR option. */
if (processed)
addReplyDouble(c,score);
else
addReply(c,shared.nullbulk);
} else { /* ZADD. */
addReplyLongLong(c,ch ? added+updated : added);
}
if (incr) /* ZINCRBY */
addReplyDouble(c,score);
else /* ZADD */
addReplyLongLong(c,added);
cleanup:
zfree(scores);
@@ -1369,11 +1307,11 @@ cleanup:
}
void zaddCommand(redisClient *c) {
zaddGenericCommand(c,ZADD_NONE);
zaddGenericCommand(c,0);
}
void zincrbyCommand(redisClient *c) {
zaddGenericCommand(c,ZADD_INCR);
zaddGenericCommand(c,1);
}
void zremCommand(redisClient *c) {
+13 -58
View File
@@ -38,10 +38,8 @@
#include <sys/time.h>
#include <float.h>
#include <stdint.h>
#include <errno.h>
#include "util.h"
#include "sha1.h"
/* Glob-style pattern matching. */
int stringmatchlen(const char *pattern, int patternLen,
@@ -171,12 +169,11 @@ int stringmatch(const char *pattern, const char *string, int nocase) {
}
/* Convert a string representing an amount of memory into the number of
* bytes, so for instance memtoll("1Gb") will return 1073741824 that is
* bytes, so for instance memtoll("1Gi") will return 1073741824 that is
* (1024*1024*1024).
*
* On parsing error, if *err is not NULL, it's set to 1, otherwise it's
* set to 0. On error the function return value is 0, regardless of the
* fact 'err' is NULL or not. */
* set to 0 */
long long memtoll(const char *p, int *err) {
const char *u;
char buf[128];
@@ -185,7 +182,6 @@ long long memtoll(const char *p, int *err) {
unsigned int digits;
if (err) *err = 0;
/* Search the first non digit character. */
u = p;
if (*u == '-') u++;
@@ -206,26 +202,16 @@ long long memtoll(const char *p, int *err) {
mul = 1024L*1024*1024;
} else {
if (err) *err = 1;
return 0;
mul = 1;
}
/* Copy the digits into a buffer, we'll use strtoll() to convert
* the digit (without the unit) into a number. */
digits = u-p;
if (digits >= sizeof(buf)) {
if (err) *err = 1;
return 0;
return LLONG_MAX;
}
memcpy(buf,p,digits);
buf[digits] = '\0';
char *endptr;
errno = 0;
val = strtoll(buf,&endptr,10);
if ((val == 0 && errno == EINVAL) || *endptr != '\0') {
if (err) *err = 1;
return 0;
}
val = strtoll(buf,NULL,10);
return val*mul;
}
@@ -442,44 +428,11 @@ int d2string(char *buf, size_t len, double value) {
* having run_id == A, and you reconnect and it has run_id == B, you can be
* sure that it is either a different instance or it was restarted. */
void getRandomHexChars(char *p, unsigned int len) {
FILE *fp = fopen("/dev/urandom","r");
char *charset = "0123456789abcdef";
unsigned int j;
/* Global state. */
static int seed_initialized = 0;
static unsigned char seed[20]; /* The SHA1 seed, from /dev/urandom. */
static uint64_t counter = 0; /* The counter we hash with the seed. */
if (!seed_initialized) {
/* Initialize a seed and use SHA1 in counter mode, where we hash
* the same seed with a progressive counter. For the goals of this
* function we just need non-colliding strings, there are no
* cryptographic security needs. */
FILE *fp = fopen("/dev/urandom","r");
if (fp && fread(seed,sizeof(seed),1,fp) == 1)
seed_initialized = 1;
if (fp) fclose(fp);
}
if (seed_initialized) {
while(len) {
unsigned char digest[20];
SHA1_CTX ctx;
unsigned int copylen = len > 20 ? 20 : len;
SHA1Init(&ctx);
SHA1Update(&ctx, seed, sizeof(seed));
SHA1Update(&ctx, (unsigned char*)&counter,sizeof(counter));
SHA1Final(digest, &ctx);
counter++;
memcpy(p,digest,copylen);
/* Convert to hex digits. */
for (j = 0; j < copylen; j++) p[j] = charset[p[j] & 0x0F];
len -= copylen;
p += copylen;
}
} else {
if (fp == NULL || fread(p,len,1,fp) == 0) {
/* If we can't read from /dev/urandom, do some reasonable effort
* in order to create some entropy, since this function is used to
* generate run_id and cluster instance IDs */
@@ -506,12 +459,14 @@ void getRandomHexChars(char *p, unsigned int len) {
x += sizeof(pid);
}
/* Finally xor it with rand() output, that was already seeded with
* time() at startup, and convert to hex digits. */
for (j = 0; j < len; j++) {
* time() at startup. */
for (j = 0; j < len; j++)
p[j] ^= rand();
p[j] = charset[p[j] & 0x0F];
}
}
/* Turn it into hex digits taking just 4 bits out of 8 for every byte. */
for (j = 0; j < len; j++)
p[j] = charset[p[j] & 0x0F];
if (fp) fclose(fp);
}
/* Given the filename, return the absolute path as an SDS string, or NULL
+1 -1
View File
@@ -1 +1 @@
#define REDIS_VERSION "3.0.3"
#define REDIS_VERSION "2.9.999"
+3 -2
View File
@@ -51,9 +51,10 @@
* <len> is the length of the following string (key or value).
* <len> lengths are encoded in a single value or in a 5 bytes value.
* If the first byte value (as an unsigned 8 bit value) is between 0 and
* 253, it's a single-byte length. If it is 254 then a four bytes unsigned
* 252, it's a single-byte length. If it is 253 then a four bytes unsigned
* integer follows (in the host byte ordering). A value of 255 is used to
* signal the end of the hash.
* signal the end of the hash. The special value 254 is used to mark
* empty space that can be used to add new key/value pairs.
*
* <free> is the number of free unused bytes after the string, resulting
* from modification of values associated to a key. For instance if "foo"
+1 -1
View File
@@ -356,7 +356,7 @@ size_t zmalloc_get_smap_bytes_by_field(char *field) {
}
#else
size_t zmalloc_get_smap_bytes_by_field(char *field) {
((void) field);
REDIS_NOTUSED(field);
return 0;
}
#endif
-2
View File
@@ -17,12 +17,10 @@ proc main {} {
}
run_tests
cleanup
end_tests
}
if {[catch main e]} {
puts $::errorInfo
if {$::pause_on_error} pause_on_error
cleanup
exit 1
}
+2 -11
View File
@@ -66,18 +66,9 @@ test "Cluster consistency during live resharding" {
}
# Write random data to random list.
set listid [randomInt $numkeys]
set key "key:$listid"
set key "key:[randomInt $numkeys]"
set ele [randomValue]
# We write both with Lua scripts and with plain commands.
# This way we are able to stress Lua -> Redis command invocation
# as well, that has tests to prevent Lua to write into wrong
# hash slots.
if {$listid % 2} {
$cluster rpush $key $ele
} else {
$cluster eval {redis.call("rpush",KEYS[1],ARGV[1])} 1 $key $ele
}
$cluster rpush $key $ele
lappend content($key) $ele
if {($j % 1000) == 0} {
-192
View File
@@ -1,192 +0,0 @@
# Check the manual failover
source "../tests/includes/init-tests.tcl"
test "Create a 5 nodes cluster" {
create_cluster 5 5
}
test "Cluster is up" {
assert_cluster_state ok
}
test "Cluster is writable" {
cluster_write_test 0
}
test "Instance #5 is a slave" {
assert {[RI 5 role] eq {slave}}
}
test "Instance #5 synced with the master" {
wait_for_condition 1000 50 {
[RI 5 master_link_status] eq {up}
} else {
fail "Instance #5 master link status is not up"
}
}
set current_epoch [CI 1 cluster_current_epoch]
set numkeys 50000
set numops 10000
set cluster [redis_cluster 127.0.0.1:[get_instance_attrib redis 0 port]]
catch {unset content}
array set content {}
test "Send CLUSTER FAILOVER to #5, during load" {
for {set j 0} {$j < $numops} {incr j} {
# Write random data to random list.
set listid [randomInt $numkeys]
set key "key:$listid"
set ele [randomValue]
# We write both with Lua scripts and with plain commands.
# This way we are able to stress Lua -> Redis command invocation
# as well, that has tests to prevent Lua to write into wrong
# hash slots.
if {$listid % 2} {
$cluster rpush $key $ele
} else {
$cluster eval {redis.call("rpush",KEYS[1],ARGV[1])} 1 $key $ele
}
lappend content($key) $ele
if {($j % 1000) == 0} {
puts -nonewline W; flush stdout
}
if {$j == $numops/2} {R 5 cluster failover}
}
}
test "Wait for failover" {
wait_for_condition 1000 50 {
[CI 1 cluster_current_epoch] > $current_epoch
} else {
fail "No failover detected"
}
}
test "Cluster should eventually be up again" {
assert_cluster_state ok
}
test "Cluster is writable" {
cluster_write_test 1
}
test "Instance #5 is now a master" {
assert {[RI 5 role] eq {master}}
}
test "Verify $numkeys keys for consistency with logical content" {
# Check that the Redis Cluster content matches our logical content.
foreach {key value} [array get content] {
assert {[$cluster lrange $key 0 -1] eq $value}
}
}
test "Instance #0 gets converted into a slave" {
wait_for_condition 1000 50 {
[RI 0 role] eq {slave}
} else {
fail "Old master was not converted into slave"
}
}
## Check that manual failover does not happen if we can't talk with the master.
source "../tests/includes/init-tests.tcl"
test "Create a 5 nodes cluster" {
create_cluster 5 5
}
test "Cluster is up" {
assert_cluster_state ok
}
test "Cluster is writable" {
cluster_write_test 0
}
test "Instance #5 is a slave" {
assert {[RI 5 role] eq {slave}}
}
test "Instance #5 synced with the master" {
wait_for_condition 1000 50 {
[RI 5 master_link_status] eq {up}
} else {
fail "Instance #5 master link status is not up"
}
}
test "Make instance #0 unreachable without killing it" {
R 0 deferred 1
R 0 DEBUG SLEEP 10
}
test "Send CLUSTER FAILOVER to instance #5" {
R 5 cluster failover
}
test "Instance #5 is still a slave after some time (no failover)" {
after 5000
assert {[RI 5 role] eq {master}}
}
test "Wait for instance #0 to return back alive" {
R 0 deferred 0
assert {[R 0 read] eq {OK}}
}
## Check with "force" failover happens anyway.
source "../tests/includes/init-tests.tcl"
test "Create a 5 nodes cluster" {
create_cluster 5 5
}
test "Cluster is up" {
assert_cluster_state ok
}
test "Cluster is writable" {
cluster_write_test 0
}
test "Instance #5 is a slave" {
assert {[RI 5 role] eq {slave}}
}
test "Instance #5 synced with the master" {
wait_for_condition 1000 50 {
[RI 5 master_link_status] eq {up}
} else {
fail "Instance #5 master link status is not up"
}
}
test "Make instance #0 unreachable without killing it" {
R 0 deferred 1
R 0 DEBUG SLEEP 10
}
test "Send CLUSTER FAILOVER to instance #5" {
R 5 cluster failover force
}
test "Instance #5 is a master after some time" {
wait_for_condition 1000 50 {
[RI 5 role] eq {master}
} else {
fail "Instance #5 is not a master after some time regardless of FORCE"
}
}
test "Wait for instance #0 to return back alive" {
R 0 deferred 0
assert {[R 0 read] eq {OK}}
}
@@ -1,59 +0,0 @@
# Manual takeover test
source "../tests/includes/init-tests.tcl"
test "Create a 5 nodes cluster" {
create_cluster 5 5
}
test "Cluster is up" {
assert_cluster_state ok
}
test "Cluster is writable" {
cluster_write_test 0
}
test "Killing majority of master nodes" {
kill_instance redis 0
kill_instance redis 1
kill_instance redis 2
}
test "Cluster should eventually be down" {
assert_cluster_state fail
}
test "Use takeover to bring slaves back" {
R 5 cluster failover takeover
R 6 cluster failover takeover
R 7 cluster failover takeover
}
test "Cluster should eventually be up again" {
assert_cluster_state ok
}
test "Cluster is writable" {
cluster_write_test 4
}
test "Instance #5, #6, #7 are now masters" {
assert {[RI 5 role] eq {master}}
assert {[RI 6 role] eq {master}}
assert {[RI 7 role] eq {master}}
}
test "Restarting the previously killed master nodes" {
restart_instance redis 0
restart_instance redis 1
restart_instance redis 2
}
test "Instance #0, #1, #2 gets converted into a slaves" {
wait_for_condition 1000 50 {
[RI 0 role] eq {slave} && [RI 1 role] eq {slave} && [RI 2 role] eq {slave}
} else {
fail "Old masters not converted into slaves"
}
}
+1 -8
View File
@@ -27,17 +27,10 @@ test "Cluster nodes are reachable" {
test "Cluster nodes hard reset" {
foreach_redis_id id {
if {$::valgrind} {
set node_timeout 10000
} else {
set node_timeout 3000
}
catch {R $id flushall} ; # May fail for readonly slaves.
R $id MULTI
R $id cluster reset hard
R $id cluster set-config-epoch [expr {$id+1}]
R $id EXEC
R $id config set cluster-node-timeout $node_timeout
R $id config set cluster-node-timeout 3000
R $id config set cluster-slave-validity-factor 10
R $id config rewrite
}
+14 -55
View File
@@ -16,10 +16,8 @@ source ../support/server.tcl
source ../support/test.tcl
set ::verbose 0
set ::valgrind 0
set ::pause_on_error 0
set ::simulate_error 0
set ::failed 0
set ::sentinel_instances {}
set ::redis_instances {}
set ::sentinel_base_port 20000
@@ -34,25 +32,6 @@ if {[catch {cd tmp}]} {
exit 1
}
# Execute the specified instance of the server specified by 'type', using
# the provided configuration file. Returns the PID of the process.
proc exec_instance {type cfgfile} {
if {$type eq "redis"} {
set prgname redis-server
} elseif {$type eq "sentinel"} {
set prgname redis-sentinel
} else {
error "Unknown instance type."
}
if {$::valgrind} {
set pid [exec valgrind --track-origins=yes --suppressions=../../../src/valgrind.sup --show-reachable=no --show-possibly-lost=no --leak-check=full ../../../src/${prgname} $cfgfile &]
} else {
set pid [exec ../../../src/${prgname} $cfgfile &]
}
return $pid
}
# Spawn a redis or sentinel instance, depending on 'type'.
proc spawn_instance {type base_port count {conf {}}} {
for {set j 0} {$j < $count} {incr j} {
@@ -79,7 +58,14 @@ proc spawn_instance {type base_port count {conf {}}} {
close $cfg
# Finally exec it and remember the pid for later cleanup.
set pid [exec_instance $type $cfgfile]
if {$type eq "redis"} {
set prgname redis-server
} elseif {$type eq "sentinel"} {
set prgname redis-sentinel
} else {
error "Unknown instance type."
}
set pid [exec ../../../src/${prgname} $cfgfile &]
lappend ::pids $pid
# Check availability
@@ -112,7 +98,6 @@ proc cleanup {} {
proc abort_sentinel_test msg {
puts "WARNING: Aborting the test."
puts ">>>>>>>> $msg"
if {$::pause_on_error} pause_on_error
cleanup
exit 1
}
@@ -128,8 +113,6 @@ proc parse_options {} {
set ::pause_on_error 1
} elseif {$opt eq "--fail"} {
set ::simulate_error 1
} elseif {$opt eq {--valgrind}} {
set ::valgrind 1
} elseif {$opt eq "--help"} {
puts "Hello, I'm sentinel.tcl and I run Sentinel unit tests."
puts "\nOptions:"
@@ -232,7 +215,6 @@ proc test {descr code} {
flush stdout
if {[catch {set retval [uplevel 1 $code]} error]} {
incr ::failed
if {[string match "assertion:*" $error]} {
set msg [string range $error 10 end]
puts [colorstr red $msg]
@@ -248,7 +230,6 @@ proc test {descr code} {
}
}
# Execute all the units inside the 'tests' directory.
proc run_tests {} {
set tests [lsort [glob ../tests/*]]
foreach test $tests {
@@ -261,17 +242,6 @@ proc run_tests {} {
}
}
# Print a message and exists with 0 / 1 according to zero or more failures.
proc end_tests {} {
if {$::failed == 0} {
puts "GOOD! No errors."
exit 0
} else {
puts "WARNING $::failed tests faield."
exit 1
}
}
# The "S" command is used to interact with the N-th Sentinel.
# The general form is:
#
@@ -390,31 +360,15 @@ proc get_instance_id_by_port {type port} {
# The instance can be restarted with restart-instance.
proc kill_instance {type id} {
set pid [get_instance_attrib $type $id pid]
set port [get_instance_attrib $type $id port]
if {$pid == -1} {
error "You tried to kill $type $id twice."
}
exec kill -9 $pid
set_instance_attrib $type $id pid -1
set_instance_attrib $type $id link you_tried_to_talk_with_killed_instance
# Remove the PID from the list of pids to kill at exit.
set ::pids [lsearch -all -inline -not -exact $::pids $pid]
# Wait for the port it was using to be available again, so that's not
# an issue to start a new server ASAP with the same port.
set retry 10
while {[incr retry -1]} {
set port_is_free [catch {set s [socket 127.0.01 $port]}]
if {$port_is_free} break
catch {close $s}
after 1000
}
if {$retry == 0} {
error "Port $port does not return available after killing instance."
}
}
# Return true of the instance of the specified type/id is killed.
@@ -431,7 +385,12 @@ proc restart_instance {type id} {
# Execute the instance with its old setup and append the new pid
# file for cleanup.
set pid [exec_instance $type $cfgfile]
if {$type eq "redis"} {
set prgname redis-server
} else {
set prgname redis-sentinel
}
set pid [exec ../../../src/${prgname} $cfgfile &]
set_instance_attrib $type $id pid $pid
lappend ::pids $pid
-24
View File
@@ -1,24 +0,0 @@
set server_path [tmpdir server.log]
set system_name [string tolower [exec uname -s]]
if {$system_name eq {linux} || $system_name eq {darwin}} {
start_server [list overrides [list dir $server_path]] {
test "Server is able to generate a stack trace on selected systems" {
r config set watchdog-period 200
r debug sleep 1
set pattern "*debugCommand*"
set retry 10
while {$retry} {
set result [exec tail -100 < [srv 0 stdout]]
if {[string match $pattern $result]} {
break
}
incr retry -1
after 1000
}
if {$retry == 0} {
error "assertion:expected stack trace not found into log file"
}
}
}
}
+19 -59
View File
@@ -1,17 +1,10 @@
start_server {tags {"repl"}} {
set A [srv 0 client]
set A_host [srv 0 host]
set A_port [srv 0 port]
start_server {} {
set B [srv 0 client]
set B_host [srv 0 host]
set B_port [srv 0 port]
test {Set instance A as slave of B} {
$A slaveof $B_host $B_port
test {First server should have role slave after SLAVEOF} {
r -1 slaveof [srv 0 host] [srv 0 port]
wait_for_condition 50 100 {
[lindex [$A role] 0] eq {slave} &&
[string match {*master_link_status:up*} [$A info replication]]
[s -1 role] eq {slave} &&
[string match {*master_link_status:up*} [r -1 info replication]]
} else {
fail "Can't turn the instance into a slave"
}
@@ -22,9 +15,9 @@ start_server {tags {"repl"}} {
$rd brpoplpush a b 5
r lpush a foo
wait_for_condition 50 100 {
[$A debug digest] eq [$B debug digest]
[r debug digest] eq [r -1 debug digest]
} else {
fail "Master and slave have different digest: [$A debug digest] VS [$B debug digest]"
fail "Master and slave have different digest: [r debug digest] VS [r -1 debug digest]"
}
}
@@ -35,36 +28,7 @@ start_server {tags {"repl"}} {
r lpush c 3
$rd brpoplpush c d 5
after 1000
assert_equal [$A debug digest] [$B debug digest]
}
test {BLPOP followed by role change, issue #2473} {
set rd [redis_deferring_client]
$rd blpop foo 0 ; # Block while B is a master
# Turn B into master of A
$A slaveof no one
$B slaveof $A_host $A_port
wait_for_condition 50 100 {
[lindex [$B role] 0] eq {slave} &&
[string match {*master_link_status:up*} [$B info replication]]
} else {
fail "Can't turn the instance into a slave"
}
# Push elements into the "foo" list of the new slave.
# If the client is still attached to the instance, we'll get
# a desync between the two instances.
$A rpush foo a b c
after 100
wait_for_condition 50 100 {
[$A debug digest] eq [$B debug digest] &&
[$A lrange foo 0 -1] eq {a b c} &&
[$B lrange foo 0 -1] eq {a b c}
} else {
fail "Master and slave have different digest: [$A debug digest] VS [$B debug digest]"
}
assert_equal [r debug digest] [r -1 debug digest]
}
}
}
@@ -149,13 +113,12 @@ foreach dl {no yes} {
start_server {} {
lappend slaves [srv 0 client]
test "Connect multiple slaves at the same time (issue #141), diskless=$dl" {
# Send SLAVEOF commands to slaves
# Send SALVEOF commands to slaves
[lindex $slaves 0] slaveof $master_host $master_port
[lindex $slaves 1] slaveof $master_host $master_port
[lindex $slaves 2] slaveof $master_host $master_port
# Wait for all the three slaves to reach the "online"
# state from the POV of the master.
# Wait for all the three slaves to reach the "online" state
set retry 500
while {$retry} {
set info [r -3 info]
@@ -170,17 +133,6 @@ foreach dl {no yes} {
error "assertion:Slaves not correctly synchronized"
}
# Wait that slaves acknowledge they are online so
# we are sure that DBSIZE and DEBUG DIGEST will not
# fail because of timing issues.
wait_for_condition 500 100 {
[lindex [[lindex $slaves 0] role] 3] eq {connected} &&
[lindex [[lindex $slaves 1] role] 3] eq {connected} &&
[lindex [[lindex $slaves 2] role] 3] eq {connected}
} else {
fail "Slaves still not connected after some time"
}
# Stop the write load
stop_write_load $load_handle0
stop_write_load $load_handle1
@@ -188,8 +140,16 @@ foreach dl {no yes} {
stop_write_load $load_handle3
stop_write_load $load_handle4
# Make sure that slaves and master have same
# number of keys
# Wait that slaves exit the "loading" state
wait_for_condition 500 100 {
![string match {*loading:1*} [[lindex $slaves 0] info]] &&
![string match {*loading:1*} [[lindex $slaves 1] info]] &&
![string match {*loading:1*} [[lindex $slaves 2] info]]
} else {
fail "Slaves still loading data after too much time"
}
# Make sure that slaves and master have same number of keys
wait_for_condition 500 100 {
[$master dbsize] == [[lindex $slaves 0] dbsize] &&
[$master dbsize] == [[lindex $slaves 1] dbsize] &&
-1
View File
@@ -13,7 +13,6 @@ proc main {} {
spawn_instance redis $::redis_base_port $::instances_count
run_tests
cleanup
end_tests
}
if {[catch main e]} {
-34
View File
@@ -1,34 +0,0 @@
# Test for the SENTINEL CKQUORUM command
source "../tests/includes/init-tests.tcl"
set num_sentinels [llength $::sentinel_instances]
test "CKQUORUM reports OK and the right amount of Sentinels" {
foreach_sentinel_id id {
assert_match "*OK $num_sentinels usable*" [S $id SENTINEL CKQUORUM mymaster]
}
}
test "CKQUORUM detects quorum cannot be reached" {
set orig_quorum [expr {$num_sentinels/2+1}]
S 0 SENTINEL SET mymaster quorum [expr {$num_sentinels+1}]
catch {[S 0 SENTINEL CKQUORUM mymaster]} err
assert_match "*NOQUORUM*" $err
S 0 SENTINEL SET mymaster quorum $orig_quorum
}
test "CKQUORUM detects failover authorization cannot be reached" {
set orig_quorum [expr {$num_sentinels/2+1}]
S 0 SENTINEL SET mymaster quorum 1
kill_instance sentinel 1
kill_instance sentinel 2
kill_instance sentinel 3
after 5000
catch {[S 0 SENTINEL CKQUORUM mymaster]} err
assert_match "*NOQUORUM*" $err
S 0 SENTINEL SET mymaster quorum $orig_quorum
restart_instance sentinel 1
restart_instance sentinel 2
restart_instance sentinel 3
}
-2
View File
@@ -226,8 +226,6 @@ proc ::redis_cluster::get_keys_from_command {cmd argv} {
# Special handling for other commands
switch -exact $cmd {
mget {return $argv}
eval {return [lrange $argv 2 1+[lindex $argv 1]]}
evalsha {return [lrange $argv 2 1+[lindex $argv 1]]}
}
# All the remaining commands are not handled.
+6 -17
View File
@@ -54,15 +54,10 @@ proc kill_server config {
# kill server and wait for the process to be totally exited
catch {exec kill $pid}
if {$::valgrind} {
set max_wait 60000
} else {
set max_wait 10000
}
while {[is_alive $config]} {
incr wait 10
if {$wait >= $max_wait} {
if {$wait >= 5000} {
puts "Forcing process $pid to exit..."
catch {exec kill -KILL $pid}
} elseif {$wait % 1000 == 0} {
@@ -75,9 +70,6 @@ proc kill_server config {
if {$::valgrind} {
check_valgrind_errors [dict get $config stderr]
}
# Remove this pid from the set of active pids in the test server.
send_data_packet $::test_server_fd server-killed $pid
}
proc is_alive config {
@@ -212,14 +204,11 @@ proc start_server {options {code undefined}} {
set stderr [format "%s/%s" [dict get $config "dir"] "stderr"]
if {$::valgrind} {
set pid [exec valgrind --track-origins=yes --suppressions=src/valgrind.sup --show-reachable=no --show-possibly-lost=no --leak-check=full src/redis-server $config_file > $stdout 2> $stderr &]
exec valgrind --suppressions=src/valgrind.sup --show-reachable=no --show-possibly-lost=no --leak-check=full src/redis-server $config_file > $stdout 2> $stderr &
} else {
set pid [exec src/redis-server $config_file > $stdout 2> $stderr &]
exec src/redis-server $config_file > $stdout 2> $stderr &
}
# Tell the test server about this new instance.
send_data_packet $::test_server_fd server-spawned $pid
# check that the server actually started
# ugly but tries to be as fast as possible...
if {$::valgrind} {set retrynum 1000} else {set retrynum 100}
@@ -245,9 +234,9 @@ proc start_server {options {code undefined}} {
return
}
# Wait for actual startup
while {![info exists _pid]} {
regexp {PID:\s(\d+)} [exec cat $stdout] _ _pid
# find out the pid
while {![info exists pid]} {
regexp {PID:\s(\d+)} [exec cat $stdout] _ pid
after 100
}
+19 -55
View File
@@ -38,7 +38,6 @@ set ::all_tests {
integration/aof
integration/rdb
integration/convert-zipmap-hash-on-load
integration/logging
unit/pubsub
unit/slowlog
unit/scripting
@@ -66,9 +65,6 @@ set ::file ""; # If set, runs only the tests in this comma separated list
set ::curfile ""; # Hold the filename of the current suite
set ::accurate 0; # If true runs fuzz tests with more iterations
set ::force_failure 0
set ::timeout 600; # 10 minutes without progresses will quit the test.
set ::last_progress [clock seconds]
set ::active_servers {} ; # Pids of active Redis instances.
# Set to 1 when we are running in client mode. The Redis test uses a
# server-client model to run tests simultaneously. The server instance
@@ -204,19 +200,11 @@ proc test_server_main {} {
vwait forever
}
# This function gets called 10 times per second.
# This function gets called 10 times per second, for now does nothing but
# may be used in the future in order to detect test clients taking too much
# time to execute the task.
proc test_server_cron {} {
set elapsed [expr {[clock seconds]-$::last_progress}]
if {$elapsed > $::timeout} {
set err "\[[colorstr red TIMEOUT]\]: clients state report follows."
puts $err
show_clients_state
kill_clients
force_kill_all_servers
the_end
}
# Do some work here.
after 100 test_server_cron
}
@@ -242,8 +230,6 @@ proc read_from_test_client fd {
set bytes [gets $fd]
set payload [read $fd $bytes]
foreach {status data} $payload break
set ::last_progress [clock seconds]
if {$status eq {ready}} {
if {!$::quiet} {
puts "\[$status\]: $data"
@@ -270,15 +256,12 @@ proc read_from_test_client fd {
set ::active_clients_task($fd) "(ERR) $data"
} elseif {$status eq {exception}} {
puts "\[[colorstr red $status]\]: $data"
kill_clients
force_kill_all_servers
foreach p $::clients_pids {
catch {exec kill -9 $p}
}
exit 1
} elseif {$status eq {testing}} {
set ::active_clients_task($fd) "(IN PROGRESS) $data"
} elseif {$status eq {server-spawned}} {
lappend ::active_servers $data
} elseif {$status eq {server-killed}} {
set ::active_servers [lsearch -all -inline -not -exact $::active_servers $data]
} else {
if {!$::quiet} {
puts "\[$status\]: $data"
@@ -286,31 +269,6 @@ proc read_from_test_client fd {
}
}
proc show_clients_state {} {
# The following loop is only useful for debugging tests that may
# enter an infinite loop. Commented out normally.
foreach x $::active_clients {
if {[info exist ::active_clients_task($x)]} {
puts "$x => $::active_clients_task($x)"
} else {
puts "$x => ???"
}
}
}
proc kill_clients {} {
foreach p $::clients_pids {
catch {exec kill $p}
}
}
proc force_kill_all_servers {} {
foreach p $::active_servers {
puts "Killing still running Redis server $p"
catch {exec kill -9 $p}
}
}
# A new client is idle. Remove it from the list of active clients and
# if there are still test units to run, launch them.
proc signal_idle_client fd {
@@ -318,7 +276,17 @@ proc signal_idle_client fd {
set ::active_clients \
[lsearch -all -inline -not -exact $::active_clients $fd]
if 0 {show_clients_state}
if 0 {
# The following loop is only useful for debugging tests that may
# enter an infinite loop. Commented out normally.
foreach x $::active_clients {
if {[info exist ::active_clients_task($x)]} {
puts "$x => $::active_clients_task($x)"
} else {
puts "$x => ???"
}
}
}
# New unit to process?
if {$::next_test != [llength $::all_tests]} {
@@ -393,8 +361,7 @@ proc print_help_screen {} {
"--quiet Don't show individual tests."
"--single <unit> Just execute the specified unit (see next option)."
"--list-tests List all the available test units."
"--clients <num> Number of test clients (default 16)."
"--timeout <sec> Test timeout in seconds (default 10 min)."
"--clients <num> Number of test clients (16)."
"--force-failure Force the execution of a test that always fails."
"--help Print this help screen."
} "\n"]
@@ -443,9 +410,6 @@ for {set j 0} {$j < [llength $argv]} {incr j} {
} elseif {$opt eq {--clients}} {
set ::numclients $arg
incr j
} elseif {$opt eq {--timeout}} {
set ::timeout $arg
incr j
} elseif {$opt eq {--help}} {
print_help_screen
exit 0
+1 -6
View File
@@ -1,20 +1,15 @@
proc test_memory_efficiency {range} {
r flushall
set rd [redis_deferring_client]
set base_mem [s used_memory]
set written 0
for {set j 0} {$j < 10000} {incr j} {
set key key:$j
set val [string repeat A [expr {int(rand()*$range)}]]
$rd set $key $val
r set $key $val
incr written [string length $key]
incr written [string length $val]
incr written 2 ;# A separator is the minimum to store key-value data.
}
for {set j 0} {$j < 10000} {incr j} {
$rd read ; # Discard replies
}
set current_mem [s used_memory]
set used [expr {$current_mem-$base_mem}]
set efficiency [expr {double($written)/$used}]
-10
View File
@@ -226,14 +226,4 @@ start_server {tags {"scan"}} {
set res [r zscan mykey 0 MATCH foo* COUNT 10000]
lsort -unique [lindex $res 1]
}
test "ZSCAN scores: regression test for issue #2175" {
r del mykey
for {set j 0} {$j < 500} {incr j} {
r zadd mykey 9.8813129168249309e-323 $j
}
set res [lindex [r zscan mykey 0] 1]
set first_score [lindex $res 1]
assert {$first_score != 0}
}
}
-88
View File
@@ -184,94 +184,6 @@ start_server {tags {"scripting"}} {
set e
} {*against a key*}
test {EVAL - JSON numeric decoding} {
# We must return the table as a string because otherwise
# Redis converts floats to ints and we get 0 and 1023 instead
# of 0.0003 and 1023.2 as the parsed output.
r eval {return
table.concat(
cjson.decode(
"[0.0, -5e3, -1, 0.3e-3, 1023.2, 0e10]"), " ")
} 0
} {0 -5000 -1 0.0003 1023.2 0}
test {EVAL - JSON string decoding} {
r eval {local decoded = cjson.decode('{"keya": "a", "keyb": "b"}')
return {decoded.keya, decoded.keyb}
} 0
} {a b}
test {EVAL - cmsgpack can pack double?} {
r eval {local encoded = cmsgpack.pack(0.1)
local h = ""
for i = 1, #encoded do
h = h .. string.format("%02x",string.byte(encoded,i))
end
return h
} 0
} {cb3fb999999999999a}
test {EVAL - cmsgpack can pack negative int64?} {
r eval {local encoded = cmsgpack.pack(-1099511627776)
local h = ""
for i = 1, #encoded do
h = h .. string.format("%02x",string.byte(encoded,i))
end
return h
} 0
} {d3ffffff0000000000}
test {EVAL - cmsgpack can pack and unpack circular references?} {
r eval {local a = {x=nil,y=5}
local b = {x=a}
a['x'] = b
local encoded = cmsgpack.pack(a)
local h = ""
-- cmsgpack encodes to a depth of 16, but can't encode
-- references, so the encoded object has a deep copy recusive
-- depth of 16.
for i = 1, #encoded do
h = h .. string.format("%02x",string.byte(encoded,i))
end
-- when unpacked, re.x.x != re because the unpack creates
-- individual tables down to a depth of 16.
-- (that's why the encoded output is so large)
local re = cmsgpack.unpack(encoded)
assert(re)
assert(re.x)
assert(re.x.x.y == re.y)
assert(re.x.x.x.x.y == re.y)
assert(re.x.x.x.x.x.x.y == re.y)
assert(re.x.x.x.x.x.x.x.x.x.x.y == re.y)
-- maximum working depth:
assert(re.x.x.x.x.x.x.x.x.x.x.x.x.x.x.y == re.y)
-- now the last x would be b above and has no y
assert(re.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x)
-- so, the final x.x is at the depth limit and was assigned nil
assert(re.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x.x == nil)
return {h, re.x.x.x.x.x.x.x.x.y == re.y, re.y == 5}
} 0
} {82a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a178c0 1 1}
test {EVAL - Numerical sanity check from bitop} {
r eval {assert(0x7fffffff == 2147483647, "broken hex literals");
assert(0xffffffff == -1 or 0xffffffff == 2^32-1,
"broken hex literals");
assert(tostring(-1) == "-1", "broken tostring()");
assert(tostring(0xffffffff) == "-1" or
tostring(0xffffffff) == "4294967295",
"broken tostring()")
} 0
} {}
test {EVAL - Verify minimal bitop functionality} {
r eval {assert(bit.tobit(1) == 1);
assert(bit.band(1) == 1);
assert(bit.bxor(1,2) == 3);
assert(bit.bor(1,2,4,8,16,32,64,128) == 255)
} 0
} {}
test {SCRIPTING FLUSH - is able to clear the scripts cache?} {
r set mykey myval
set v [r evalsha fd758d1589d044dd850a6f05d52f2eefd27f033f 1 mykey]
-18
View File
@@ -246,24 +246,6 @@ start_server {
r sort mylist by num get x:*->
} {100}
test "SORT by nosort retains native order for lists" {
r del testa
r lpush testa 2 1 4 3 5
r sort testa by nosort
} {5 3 4 1 2}
test "SORT by nosort plus store retains native order for lists" {
r del testa
r lpush testa 2 1 4 3 5
r sort testa by nosort store testb
r lrange testb 0 -1
} {5 3 4 1 2}
test "SORT by nosort with limit returns based on original list order" {
r sort testa by nosort limit 0 3 store testb
r lrange testb 0 -1
} {5 3 4}
tags {"slow"} {
set num 100
set res [create_random_dataset $num lpush]
-1
View File
@@ -450,7 +450,6 @@ start_server {
test "SMOVE non existing key" {
setup_move
assert_equal 0 [r smove myset1 myset2 foo]
assert_equal 0 [r smove myset1 myset1 foo]
assert_equal {1 a b} [lsort [r smembers myset1]]
assert_equal {2 3 4} [lsort [r smembers myset2]]
}
-80
View File
@@ -43,84 +43,6 @@ start_server {tags {"zset"}} {
assert_error "*not*float*" {r zadd myzset nan abc}
}
test "ZADD with options syntax error with incomplete pair" {
r del ztmp
catch {r zadd ztmp xx 10 x 20} err
set err
} {ERR*}
test "ZADD XX option without key - $encoding" {
r del ztmp
assert {[r zadd ztmp xx 10 x] == 0}
assert {[r type ztmp] eq {none}}
}
test "ZADD XX existing key - $encoding" {
r del ztmp
r zadd ztmp 10 x
assert {[r zadd ztmp xx 20 y] == 0}
assert {[r zcard ztmp] == 1}
}
test "ZADD XX returns the number of elements actually added" {
r del ztmp
r zadd ztmp 10 x
set retval [r zadd ztmp 10 x 20 y 30 z]
assert {$retval == 2}
}
test "ZADD XX updates existing elements score" {
r del ztmp
r zadd ztmp 10 x 20 y 30 z
r zadd ztmp xx 5 foo 11 x 21 y 40 zap
assert {[r zcard ztmp] == 3}
assert {[r zscore ztmp x] == 11}
assert {[r zscore ztmp y] == 21}
}
test "ZADD XX and NX are not compatible" {
r del ztmp
catch {r zadd ztmp xx nx 10 x} err
set err
} {ERR*}
test "ZADD NX with non exisitng key" {
r del ztmp
r zadd ztmp nx 10 x 20 y 30 z
assert {[r zcard ztmp] == 3}
}
test "ZADD NX only add new elements without updating old ones" {
r del ztmp
r zadd ztmp 10 x 20 y 30 z
assert {[r zadd ztmp nx 11 x 21 y 100 a 200 b] == 2}
assert {[r zscore ztmp x] == 10}
assert {[r zscore ztmp y] == 20}
assert {[r zscore ztmp a] == 100}
assert {[r zscore ztmp b] == 200}
}
test "ZADD INCR works like ZINCRBY" {
r del ztmp
r zadd ztmp 10 x 20 y 30 z
r zadd ztmp INCR 15 x
assert {[r zscore ztmp x] == 25}
}
test "ZADD INCR works with a single score-elemenet pair" {
r del ztmp
r zadd ztmp 10 x 20 y 30 z
catch {r zadd ztmp INCR 15 x 10 y} err
set err
} {ERR*}
test "ZADD CH option changes return value to all changed elements" {
r del ztmp
r zadd ztmp 10 x 20 y 30 z
assert {[r zadd ztmp 11 x 21 y 30 z] == 0}
assert {[r zadd ztmp ch 12 x 22 y 30 z] == 2}
}
test "ZINCRBY calls leading to NaN result in error" {
r zincrby myzset +inf abc
assert_error "*NaN*" {r zincrby myzset -inf abc}
@@ -155,8 +77,6 @@ start_server {tags {"zset"}} {
}
test "ZCARD basics - $encoding" {
r del ztmp
r zadd ztmp 10 a 20 b 30 c
assert_equal 3 [r zcard ztmp]
assert_equal 0 [r zcard zdoesntexist]
}
-50
View File
@@ -1,50 +0,0 @@
# This simple script is used in order to estimate the average PFAIL->FAIL
# state switch after a failure.
set ::sleep_time 10 ; # How much to sleep to trigger PFAIL.
set ::fail_port 30016 ; # Node to put in sleep.
set ::other_port 30001 ; # Node to use to monitor the flag switch.
proc avg vector {
set sum 0.0
foreach x $vector {
set sum [expr {$sum+$x}]
}
expr {$sum/[llength $vector]}
}
set samples {}
while 1 {
exec redis-cli -p $::fail_port debug sleep $::sleep_time > /dev/null &
# Wait for fail? to appear.
while 1 {
set output [exec redis-cli -p $::other_port cluster nodes]
if {[string match {*fail\?*} $output]} break
after 100
}
puts "FAIL?"
set start [clock milliseconds]
# Wait for fail? to disappear.
while 1 {
set output [exec redis-cli -p $::other_port cluster nodes]
if {![string match {*fail\?*} $output]} break
after 100
}
puts "FAIL"
set now [clock milliseconds]
set elapsed [expr {$now-$start}]
puts $elapsed
lappend samples $elapsed
puts "AVG([llength $samples]): [avg $samples]"
# Wait for the instance to be available again.
exec redis-cli -p $::fail_port ping
# Wait for the fail flag to be cleared.
after 2000
}
-5
View File
@@ -1,5 +0,0 @@
config.sh
*.rdb
*.aof
*.conf
*.log
-27
View File
@@ -1,27 +0,0 @@
Create-custer is a small script used to easily start a big number of Redis
instances configured to run in cluster mode. Its main goal is to allow manual
testing in a condition which is not easy to replicate with the Redis cluster
unit tests, for example when a lot of instances are needed in order to trigger
a give bug.
The tool can also be used just to easily create a number of instances in a
Redis Cluster in order to experiment a bit with the system.
USAGE
---
To create a cluster, follow this steps:
1. Edit create-cluster and change the start / end port, depending on the
number of instances you want to create.
2. Use "./create-cluster start" in order to run the instances.
3. Use "./create-cluster create" in order to execute redis-trib create, so that
an actual Redis cluster will be created.
4. Now you are ready to play with the cluster. AOF files and logs for each instances are created in the current directory.
In order to stop a cluster:
1. Use "./craete-cluster stop" to stop all the instances. After you stopped the instances you can use "./create-cluster start" to restart them if you change ideas.
2. Use "./create-cluster clean" to remove all the AOF / log files to restat with a clean environment.
Use the command "./create-cluster help" to get the full list of features.
-95
View File
@@ -1,95 +0,0 @@
#!/bin/bash
# Settings
PORT=30000
TIMEOUT=2000
NODES=6
REPLICAS=1
# You may want to put the above config parameters into config.sh in order to
# override the defaults without modifying this script.
if [ -a config.sh ]
then
source "config.sh"
fi
# Computed vars
ENDPORT=$((PORT+NODES))
if [ "$1" == "start" ]
then
while [ $((PORT < ENDPORT)) != "0" ]; do
PORT=$((PORT+1))
echo "Starting $PORT"
../../src/redis-server --port $PORT --cluster-enabled yes --cluster-config-file nodes-${PORT}.conf --cluster-node-timeout $TIMEOUT --appendonly yes --appendfilename appendonly-${PORT}.aof --dbfilename dump-${PORT}.rdb --logfile ${PORT}.log --daemonize yes
done
exit 0
fi
if [ "$1" == "create" ]
then
HOSTS=""
while [ $((PORT < ENDPORT)) != "0" ]; do
PORT=$((PORT+1))
HOSTS="$HOSTS 127.0.0.1:$PORT"
done
../../src/redis-trib.rb create --replicas $REPLICAS $HOSTS
exit 0
fi
if [ "$1" == "stop" ]
then
while [ $((PORT < ENDPORT)) != "0" ]; do
PORT=$((PORT+1))
echo "Stopping $PORT"
../../src/redis-cli -p $PORT shutdown nosave
done
exit 0
fi
if [ "$1" == "watch" ]
then
PORT=$((PORT+1))
while [ 1 ]; do
clear
date
../../src/redis-cli -p $PORT cluster nodes | head -30
sleep 1
done
exit 0
fi
if [ "$1" == "tail" ]
then
INSTANCE=$2
PORT=$((PORT+INSTANCE))
tail -f ${PORT}.log
exit 0
fi
if [ "$1" == "call" ]
then
while [ $((PORT < ENDPORT)) != "0" ]; do
PORT=$((PORT+1))
../../src/redis-cli -p $PORT $2 $3 $4 $5 $6 $7 $8 $9
done
exit 0
fi
if [ "$1" == "clean" ]
then
rm -rf *.log
rm -rf appendonly*.aof
rm -rf dump*.rdb
rm -rf nodes*.conf
exit 0
fi
echo "Usage: $0 [start|create|stop|watch|tail|clean]"
echo "start -- Launch Redis Cluster instances."
echo "create -- Create a cluster using redis-trib create."
echo "stop -- Stop Redis Cluster instances."
echo "watch -- Show CLUSTER NODES output (first 30 lines) of first node."
echo "tail <id> -- Run tail -f of instance at base port + ID."
echo "clean -- Remove all instances data, logs, configs."
+2 -3
View File
@@ -26,12 +26,11 @@ case "$1" in
fi
;;
status)
PID=$(cat $PIDFILE)
if [ ! -x /proc/${PID} ]
if [ ! -f $PIDFILE ]
then
echo 'Redis is not running'
else
echo "Redis is running ($PID)"
echo "Redis is running ($(<$PIDFILE))"
fi
;;
restart)
+24
View File
@@ -0,0 +1,24 @@
#!/bin/sh
if [ $# != "1" ]
then
echo "Usage: ${0} <git-ref>"
exit 1
fi
TAG=$1
TARNAME="redis-${TAG}.tar.gz"
DOWNLOADURL="http://download.redis.io/releases/${TARNAME}"
ssh antirez@metal "export TERM=xterm;
cd /tmp;
rm -rf test_release_tmp_dir;
cd test_release_tmp_dir;
wget $DOWNLOADURL;
tar xvzf $TARNAME;
cd redis-${TAG};
make;
./runtest;
./runtest-sentinel;
if [ -x runtest-cluster ]; then
./runtest-cluster;
fi"
@@ -8,6 +8,7 @@ fi
TAG=$1
TARNAME="redis-${TAG}.tar"
echo "Generating /tmp/${TARNAME}"
cd ~/hack/redis
git archive $TAG --prefix redis-${TAG}/ > /tmp/$TARNAME || exit 1
echo "Gizipping the archive"
rm -f /tmp/$TARNAME.gz
+6
View File
@@ -0,0 +1,6 @@
#!/bin/bash
echo "Uploading..."
scp /tmp/redis-${1}.tar.gz antirez@antirez.com:/var/virtual/download.redis.io/httpdocs/releases/
echo "Updating web site... (press any key if it is a stable release, or Ctrl+C)"
read x
ssh antirez@antirez.com "cd /var/virtual/download.redis.io/httpdocs; ./update.sh ${1}"
+8
View File
@@ -0,0 +1,8 @@
#!/bin/bash
SHA=$(curl -s http://download.redis.io/releases/redis-${1}.tar.gz | shasum | cut -f 1 -d' ')
ENTRY="hash redis-${1}.tar.gz sha1 $SHA http://download.redis.io/releases/redis-${1}.tar.gz"
echo $ENTRY >> ~/hack/redis-hashes/README
vi ~/hack/redis-hashes/README
echo "Press any key to commit, Ctrl-C to abort)."
read yes
(cd ~/hack/redis-hashes; git commit -a -m "${1} hash."; git push)
+1 -7
View File
@@ -1,15 +1,9 @@
# This script is from http://poormansprofiler.org/
#
# NOTE: Instead of using this script, you should use the Redis
# Software Watchdog, which provides a similar functionality but in
# a more reliable / easy to use way.
#
# Check http://redis.io/topics/latency for more information.
#!/bin/bash
nsamples=1
sleeptime=0
pid=$(ps auxww | grep '[r]edis-server' | awk '{print $2}')
pid=$(pidof redis-server)
for x in $(seq 1 $nsamples)
do