Compare commits

..
416 Commits
Author SHA1 Message Date
antirez b41f4e5a48 Redis 2.6.6 2012-11-28 18:53:09 +01:00
antirez 21b26915ce Jemalloc updated to version 3.2.0. 2012-11-28 18:42:41 +01:00
antirez 3ccfb5a4d6 Redis 2.6.5 2012-11-22 17:03:27 +01:00
antirez 9120275dc9 EVALSHA is now case insensitive.
EVALSHA used to crash if the SHA1 was not lowercase (Issue #783).
Fixed using a case insensitive dictionary type for the sha -> script
map used for replication of scripts.
2012-11-22 15:51:01 +01:00
antirez de00a5a092 Fix integer overflow in zunionInterGenericCommand().
This fixes issue #761.
2012-11-22 15:28:45 +01:00
antirez 2b45ef020c Test: MULTI state is cleared after EXECABORT error. 2012-11-22 10:36:36 +01:00
antirez 52bd3d8a39 Test: make sure EXEC fails after previous transaction errors. 2012-11-22 10:36:32 +01:00
antirez 37d978d9b4 Test: MULTI/EXEC tests moved into multi.tcl. 2012-11-22 10:36:28 +01:00
antirez 41f0f927c9 Safer handling of MULTI/EXEC on errors.
After the transcation starts with a MULIT, the previous behavior was to
return an error on problems such as maxmemory limit reached. But still
to execute the transaction with the subset of queued commands on EXEC.

While it is true that the client was able to check for errors
distinguish QUEUED by an error reply, MULTI/EXEC in most client
implementations uses pipelining for speed, so all the commands and EXEC
are sent without caring about replies.

With this change:

1) EXEC fails if at least one command was not queued because of an
error. The EXECABORT error is used.
2) A generic error is always reported on EXEC.
3) The client DISCARDs the MULTI state after a failed EXEC, otherwise
pipelining multiple transactions would be basically impossible:
After a failed EXEC the next transaction would be simply queued as
the tail of the previous transaction.
2012-11-22 10:36:20 +01:00
antirez 5ab4151d7f Children creating AOF or RDB files now report memory used by COW.
Finally Redis is able to report the amount of memory used by
copy-on-write while saving an RDB or writing an AOF file in background.

Note that this information is currently only logged (at NOTICE level)
and not shown in INFO because this is less trivial (but surely doable
with some minor form of interprocess communication).

The reason we can't capture this information on the parent before we
call wait3() is that the Linux kernel will release the child memory
ASAP, and only retain the minimal state for the process that is useful
to report the child termination to the parent.

The COW size is obtained by summing all the Private_Dirty fields found
in the "smap" file inside the proc filesystem for the process.

All this is Linux specific and is not available on other systems.
2012-11-20 12:32:19 +01:00
antirez e95ca66320 zmalloc_get_private_dirty() function added (Linux only).
For non Linux systmes it just returns 0.

This function is useful to estimate copy-on-write because of childs
saving stuff on disk.
2012-11-20 12:32:14 +01:00
antirez 681c191006 zmalloc: kill unused __size parameter in update_zmalloc_stat_alloc() macro. 2012-11-14 13:07:48 +01:00
antirez d85a09dfc6 MIGRATE: fix default timeout to 1000 milliseconds.
When a timeout <= 0 is provided we set a default timeout of 1 second.
It was set to 1 millisecond for an error resulting from a recent change.
2012-11-12 18:58:23 +01:00
antirez 26a48504cb MIGRATE timeout should be in milliseconds.
While it is documented that the MIGRATE timeout is in milliseconds, it
was in seconds instead. This commit fixes the problem.
2012-11-12 18:57:55 +01:00
antirez ae3aeca828 MIGRATE: fix fd leak due to missing close on error. 2012-11-12 18:56:29 +01:00
antirez 5a8234d2c9 Copyright date fixed in COPYING file. 2012-11-08 19:14:29 +01:00
antirez 2861cd84f9 Redis 2.6.4 2012-11-08 18:47:58 +01:00
antirez 8e30092f06 Make clear that contributing code to the Redis project means to release it under the terms of the BSD license. 2012-11-08 18:43:37 +01:00
antirez d288ee655f BSD license added to every C source and header file. 2012-11-08 18:33:13 +01:00
antirez 89514f80e9 Redis 2.6.3 2012-11-06 22:07:51 +01:00
Runzhen Wangandantirez 3f006679b1 fix a typo in redis.h line 595 comment 2012-11-02 12:12:01 +01:00
antirez 36f026a3a0 More robust handling of AOF rewrite child.
After the wait3() syscall we used to do something like that:

    if (pid == server.rdb_child_pid) {
        backgroundSaveDoneHandler(exitcode,bysignal);
    } else {
        ....
    }

So the AOF rewrite was handled in the else branch without actually
checking if the pid really matches. This commit makes the check explicit
and logs at WARNING level if the pid returned by wait3() does not match
neither the RDB or AOF rewrite child.
2012-11-01 22:41:57 +01:00
Salvatore Sanfilippo cc017c5839 Merge pull request #743 from Cofyc/fixtypo
fix typo in comments (redis.c, networking.c)
2012-11-01 14:26:01 -07:00
antirez b6ffa85f11 Unix socket clients properly displayed in MONITOR and CLIENT LIST.
This also fixes issue #745.
2012-11-01 22:12:45 +01:00
antirez 4917a6a8b3 32 bit build fixed on Linux.
It failed because of the way jemalloc was compiled (without passing the
right flags to make, but just to configure). Now the same set of flags
are also passed to the make command, fixing the issue.

This fixes issue #744
2012-11-01 15:37:38 +01:00
Yecheng Fu ed44a74e72 fix typo in comments (redis.c, networking.c) 2012-11-01 18:14:55 +08:00
YAMAMOTO Takashiandantirez 47ff443b53 fix a typo in a comment 2012-10-31 09:29:56 +01:00
antirez 4d9bd53530 Invert two sides of if expression in SET to avoid a lookup.
Because of the short circuit behavior of && inverting the two sides of
the if expression avoids an hash table lookup if the non-EX variant of
SET is called.

Thanks to Weibin Yao (@yaoweibin on github) for spotting this.
2012-10-31 09:27:48 +01:00
antirez 5bf0997ff8 No longer used macro rdbIsOpcode() removed. 2012-10-30 19:10:56 +01:00
antirez ec0f483de2 help.h update (adds bitop, bitcount, evalsha...) 2012-10-30 18:57:28 +01:00
antirez 0dd6241314 Redis 2.6.2 2012-10-26 16:20:24 +02:00
antirez 3b1a41b172 Ctrl+w support in linenoise. 2012-10-26 16:06:36 +02:00
antirez 9b3b110004 Marginally more robust glibc version test for sync_file_range detection. 2012-10-26 15:55:10 +02:00
charsyamandantirez b25b0dc5f4 patch config.h for sync_file_range 2012-10-26 09:30:35 +02:00
antirez 2b5fc529a4 Redis 2.6.1 2012-10-25 22:07:27 +02:00
antirez be407c015c Fix compilation on Linux kernels or glibc versions lacking sync_file_range().
This fixes issue #667.

Many thanks to Didier Spezia for the fix.
2012-10-25 22:01:07 +02:00
antirez 1eb9145089 Update memory peak stats while loading RDB / AOF. 2012-10-24 12:21:34 +02:00
antirez 5eec376c2f Redis 2.6.0 2012-10-22 23:27:18 +02:00
antirez 99d7dbe669 A filed called slave_read_only added in INFO output.
This was an important information missing from the INFO output in the
replication section.

It obviously reflects if the slave is read only or not.
2012-10-22 19:22:48 +02:00
Greg Hurrellandantirez a61705dd95 Fix (cosmetic) typos in dict.h 2012-10-22 11:56:06 +02:00
Schusterandantirez 16144589f2 redis-check-dump now understands dumps produced by Redis 2.6
(Commit message from @antirez as it was missign in the original commits,
also the patch was modified a bit to still work with 2.4 dumps and to
avoid if expressions that are always true due to checked types range)

This commit changes redis-check-dump to account for new encodings and
for the new MSTIME expire format. It also refactors the test for valid
type into a function.

The code is still compatible with Redis 2.4 generated dumps.

This fixes issue #709.
2012-10-22 11:53:43 +02:00
antirez a25b25f4ef Default memory limit for 32bit instanced moved from 3.5 GB to 3 GB.
In some system, notably osx, the 3.5 GB limit was too far and not able
to prevent a crash for out of memory. The 3 GB limit works better and it
is still a lot of memory within a 4 GB theorical limit so it's not going
to bore anyone :-)

This fixes issue #711
2012-10-22 10:45:55 +02:00
antirez ab55180883 Differentiate SCRIPT KILL error replies.
When calling SCRIPT KILL currently you can get two errors:

* No script in timeout (busy) state.
* The script already performed a write.

It is useful to be able to distinguish the two errors, but right now both
start with "ERR" prefix, so string matching (that is fragile) must be used.

This commit introduces two different prefixes.

-NOTBUSY and -UNKILLABLE respectively to reply with an error when no
script is busy at the moment, and when the script already executed a
write operation and can not be killed.
2012-10-22 10:31:46 +02:00
NanXiaoandantirez a03c32702b Update src/redis-benchmark.c
The code of current implementation:

if (c->pending == 0) clientDone(c);
In clientDone function, the c's memory has been freed, then the loop will continue: while(c->pending). The memory of c has been freed now, so c->pending is invalid (c is an invalid pointer now), and this will cause memory dump in some platforams(eg: Solaris).

So I think the code should be modified as:
if (c->pending == 0)
{
clientDone(c);
break;
}
and this will not lead to while(c->pending).
2012-10-18 11:05:47 +02:00
antirez 2164523244 Fix MULTI / EXEC rendering in MONITOR output.
Before of this commit it used to be like this:

MULTI
EXEC
... actual commands of the transaction ...

Because after all that is the natural order of things. Transaction
commands are queued and executed *only after* EXEC is called.

However this makes debugging with MONITOR a mess, so the code was
modified to provide a coherent output.

What happens is that MULTI is rendered in the MONITOR output as far as
possible, instead EXEC is propagated only after the transaction is
executed, or even in the case it fails because of WATCH, so in this case
you'll simply see:

MULTI
EXEC

An empty transaction.
2012-10-16 17:41:39 +02:00
antirez c3ff470889 Merge remote-tracking branch 'origin/2.6' into 2.6 2012-10-11 18:36:18 +02:00
antirez 0e25c0ccf4 Allow AUTH when Redis is busy because of timedout Lua script.
If the server is password protected we need to accept AUTH when there is
a server busy (-BUSY) condition, otherwise it will be impossible to send
SHUTDOWN NOSAVE or SCRIPT KILL.

This fixes issue #708.
2012-10-11 18:35:52 +02:00
antirez f4eb4b3352 CONTRIBUTING file updated. 2012-10-06 12:04:38 +02:00
antirez 500dddc264 Redis 2.6.0 RC8 (2.5.14) 2012-10-05 19:08:55 +02:00
Salvatore Sanfilippo a6305f13d5 Merge pull request #544 from dvirsky/2.6
fixed install script to rewrite the default config
2012-10-05 03:30:58 -07:00
antirez 99c3338c23 Hash function switched to murmurhash2.
The previously used hash function, djbhash, is not secure against
collision attacks even when the seed is randomized as there are simple
ways to find seed-independent collisions.

The new hash function appears to be safe (or much harder to exploit at
least) in this case, and has better distribution.

Better distribution does not always means that's better. For instance in
a fast benchmark with "DEBUG POPULATE 1000000" I obtained the following
results:

    1.6 seconds with djbhash
    2.0 seconds with murmurhash2

This is due to the fact that djbhash will hash objects that follow the
pattern `prefix:<id>` and where the id is numerically near, to near
buckets. This improves the locality.

However in other access patterns with keys that have no relation
murmurhash2 has some (apparently minimal) speed advantage.

On the other hand a better distribution should significantly
improve the quality of the distribution of elements returned with
dictGetRandomKey() that is used in SPOP, SRANDMEMBER, RANDOMKEY, and
other commands.

Everything considered, and under the suspect that this commit fixes a
security issue in Redis, we are switching to the new hash function.
If some serious speed regression will be found in the future we'll be able
to step back easiliy.

This commit fixes issue #663.
2012-10-05 11:16:40 +02:00
antirez 05e06e1543 Warn when configured maxmemory value seems odd.
This commit warns the user with a log at "warning" level if:

1) After the server startup the maxmemory limit was found to be < 1MB.
2) After a CONFIG SET command modifying the maxmemory setting the limit
is set to a value that is smaller than the currently used memory.

The behaviour of the Redis server is unmodified, and this wil not make
the CONFIG SET command or a wrong configuration in redis.conf less
likely to create problems, but at least this will make aware most users
about a possbile error they committed without resorting to external
help.

However no warning is issued if, as a result of loading the AOF or RDB
file, we are very near the maxmemory setting, or key eviction will be
needed in order to go under the specified maxmemory setting. The reason
is that in servers configured as a cache with an aggressive
maxmemory-policy most of the times restarting the server will cause this
condition to happen if persistence is not switched off.

This fixes issue #429.
2012-10-05 10:56:35 +02:00
antirez e2f2dab3ef Include time.h in ae.c as we now use time(). 2012-10-05 10:10:34 +02:00
Jokeaandantirez e480c279bc Force expire all timer events when system clock skew is detected.
When system time changes back, the timer will not worker properly
hence some core functionality of redis will stop working(e.g. replication,
bgsave, etc). See issue #633 for details.

The patch saves the previous time and when a system clock skew is detected,
it will force expire all timers.

Modiifed by @antirez: the previous time was moved into the eventLoop
structure to make sure the library is still thread safe as long as you
use different event loops into different threads (otherwise you need
some synchronization). More comments added about the reasoning at the
base of the patch, that's worth reporting here:

/* If the system clock is moved to the future, and then set back to the
 * right value, time events may be delayed in a random way. Often this
 * means that scheduled operations will not be performed soon enough.
 *
 * Here we try to detect system clock skews, and force all the time
 * events to be processed ASAP when this happens: the idea is that
 * processing events earlier is less dangerous than delaying them
 * indefinitely, and practice suggests it is. */
2012-10-04 19:30:48 +02:00
antirez 0c19880ce6 "Timeout receiving bulk data" error message modified.
The new message now contains an hint about modifying the repl-timeout
configuration directive if the problem persists.

This should normally not be needed, because while the master generates
the RDB file it makes sure to send newlines to the replication channel
to prevent timeouts. However there are times when masters running on
very slow systems can completely stop for seconds during the RDB saving
process. In such a case enlarging the timeout value can fix the problem.

See issue #695 for an example of this problem in an EC2 deployment.
2012-10-04 11:49:17 +02:00
antirez 2ba962714a "SORT by nosort" (skip sorting) respect sorted set ordering.
When SORT is called with the option BY set to a string constant not
inclduing the wildcard character "*", there is no way to sort the output
so any ordering is valid. This allows the SORT internals to optimize its
work and don't really sort the output at all.

However it was odd that this option was not able to retain the natural
order of a sorted set. This feature was requested by users multiple
times as sometimes to call SORT with GET against sorted sets as a way to
mass-fetch objects can be handy.

This commit introduces two things:

1) The ability of SORT to return sorted sets elements in their natural
ordering when `BY nosort` is specified, accordingly to `DESC / ASC` options.
2) The ability of SORT to optimize this case further if LIMIT is passed
as well, avoiding to really fetch the whole sorted set, but directly
obtaining the specified range.

Because in this case the sorting is always deterministic, no
post-sorting activity is performed when SORT is called from a Lua
script.

This commit fixes issue #98.
2012-10-03 18:46:48 +02:00
antirez 151b606c11 Revert "Scripting: redis.NIL to return nil bulk replies."
This reverts commit e061d797d7.

Conflicts:

	src/scripting.c
2012-10-01 10:10:03 +02:00
antirez f1466e11cd Scripting: add helper functions redis.error_reply() and redis.status_reply().
A previous commit introduced Redis.NIL. This commit adds similar helper
functions to return tables with a single field set to the specified
string so that instead of using 'return {err="My Error"}' it is possible
to use a more idiomatic form:

    return redis.error_reply("My Error")
    return redis.status_reply("OK")
2012-09-28 16:58:38 +02:00
antirez e061d797d7 Scripting: redis.NIL to return nil bulk replies.
Lua arrays can't contain nil elements (see
http://www.lua.org/pil/19.1.html for more information), so Lua scripts
were not able to return a multi-bulk reply containing nil bulk
elements inside.

This commit introduces a special conversion: a table with just
a "nilbulk" field set to a boolean value is converted by Redis as a nil
bulk reply, but at the same time for Lua this type is not a "nil" so can
be used inside Lua arrays.

This type is also assigned to redis.NIL, so the following two forms
are equivalent and will be able to return a nil bulk reply as second
element of a three elements array:

    EVAL "return {1,redis.NIL,3}" 0
    EVAL "return {1,{nilbulk=true},3}" 0

The result in redis-cli will be:

    1) (integer) 1
    2) (nil)
    3) (integer) 3
2012-09-28 14:19:15 +02:00
Erik Dubbelboerandantirez 04779bdfe1 Fixed some spelling errors in the comments 2012-09-27 13:22:42 +02:00
Erik Dubbelboerandantirez e04be06e89 Added consts keyword where possible 2012-09-27 13:12:05 +02:00
antirez c4cbffa3a1 Final merge of Sentinel into 2.6.
After cherry-picking Sentinel commits a few spurious issues remained
about references to Redis Cluster that is not present in the 2.6 branch.
2012-09-27 13:10:36 +02:00
antirez dfb7194cba Sentinel: Support for AUTH. 2012-09-27 13:06:17 +02:00
antirez b8ce9a84c5 Sentinel: reply -IDONTKNOW to get-master-addr-by-name on lack of info.
If we don't have any clue about a master since it never replied to INFO
so far, reply with an -IDONTKNOW error to SENTINEL
get-master-addr-by-name requests.
2012-09-27 13:06:12 +02:00
antirez 1f8bd82332 Sentinel: more easy master redirection if master is a slave.
Before this commit Sentienl used to redirect master ip/addr if the
current instance reported to be a slave only if this was the first INFO
output received, and the role was found to be slave.

Now instead also if we find that the runid is different, and the
reported role is slave, we also redirect to the reported master ip/addr.

This unifies the behavior of Sentinel in the case of a reboot (where it
will see the first INFO output with the wrong role and will perform the
redirection), with the behavior of Sentinel in the case of a change in
what it sees in the INFO output of the master.
2012-09-27 13:06:05 +02:00
antirez ef792fc950 Sentinel: do not crash against slaves not publishing the runid.
Older versions of Redis (before 2.4.17) don't publish the runid field in
INFO. This commit makes Sentinel able to handle that without crashing.
2012-09-27 13:06:01 +02:00
antirez de499f7f7e Sentinel: INFO command implementation. 2012-09-27 13:05:58 +02:00
antirez b65f3c2176 Sentinel: add Redis execution mode to INFO output.
The new "redis_mode" field in the INFO output will show if Redis is
running in standalone mode, cluster, or sentinel mode.
2012-09-27 13:05:53 +02:00
antirez 161e137c55 Sentinel: Sentinel-side support for slave priority.
The slave priority that is now published by Redis in INFO output is
now used by Sentinel in order to select the slave with minimum priority
for promotion, and in order to consider slaves with priority set to 0 as
not able to play the role of master (they will never be promoted by
Sentinel).

The "slave-priority" field is now one of the fileds that Sentinel
publishes when describing an instance via the SENTINEL commands such as
"SENTINEL slaves mastername".
2012-09-27 13:05:49 +02:00
antirez d480b9ce7f Sentinel: suppress harmless warning by initializing 'table' to NULL.
Note that the assertion guarantees that one of the if branches setting
table is always entered.
2012-09-27 13:05:45 +02:00
antirez fa23fc3363 Sentinel: send SCRIPT KILL on -BUSY reply and SDOWN instance.
From the point of view of Redis an instance replying -BUSY is down,
since it is effectively not able to reply to user requests. However
a looping script is a recoverable condition in Redis if the script still
did not performed any write to the dataset. In that case performing a
fail over is not optimal, so Sentinel now tries to restore the normal server
condition killing the script with a SCRIPT KILL command.

If the script already performed some write before entering an infinite
(or long enough to timeout) loop, SCRIPT KILL will not work and the
fail over will be triggered anyway.
2012-09-27 13:05:41 +02:00
antirez fc0a0d4aa7 Sentinel: fixed a crash on script execution.
The call to sentinelScheduleScriptExecution() lacked the final NULL
argument to signal the end of arguments. This resulted into a crash.
2012-09-27 13:05:38 +02:00
antirez ea9bec50c6 Sentinel: SENTINEL FAILOVER command implemented.
This command can be used in order to force a Sentinel instance to start
a failover for the specified master, as leader, forcing the failover
even if the master is up.

The commit also adds some minor refactoring and other improvements to
functions already implemented that make them able to work when the
master is not in SDOWN condition. For instance slave selection
assumed that we ask INFO every second to every slave, this is true
only when the master is in SDOWN condition, so slave selection did not
worked when the master was not in SDOWN condition.
2012-09-27 13:05:33 +02:00
antirez 26a340095d Sentinel: client reconfiguration script execution.
This commit adds support to optionally execute a script when one of the
following events happen:

* The failover starts (with a slave already promoted).
* The failover ends.
* The failover is aborted.

The script is called with enough parameters (documented in the example
sentinel.conf file) to provide information about the old and new ip:port
pair of the master, the role of the sentinel (leader or observer) and
the name of the master.

The goal of the script is to inform clients of the configuration change
in a way specific to the environment Sentinel is running, that can't be
implemented in a genereal way inside Sentinel itself.
2012-09-27 13:05:30 +02:00
antirez 524b79d231 Sentinel: when leader in wait-start, sense another leader as race.
When we are in wait start, if another leader (or any other external
entity) turns a slave into a master, abort the failover, and detect it
as an observer.

Note that the wait-start state is mainly there for this reason but the
abort was yet not implemented.

This adds a new sentinel event -failover-abort-race.
2012-09-27 13:05:26 +02:00
antirez 201ed6d42e Sentinel: sentinelRefreshInstanceInfo() comments improved a bit. 2012-09-27 13:05:19 +02:00
antirez 7c9bfe1036 Sentinel: sentinel.conf self-documenation improved. 2012-09-27 13:05:15 +02:00
antirez 3da75e2ca4 Sentinel: abort failover when in wait-start if master is back.
When we are a Leader Sentinel in wait-start state, starting with this
commit the failover is aborted if the master returns online.

This improves the way we handle a notable case of net split, that is the
split between Sentinels and Redis servers, that will be a very common
case of split becase Sentinels will often be installed in the client's
network and servers can be in a differnt arm of the network.

When Sentinels and Redis servers are isolated the master is in ODOWN
condition since the Sentinels can agree about this state, however the
failover does not start since there are no good slaves to promote (in
this specific case all the slaves are unreachable).

However when the split is resolved, Sentinels may sense the slave back
a moment before they sense the master is back, so the failover may start
without a good reason (since the master is actually working too).

Now this condition is reversible, so the failover will be aborted
immediately after if the master is detected to be working again, that
is, not in SDOWN nor in ODOWN condition.
2012-09-27 13:05:11 +02:00
antirez e328e41a3a Sentinel: scripts execution engine improved.
We no longer use a vanilla fork+execve but take a queue of jobs of
scripts to execute, with retry on error, timeouts, and so forth.

Currently this is used only for notifications but soon the ability to
also call clients reconfiguration scripts will be added.
2012-09-27 13:05:07 +02:00
Jan-Erik Redigerandantirez 8a8e560b87 Include sys/wait.h to avoid compiler warning
gcc warned about an implicit declaration of function 'wait3'.
Including this header fixes this.
2012-09-27 13:04:51 +02:00
antirez 0d0975f2b6 Sentinel: don't start a failover as leader if there is no good slave. 2012-09-27 13:04:48 +02:00
Jeremy Zawodnyandantirez af41f6cffb comment fix
improve English a bit. :-)
2012-09-27 13:04:44 +02:00
antirez 999fe0d352 Sentinel: ability to execute notification scripts. 2012-09-27 13:04:41 +02:00
mrbandantirez f1057534e7 Fix warning in redis.c for sentinel config load 2012-09-27 13:04:37 +02:00
mrbandantirez fcc8bf9974 Some cleanup in sentinel.conf 2012-09-27 13:04:32 +02:00
antirez 374eed7d2a Sentinel: abort failover if no good slave is available.
The previous behavior of the state machine was to wait some time and
retry the slave selection, but this is not robust enough against drastic
changes in the conditions of the monitored instances.

What we do now when the slave selection fails is to abort the failover
and return back monitoring the master. If the ODOWN condition is still
present a new failover will be triggered and so forth.

This commit also refactors the code we use to abort a failover.
2012-09-27 13:04:28 +02:00
antirez 2085fdb1f4 Sentinel: reset pending_commands in a more generic way. 2012-09-27 13:04:24 +02:00
antirez f8a19e32e2 Prevent a spurious +sdown event on switch.
When we reset the master we should start with clean timestamps for ping
replies otherwise we'll detect a spurious +sdown event, because on
+master-switch event the previous master instance was probably in +sdown
condition. Since we updated the address we should count time from
scratch again.

Also this commit makes sure to explicitly reset the count of pending
commands, now we can do this because of the new way the hiredis link
is closed.
2012-09-27 13:04:19 +02:00
antirez 7c39b55d42 Sentinel: debugging message removed. 2012-09-27 13:04:16 +02:00
antirez e47236d8d4 Sentinel: changes to connection handling and redirection.
We disconnect the Redis instances hiredis link in a more robust way now.
Also we change the way we perform the redirection for the +switch-master
event, that is not just an instance reset with an address change.

Using the same system we now implement the +redirect-to-master event
that is triggered by an instance that is configured to be master but
found to be a slave at the first INFO reply. In that case we monitor the
master instead, logging the incident as an event.
2012-09-27 13:04:12 +02:00
antirez 8ab7e998d1 Sentinel: check that instance still exists in reply callbacks.
We can't be sure the instance object still exists when the reply
callback is called.
2012-09-27 13:04:08 +02:00
antirez e01a415d37 Sentinel: more robust failover detection as observer.
Sentinel observers detect failover checking if a slave attached to the
monitored master turns into its replication state from slave to master.
However while this change may in theory only happen after a SLAVEOF NO
ONE command, in practie it is very easy to reboot a slave instance with
a wrong configuration that turns it into a master, especially if it was
a past master before a successfull failover.

This commit changes the detection policy so that if an instance goes
from slave to master, but at the same time the runid has changed, we
sense a reboot, and in that case we don't detect a failover at all.

This commit also introduces the "reboot" sentinel event, that is logged
at "warning" level (so this will trigger an admin notification).

The commit also fixes a problem in the disconnect handler that assumed
that the instance object always existed, that is not the case. Now we
no longer assume that redisAsyncFree() will call the disconnection
handler before returning.
2012-09-27 13:04:02 +02:00
antirez d26a8fb4db Fixed an error in the example sentinel.conf. 2012-09-27 13:03:55 +02:00
antirez 5b5eb192f5 Typo. 2012-09-27 13:03:49 +02:00
antirez 120ba3922d First implementation of Redis Sentinel.
This commit implements the first, beta quality implementation of Redis
Sentinel, a distributed monitoring system for Redis with notification
and automatic failover capabilities.

More info at http://redis.io/topics/sentinel
2012-09-27 13:03:41 +02:00
antirez 2812b945f0 Test for SRANDMEMBER with <count>. 2012-09-21 11:56:04 +02:00
antirez 31fe053a62 SRANDMEMBER <count> leak fixed.
For "CASE 4" (see code) we need to free the element if it's already in
the result dictionary and adding it failed.
2012-09-21 11:56:00 +02:00
antirez dd94771578 Added the SRANDMEMBER key <count> variant.
SRANDMEMBER called with just the key argument can just return a single
random element from a Redis Set. However many users need to return
multiple unique elements from a Set, this is not a trivial problem to
handle in the client side, and for truly good performance a C
implementation was required.

After many requests for this feature it was finally implemented.

The problem implementing this command is the strategy to follow when
the number of elements the user asks for is near to the number of
elements that are already inside the set. In this case asking random
elements to the dictionary API, and trying to add it to a temporary set,
may result into an extremely poor performance, as most add operations
will be wasted on duplicated elements.

For this reason this implementation uses a different strategy in this
case: the Set is copied, and random elements are returned to reach the
specified count.

The code actually uses 4 different algorithms optimized for the
different cases.

If the count is negative, the command changes behavior and allows for
duplicated elements in the returned subset.
2012-09-21 11:55:56 +02:00
antirez 8b6b1b27cc Fix compilation on FreeBSD. Thanks to @koobs on twitter. 2012-09-17 12:45:57 +02:00
antirez 4403862623 Redis 2.5.13 (2.6.0 RC7). 2012-09-17 11:02:49 +02:00
antirez 174518ffb7 .gitignore modified to be more general with less entries. 2012-09-17 10:50:07 +02:00
antirez f444e2afdc A reimplementation of blocking operation internals.
Redis provides support for blocking operations such as BLPOP or BRPOP.
This operations are identical to normal LPOP and RPOP operations as long
as there are elements in the target list, but if the list is empty they
block waiting for new data to arrive to the list.

All the clients blocked waiting for th same list are served in a FIFO
way, so the first that blocked is the first to be served when there is
more data pushed by another client into the list.

The previous implementation of blocking operations was conceived to
serve clients in the context of push operations. For for instance:

1) There is a client "A" blocked on list "foo".
2) The client "B" performs `LPUSH foo somevalue`.
3) The client "A" is served in the context of the "B" LPUSH,
synchronously.

Processing things in a synchronous way was useful as if "A" pushes a
value that is served by "B", from the point of view of the database is a
NOP (no operation) thing, that is, nothing is replicated, nothing is
written in the AOF file, and so forth.

However later we implemented two things:

1) Variadic LPUSH that could add multiple values to a list in the
context of a single call.
2) BRPOPLPUSH that was a version of BRPOP that also provided a "PUSH"
side effect when receiving data.

This forced us to make the synchronous implementation more complex. If
client "B" is waiting for data, and "A" pushes three elemnents in a
single call, we needed to propagate an LPUSH with a missing argument
in the AOF and replication link. We also needed to make sure to
replicate the LPUSH side of BRPOPLPUSH, but only if in turn did not
happened to serve another blocking client into another list ;)

This were complex but with a few of mutually recursive functions
everything worked as expected... until one day we introduced scripting
in Redis.

Scripting + synchronous blocking operations = Issue #614.

Basically you can't "rewrite" a script to have just a partial effect on
the replicas and AOF file if the script happened to serve a few blocked
clients.

The solution to all this problems, implemented by this commit, is to
change the way we serve blocked clients. Instead of serving the blocked
clients synchronously, in the context of the command performing the PUSH
operation, it is now an asynchronous and iterative process:

1) If a key that has clients blocked waiting for data is the subject of
a list push operation, We simply mark keys as "ready" and put it into a
queue.
2) Every command pushing stuff on lists, as a variadic LPUSH, a script,
or whatever it is, is replicated verbatim without any rewriting.
3) Every time a Redis command, a MULTI/EXEC block, or a script,
completed its execution, we run the list of keys ready to serve blocked
clients (as more data arrived), and process this list serving the
blocked clients.
4) As a result of "3" maybe more keys are ready again for other clients
(as a result of BRPOPLPUSH we may have push operations), so we iterate
back to step "3" if it's needed.

The new code has a much simpler semantics, and a simpler to understand
implementation, with the disadvantage of not being able to "optmize out"
a PUSH+BPOP as a No OP.

This commit will be tested with care before the final merge, more tests
will be added likely.
2012-09-17 10:26:50 +02:00
antirez b58f03a0e8 Make sure that SELECT argument is an integer or return an error.
Unfortunately we had still the lame atoi() without any error checking in
place, so "SELECT foo" would work as "SELECT 0". This was not an huge
problem per se but some people expected that DB can be strings and not
just numbers, and without errors you get the feeling that they can be
numbers, but not the behavior.

Now getLongFromObjectOrReply() is used as almost everybody else across
the code, generating an error if the number is not an integer or
overflows the long type.

Thanks to @mipearson for reporting that on Twitter.
2012-09-11 10:34:48 +02:00
antirez efb54f0593 Match printf format with actual type in genRedisInfoString(). 2012-09-10 12:43:20 +02:00
antirez 58889867bb BITCOUNT regression test for #582 fixed for 32 bit target.
Bug #582 was not present in 32 bit builds of Redis as
getObjectFromLong() will return an error for overflow.

This commit makes sure that the test does not fail because of the error
returned when running against 32 bit builds.
2012-09-05 17:50:24 +02:00
Haruto Otakeandantirez 4c3d419013 BITCOUNT: fix segmentation fault.
remove unsafe and unnecessary cast.
until now, this cast may lead segmentation fault when end > UINT_MAX

setbit foo 0 1
bitcount  0 4294967295
=> ok
bitcount  0 4294967296
=> cause segmentation fault.

Note by @antirez: the commit was modified a bit to also change the
string length type to long, since it's guaranteed to be at max 512 MB in
size, so we can work with the same type across all the code path.

A regression test was also added.
2012-09-05 16:20:21 +02:00
Saj Goonatillekeandantirez 0671d88cab Bug fix: slaves being pinged every second
REDIS_REPL_PING_SLAVE_PERIOD controls how often the master should
transmit a heartbeat (PING) to its slaves.  This period, which defaults
to 10, is measured in seconds.

Redis 2.4 masters used to ping their slaves every ten seconds, just like
it says on the tin.

The Redis 2.6 masters I have been experimenting with, on the other hand,
ping their slaves *every second*.  (master_last_io_seconds_ago never
approaches 10.)  I think the ping period was inadvertently slashed to
one-tenth of its nominal value around the time REDIS_HZ was introduced.
This commit reintroduces correct ping schedule behaviour.
2012-09-05 16:01:01 +02:00
antirez 5ddee9b7d5 Scripting: Force SORT BY constant determinism inside SORT itself.
SORT is able to return (faster than when ordering) unordered output if
the "BY" clause is used with a constant value. However we try to play
well with scripting requirements of determinism providing always sorted
outputs when SORT (and other similar commands) are called by Lua
scripts.

However we used the general mechanism in place in scripting in order to
reorder SORT output, that is, if the command has the "S" flag set, the
Lua scripting engine will take an additional step when converting a
multi bulk reply to Lua value, calling a Lua sorting function.

This is suboptimal as we can do it faster inside SORT itself.
This is also broken as issue #545 shows us: basically when SORT is used
with a constant BY, and additionally also GET is used, the Lua scripting
engine was trying to order the output as a flat array, while it was
actually a list of key-value pairs.

What we do know is to recognized if the caller of SORT is the Lua client
(since we can check this using the REDIS_LUA_CLIENT flag). If so, and if
a "don't sort" condition is triggered by the BY option with a constant
string, we force the lexicographical sorting.

This commit fixes this bug and improves the performance, and at the same
time simplifies the implementation. This does not mean I'm smart today,
it means I was stupid when I committed the original implementation ;)
2012-09-05 01:19:47 +02:00
antirez fd2a8951bf Send an async PING before starting replication with master.
During the first synchronization step of the replication process, a Redis
slave connects with the master in a non blocking way. However once the
connection is established the replication continues sending the REPLCONF
command, and sometimes the AUTH command if needed. Those commands are
send in a partially blocking way (blocking with timeout in the order of
seconds).

Because it is common for a blocked master to accept connections even if
it is actually not able to reply to the slave requests, it was easy for
a slave to block if the master had serious issues, but was still able to
accept connections in the listening socket.

For this reason we now send an asynchronous PING request just after the
non blocking connection ended in a successful way, and wait for the
reply before to continue with the replication process. It is very
unlikely that a master replying to PING can't reply to the other
commands.

This solution was proposed by Didier Spezia (Thanks!) so that we don't
need to turn all the replication process into a non blocking affair, but
still the probability of a slave blocked is minimal even in the event of
a failing master.

Also we now use getsockopt(SO_ERROR) in order to check errors ASAP
in the event handler, instead of waiting for actual I/O to return an
error.

This commit fixes issue #632.
2012-09-03 11:48:27 +02:00
antirez 42a239b888 Scripting: Reset Lua fake client reply_bytes after command execution.
Lua scripting uses a fake client in order to run commands in the context
of a client, accumulate the reply, and convert it into a Lua object
to return to the caller. This client is reused again and again, and is
referenced by the server.lua_client globally accessible pointer.

However after every call to redis.call() or redis.pcall(), that is
handled by the luaRedisGenericCommand() function, the reply_bytes field
of the client was not set back to zero. This filed is used to estimate
the amount of memory currently used in the reply. Because of the lack of
reset, script after script executed, this value used to get bigger and
bigger, and in the end on 32 bit systems it triggered the following
assert:

    redisAssert(c->reply_bytes < ULONG_MAX-(1024*64));

On 64 bit systems this does not happen because it takes too much time to
reach values near to 2^64 for users to see the practical effect of the
bug.

Now in the cleanup stage of luaRedisGenericCommand() we reset the
reply_bytes counter to zero, avoiding the issue. It is not practical to
add a test for this bug, but the fix was manually tested using a
debugger.

This commit fixes issue #656.
2012-08-31 11:08:53 +02:00
antirez 851ac9d072 Sentinel: added documentation about slave-priority in redis.conf 2012-08-31 10:30:51 +02:00
antirez 48d26a483d Sentinel: Redis-side support for slave priority.
A Redis slave can now be configured with a priority, that is an integer
number that is shown in INFO output and can be get and set using the
redis.conf file or the CONFIG GET/SET command.

This field is used by Sentinel during slave election. A slave with lower
priority is preferred. A slave with priority zero is never elected (and
is considered to be impossible to elect even if it is the only slave
available).

A next commit will add support in the Sentinel side as well.
2012-08-31 10:30:29 +02:00
antirez edfaa64f49 Scripting: require at least one argument for redis.call().
Redis used to crash with a call like the following:

    EVAL "redis.call()" 0

Now the explicit check for at least one argument prevents the problem.

This commit fixes issue #655.
2012-08-31 10:28:36 +02:00
antirez 13732168a5 Incrementally flush RDB on disk while loading it from a master.
This fixes issue #539.

Basically if there is enough free memory the OS may buffer the RDB file
that the slave transfers on disk from the master. The file may
actually be flused on disk at once by the operating system when it gets
closed by Redis, causing the close system call to block for a long time.

This patch is a modified version of one provided by yoav-steinberg of
@garantiadata (the original version was posted in the issue #539
comments), and tries to flush the OS buffers incrementally (every 8 MB
of loaded data).
2012-08-28 12:47:35 +02:00
antirez 06bd3b9acd Fix a forget zmalloc_oom() -> zmalloc_oom_handler() replacement. 2012-08-24 15:41:49 +02:00
antirez 5de75120ba Better Out of Memory handling.
The previous implementation of zmalloc.c was not able to handle out of
memory in an application-specific way. It just logged an error on
standard error, and aborted.

The result was that in the case of an actual out of memory in Redis
where malloc returned NULL (In Linux this actually happens under
specific overcommit policy settings and/or with no or little swap
configured) the error was not properly logged in the Redis log.

This commit fixes this problem, fixing issue #509.
Now the out of memory is properly reported in the Redis log and a stack
trace is generated.

The approach used is to provide a configurable out of memory handler
to zmalloc (otherwise the default one logging the event on the
standard output is used).
2012-08-24 13:03:40 +02:00
antirez 32095c4057 redis-benchmark: disable big buffer cleanup in hiredis context.
This new hiredis features allows us to reuse a previous context reader
buffer even if already very big in order to maximize performances with
big payloads (Usually hiredis re-creates buffers when they are too big
and unused in order to save memory).
2012-08-22 11:34:03 +02:00
antirez 7fcba9fd9a hiredis library updated.
This version of hiredis merges modifications of the Redis fork with
latest changes in the hiredis repository.

The same version was pushed on the hiredis repository and will probably
merged into the master branch in short time.
2012-08-22 11:33:57 +02:00
Pieter Noordhuisandantirez 2f44452612 Set p to its new offset before modifying it 2012-08-22 11:33:52 +02:00
Pieter Noordhuisandantirez 89bf6f58fd Add ziplist test for deleting next to last entries 2012-08-22 11:33:47 +02:00
Tobias Schwabandantirez 013189e7db Fix version numbers 2012-08-02 14:38:42 +02:00
antirez 73d3e8751b Redis 2.5.12 (2.6 RC6). 2012-08-01 12:06:03 +02:00
Michael Parkerandantirez 628890e43e Use correct variable name for value to convert.
Note by @antirez: this code was never compiled because utils.c lacked the
float.h include, so we never noticed this variable was mispelled in the
past.

This should provide a noticeable speed boost when saving certain types
of databases with many sorted sets inside.
2012-07-31 11:50:51 +02:00
Saj Goonatillekeandantirez 4c0c1fff5a Truncate short write from the AOF
If Redis only manages to write out a partial buffer, the AOF file won't
load back into Redis the next time it starts up.  It is better to
discard the short write than waste time running redis-check-aof.
2012-07-31 10:58:16 +02:00
Saj Goonatillekeandantirez f00b0844c9 New in INFO: aof_last_bgrewrite_status
Behaves like rdb_last_bgsave_status -- even down to reporting 'ok' when
no rewrite has been done yet.  (You might want to check that
aof_last_rewrite_time_sec is not -1.)
2012-07-31 10:58:12 +02:00
Steeve Lennmarkandantirez 889e443ce5 Check that we have connection before enabling pipe mode 2012-07-22 17:19:05 +02:00
antirez 3bb3f12539 Allow Pub/Sub in contexts where other commands are blocked.
Redis loading data from disk, and a Redis slave disconnected from its
master with serve-stale-data disabled, are two conditions where
commands are normally refused by Redis, returning an error.

However there is no reason to disable Pub/Sub commands as well, given
that this layer does not interact with the dataset. To allow Pub/Sub in
as many contexts as possible is especially interesting now that Redis
Sentinel uses Pub/Sub of a Redis master as a communication channel
between Sentinels.

This commit allows Pub/Sub to be used in the above two contexts where
it was previously denied.
2012-07-22 17:18:12 +02:00
antirez 82675c86a6 Don't assume that "char" is signed.
For the C standard char can be either signed or unsigned, it's up to the
compiler, but Redis assumed that it was signed in a few places.

The practical effect of this patch is that now Redis 2.6 will run
correctly in every system where char is unsigned, notably the RaspBerry
PI and other ARM systems with GCC.

Thanks to Georgi Marinov (@eesn on twitter) that reported the problem
and allowed me to use his RaspBerry via SSH to trace and fix the issue!
2012-07-18 12:01:43 +02:00
jokeaandantirez 8a8e01f4a7 mark fd as writable when EPOLLERR or EPOLLHUP is returned by epoll_wait. 2012-07-09 12:15:07 +02:00
antirez d3d567428a Typo in comment. 2012-07-07 17:24:40 +02:00
antirez dbd8c753c4 REPLCONF internal command introduced.
The REPLCONF command is an internal command (not designed to be directly
used by normal clients) that allows a slave to set some replication
related state in the master before issuing SYNC to start the
replication.

The initial motivation for this command, and the only reason currently
it is used by the implementation, is to let the slave instance
communicate its listening port to the slave, so that the master can
show all the slaves with their listening ports in the "replication"
section of the INFO output.

This allows clients to auto discover and query all the slaves attached
into a master.

Currently only a single option of the REPLCONF command is supported, and
it is called "listening-port", so the slave now starts the replication
process with something like the following chat:

    REPLCONF listening-prot 6380
    SYNC

Note that this works even if the master is an older version of Redis and
does not understand REPLCONF, because the slave ignores the REPLCONF
error.

In the future REPLCONF can be used for partial replication and other
replication related features where there is the need to exchange
information between master and slave.

NOTE: This commit also fixes a bug: the INFO outout already carried
information about slaves, but the port was broken, and was obtained
with getpeername(2), so it was actually just the ephemeral port used
by the slave to connect to the master as a client.
2012-07-07 17:24:33 +02:00
antirez b3f28b90d2 Fixed comment typo into time_independent_strcmp(). 2012-06-21 14:26:00 +02:00
antirez 4b3865cbdb Fixed a timing attack on AUTH (Issue #560).
The way we compared the authentication password using strcmp() allowed
an attacker to gain information about the password using a well known
class of attacks called "timing attacks".

The bug appears to be practically not exploitable in most modern systems
running Redis since even using multiple bytes of differences in the
input at a time instead of one the difference in running time in in the
order of 10 nanoseconds, making it hard to exploit even on LAN. However
attacks always get better so we are providing a fix ASAP.

The new implementation uses two fixed length buffers and a constant time
comparison function, with the goal of:

1) Completely avoid leaking information about the content of the
password, since the comparison is always performed between 512
characters and without conditionals.
2) Partially avoid leaking information about the length of the
password.

About "2" we still have a stage in the code where the real password and
the user provided password are copied in the static buffers, we also run
two strlen() operations against the two inputs, so the running time
of the comparison is a fixed amount plus a time proportional to
LENGTH(A)+LENGTH(B). This means that the absolute time of the operation
performed is still related to the length of the password in some way,
but there is no way to change the input in order to get a difference in
the execution time in the comparison that is not just proportional to
the string provided by the user (because the password length is fixed).

Thus in practical terms the user should try to discover LENGTH(PASSWORD)
looking at the whole execution time of the AUTH command and trying to
guess a proportionality between the whole execution time and the
password length: this appears to be mostly unfeasible in the real world.

Also protecting from this attack is not very useful in the case of Redis
as a brute force attack is anyway feasible if the password is too short,
while with a long password makes it not an issue that the attacker knows
the length.
2012-06-21 12:01:07 +02:00
antirez 0c9cf45270 Redis 2.5.11 (2.6 RC5). 2012-06-15 13:44:17 +02:00
antirez 6fe9c402a2 Fix c->reply_bytes computation in setDeferredMultiBulkLength()
In order to implement reply buffer limits introduced in 2.6 and useful
to close the connection under user-selected circumastances of big output
buffers (for instance slow consumers in pub/sub, a blocked slave, and so
forth) Redis takes a counter with the amount of used memory in objects
inside the output list stored into c->reply.

The computation was broken in the function setDeferredMultiBulkLength(),
in the case the object was glued with the next one. This caused the
c->reply_bytes field to go out of sync, be subtracted more than needed,
and wrap back near to ULONG_MAX values.

This commit fixes this bug and adds an assertion that is able to trap
this class of problems.

This problem was discovered looking at the INFO output of an unrelated
issue (issue #547).
2012-06-15 10:11:27 +02:00
antirez 8361d6c406 ziplistFind(): don't assume that entries are comparable by encoding.
Because Redis 2.6 introduced new integer encodings it is no longer true
that if two entries have a different encoding they are not equal.

An old ziplist can be loaded from an RDB file generated with Redis 2.4,
in this case for instance a small unsigned integers is encoded with a
16 bit encoding, while in Redis 2.6 a more specific 8 bit encoding
format is used.

Because of this bug hashes ended with duplicated values or fields lookup
failed, causing many bad behaviors.
This in turn caused a crash while converting the ziplist encoded hash into
a real hash table because an assertion was raised on duplicated elements.

This commit fixes issue #547.

Many thanks to Pinterest's Marty Weiner and colleagues for discovering
the problem and helping us in the debugging process.
2012-06-15 10:11:23 +02:00
Ted Nymanandantirez e612508d38 Standardize punctuation in redis-cli help.
Right there is a mix of help entries ending with periods or
without periods. This standardizes the end of command as without
periods, which seems to be the general custom in most unix tools,
at least.
2012-06-13 10:27:23 +02:00
antirez 1364395f18 Added a new hash fuzzy tester.
The new fuzzy tester also removes elements from the hash instead of just
adding random fields. This should increase the probability to find bugs
in the implementations of the hash type internal representations.
2012-06-12 15:20:16 +02:00
antirez 0b8441c1b2 Dump ziplist hex value on failed assertion.
The ziplist -> hashtable conversion code is triggered every time an hash
value must be promoted to a full hash table because the number or size of
elements reached the threshold.

If a problem in the ziplist causes the same field to be present
multiple times, the assertion of successful addition of the element
inside the hash table will fail, crashing server with a failed
assertion, but providing little information about the problem.

This code adds a new logging function to perform the hex dump of binary
data, and makes sure that the ziplist -> hashtable conversion code uses
this new logging facility to dump the content of the ziplist when the
assertion fails.

This change was originally made in order to investigate issue #547.
2012-06-12 00:49:32 +02:00
antirez a85c89854f New test: hash ziplist -> hashtable encoding conversion.
A new stress test was added to stress test the code converting a ziplist
into an hash table.

In this commit also randomValue helper function was modified to also
return negative values.
2012-06-11 15:21:44 +02:00
dvir volk 0ea1a9c452 fixed server install script to rewrite the default configuration file and not a template, and removed the old config template 2012-06-08 16:03:18 +03:00
antirez 1a3e9d951b Version bumped to 2.5.10 (2.6 RC4) + Release Notes. 2012-06-07 11:41:38 +02:00
antirez 3f12656781 EVAL replication test: less false positives.
wait_for_condition is now used instead of the usual "after 1000" (that
is the way to sleep in Tcl). This should avoid to find the replica in
a state where it is loading the RDB in memory, returning -LOADING error.

This test used to fail when running the test over valgrind, due to the
added latencies.
2012-06-02 23:32:01 +02:00
Alex Mitrofanovandantirez 591c9e6543 Fixed RESTORE hash failure (Issue #532)
(additional commit notes by antirez@gmail.com):

The rdbIsObjectType() macro was not updated when the new RDB object type
of ziplist encoded hashes was added.

As a result RESTORE, that uses rdbLoadObjectType(), failed when a
ziplist encoded hash was loaded.
This does not affected normal RDB loading because in that case we use
the lower-level function rdbLoadType().

The commit also adds a regression test.
2012-06-02 10:27:31 +02:00
antirez ebdcd723a7 RDB type loading functions clarified in comments.
Improved comments to make clear that rdbLoadType() just loads a
general TYPE in the context of RDB that can be an object type or an
expire type, end-of-file, and so forth.

While rdbLoadObjectType() enforces that the type is a valid Object Type
otherwise it returns -1.
2012-06-02 10:27:27 +02:00
antirez 8f0658cdd0 BITOP bug when called against non existing keys fixed.
In the issue #529 an user reported a bug that can be triggered with the
following code:

flushdb
set a
"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
bitop or x a b

The bug was introduced with the speed optimization in commit 8bbc076
that specializes every BITOP operation loop up to the minimum length of
the input strings.

However the computation of the minimum length contained an error when a
non existing key was present in the input, after a key that was non zero
length.

This commit fixes the bug and adds a regression test for it.
2012-05-31 21:51:52 +02:00
antirez 9a8d51add5 2.6 Release notes: added more new features to the list. 2012-05-27 17:17:22 +02:00
antirez 473f3090f2 Tests modified to account for INFO fields renaming.
Commit 33e1db36fa modified the name of a
few INFO fields. This commit changes the Redis test to account for this
changes.
2012-05-25 16:25:07 +02:00
antirez 69396249e5 Release notes: more info about 2.4 -> 2.6 migration.
Migration information moved at the end of the file as now they start to
be too long to stay at the top, but a warning was added at the top of
the file to remember the user to check.

Added information about INFO fields that changed name between 2.4 and
2.6.
2012-05-25 12:24:20 +02:00
antirez 6b4d92e21d Four new persistence fields in INFO. A few renamed.
The 'persistence' section of INFO output now contains additional four
fields related to RDB and AOF persistence:

 rdb_last_bgsave_time_sec       Duration of latest BGSAVE in sec.
 rdb_current_bgsave_time_sec    Duration of current BGSAVE in sec.
 aof_last_rewrite_time_sec      Duration of latest AOF rewrite in sec.
 aof_current_rewrite_time_sec   Duration of current AOF rewrite in sec.

The 'current' fields are set to -1 if a BGSAVE / AOF rewrite is not in
progress. The 'last' fileds are set to -1 if no previous BGSAVE / AOF
rewrites were performed.

Additionally a few fields in the persistence section were renamed for
consistency:

 changes_since_last_save -> rdb_changes_since_last_save
 bgsave_in_progress -> rdb_bgsave_in_progress
 last_save_time -> rdb_last_save_time
 last_bgsave_status -> rdb_last_bgsave_status
 bgrewriteaof_in_progress -> aof_rewrite_in_progress
 bgrewriteaof_scheduled -> aof_rewrite_scheduled

After the renaming, fields in the persistence section start with rdb_ or
aof_ prefix depending on the persistence method they describe.
The field 'loading' and related fields are not prefixed because they are
unique for both the persistence methods.
2012-05-25 12:20:35 +02:00
antirez 8bbc0768b8 BITOP command 10x speed improvement.
This commit adds a fast-path to the BITOP that can be used for all the
bytes from 0 to the minimal length of the string, and if there are
at max 16 input keys.

Often the intersected bitmaps are roughly the same size, so this
optimization can provide a 10x speed boost to most real world usages
of the command.

Bytes are processed four full words at a time, in loops specialized
for the specific BITOP sub-command, without the need to check for
length issues with the inputs (since we run this algorithm only as far
as there is data from all the keys at the same time).

The remaining part of the string is intersected in the usual way using
the slow but generic algorith.

It is possible to do better than this with inputs that are not roughly
the same size, sorting the input keys by length, by initializing the
result string in a smarter way, and noticing that the final part of the
output string composed of only data from the longest string does not
need any proecessing since AND, OR and XOR against an empty string does
not alter the output (zero in the first case, and the original string in
the other two cases).

More implementations will be implemented later likely, but this should
be enough to release Redis 2.6-RC4 with bitops merged in.

Note: this commit also adds better testing for BITOP NOT command, that
is currently the faster and hard to optimize further since it just
flips the bits of a single input string.
2012-05-24 15:25:13 +02:00
antirez f0d962c0ec BITOP: handle integer encoded objects correctly.
A bug in the implementation caused BITOP to crash the server if at least
one one of the source objects was integer encoded.

The new implementation takes an additional array of Redis objects
pointers and calls getDecodedObject() to get a reference to a string
encoded object, and then uses decrRefCount() to release the object.

Tests modified to cover the regression and improve coverage.
2012-05-24 15:25:10 +02:00
antirez c19bfe9fcf BITCOUNT performance improved.
At Redis's default optimization level the command is now much faster,
always using a constant-time bit manipualtion technique to count bits
instead of GCC builtin popcount, and unrolling the loop.

The current implementation performance is 1.5GB/s in a MBA 11" (1.8 Ghz
i7) compiled with both GCC and clang.

The algorithm used is described here:

http://graphics.stanford.edu/~seander/bithacks.html
2012-05-24 15:25:06 +02:00
antirez e53a1fdd60 bitop.c renamed bitops.c
bitop.c contains the "Bit related string operations" so it seems more
logical to call it bitops instead of bitop.
This also makes it matching the name of the test (unit/bitops.tcl).
2012-05-24 15:24:57 +02:00
antirez 951213e2d2 Bit operations tests improved.
Fuzzing tests of BITCOUNT / BITOP are iterated multiple times.
The new BITCOUNT fuzzing test uses random strings in a wider interval of
lengths including zero-len strings.
2012-05-24 15:24:35 +02:00
antirez d12a68d415 popcount() optimization for speed.
We run the array by 32 bit words instead of processing it byte per byte.
If the code is compiled using GCC __builtin_popcount() builtin function
is used instead.
2012-05-24 15:24:31 +02:00
antirez 7ffa248c7c BITCOUNT refactoring.
The low level popualtion counting function is now separated from the
BITCOUNT command implementation, so that the low level function can be
further optimized and eventually used in other contexts if needed.
2012-05-24 15:24:27 +02:00
antirez 84ec6706cf Bit-related string operations moved to bitop.c
All the general string operations are implemented in t_string.c, however
the bit operations, while targeting the string type, are better served
in a specific file where we have the implementations of the following
four commands and helper functions:

    GETBIT
    SETBIT
    BITOP
    BITCOUNT

In the future this file will probably contain more code related to
making the BITOP and BITCOUNT operations faster.
2012-05-24 15:24:14 +02:00
antirez 1f40cdd0e5 BITOP and BITCOUNT tests.
The Redis implementation is tested against Tcl implementations of the
same operation. Both fuzzing and testing of specific aspects of the
commands behavior are performed.
2012-05-24 15:22:45 +02:00
antirez 4a789decaa New commands: BITOP and BITCOUNT.
The motivation for this new commands is to be search in the usage of
Redis for real time statistics. See the article "Fast real time metrics
using Redis".

http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-using-redis-bitmaps/

In general Redis strings when used as bitmaps using the SETBIT/GETBIT
command provide a very space-efficient and fast way to store statistics.
For instance in a web application with users, every user can be
associated with a key that shows every day in which the user visited the
web service. This information can be really valuable to extract user
behaviour information.

With Redis bitmaps doing this is very simple just saying that a given
day is 0 (the data the service was put online) and all the next days are
1, 2, 3, and so forth. So with SETBIT it is possible to set the bit
corresponding to the current day every time the user visits the site.

It is possible to take the count of the bit sets on the run, this is
extremely easy using a Lua script. However a fast bit count native
operation can be useful, especially if it can operate on ranges, or when
the string is small like in the case of days (even if you consider many
years it is still extremely little data).

For this reason BITOP was introduced. The command counts the number of
bits set to 1 in a string, with optional range:

BITCOUNT key [start end]

The start/end parameters are similar to GETRANGE. If omitted the whole
string is tested.

Population counting is more useful when bit-level operations like AND,
OR and XOR are avaialble. For instance I can test multiple users to see
the number of days three users visited the site at the same time. To do
this we can take the AND of all the bitmaps, and then count the set bits.

For this reason the BITOP command was introduced:

BITOP [AND|OR|XOR|NOT] dest_key src_key1 src_key2 src_key3 ... src_keyN

In the special case of NOT (that inverts the bits) only one source key
can be passed.

The judicious use of BITCOUNT and BITOP combined can lead to interesting
use cases with very space efficient representation of data.

The implementation provided is still not tested and optimized for speed,
next commits will introduce unit tests. Later the implementation will be
profiled to see if it is possible to gain an important amount of speed
without making the code much more complex.
2012-05-24 15:22:42 +02:00
antirez bec74b1b9c Add aof_rewrite_buffer_length INFO field.
The INFO output, persistence section, already contained the field
describing the size of the current AOF buffer to flush on disk. However
the other AOF buffer, used to accumulate changes during an AOF rewrite,
was not mentioned in the INFO output.

This commit introduces a new field called aof_rewrite_buffer_length with
the length of the rewrite buffer.
2012-05-24 15:22:39 +02:00
antirez edb952532d Allow an AOF rewrite buffer > 2GB (Fix for issue #504).
During the AOF rewrite process, the parent process needs to accumulate
the new writes in an in-memory buffer: when the child will terminate the
AOF rewriting process this buffer (that ist the difference between the
dataset when the rewrite was started, and the current dataset) is
flushed to the new AOF file.

We used to implement this buffer using an sds.c string, but sds.c has a
2GB limit. Sometimes the dataset can be big enough, the amount of writes
so high, and the rewrite process slow enough that we overflow the 2GB
limit, causing a crash, documented on github by issue #504.

In order to prevent this from happening, this commit introduces a new
system to accumulate writes, implemented by a linked list of blocks of
10 MB each, so that we also avoid paying the reallocation cost.

Note that theoretically modern operating systems may implement realloc()
simply as a remaping of the old pages, thus with very good performances,
see for instance the mremap() syscall on Linux. However this is not
always true, and jemalloc by default avoids doing this because there are
issues with the current implementation of mremap().

For this reason we are using a linked list of blocks instead of a single
block that gets reallocated again and again.

The changes in this commit lacks testing, that will be performed before
merging into the unstable branch. This fix will not enter 2.4 because it
is too invasive. However 2.4 will log a warning when the AOF rewrite
buffer is near to the 2GB limit.
2012-05-24 15:22:35 +02:00
antirez 8152d0c046 Dead code removed from replication.c.
The user @jokea noticed that the following line of code into
replication.c made little sense:

    addReplySds(slave,sdsempty());

Investigating a bit I found that this was introduced by commit 6208b3a7
three years ago in the early stages of Redis. The code apparently is not
useful at all, so I'm removing it.

This change will not be backported into 2.4 so that in the rare case
this should introduce a bug, we'll have a chance to detect it into the
development branch. However following the code path it seems like the
code is not useful at all, so the risk is truly small.
2012-05-24 15:22:31 +02:00
antirez 9b6035bc7f TODO file removed.
The list of things to do is since long time in two places:

1) Github issues.
2) I've a private TOOD list of random ideas, what makes sense is later
moved to github issues. So github is anyway the true source of things to
do.
2012-05-23 11:52:47 +02:00
antirez 27fc5bf582 Use comments to split aof.c into sections.
This makes the code more readable, it is still not the case to split the
file itself into three different files, but the logical separation
improves the readability especially since new commits are going to
introduce an additional section.
2012-05-23 11:52:40 +02:00
antirez dfa90b5969 Redis test: include bug report on crash.
Due to a change in the format of the bug report in case of crash of
failed assertion the test suite was no longer able to properly log it.
Instead just a protocol error was logged by the Redis TCL client that
provided no clue about the actual problem.

This commit resolves the issue by logging everything from the first line
of the log including the string REDIS BUG REPORT, till the end of the
file.
2012-05-23 11:52:35 +02:00
jokeaandantirez a5a037bf81 Set fd to writable when poll(2) detects POLLERR or POLLHUP event. 2012-05-23 11:33:25 +02:00
antirez 4dada1b5bc Fixed issue #516 (ZINTERSTORE mixing sets and zsets).
Weeks ago trying to fix an harmless GCC warning I introduced a bug in
the ziplist-encoded implementations of sorted sets.

The bug completely broke zuiNext() iterator, that is used in the
ZINTERSTORE and ZUNIONSTORE implementation, so those two commands are no
longer reliable starting from Redis version 2.4.12 and latest 2.6.0-RC
releases.

This commit fixes the problem and adds a regression test.
2012-05-23 11:12:38 +02:00
antirez afc6bd1b48 Deleted jemalloc.orig from /deps.
In the commit upgrading jemalloc to version 3.0.0 I added the old
version of Jemalloc in the 'jemalloc.orig' directory for an error.
This commit removes the not useful version of jemalloc.
2012-05-16 12:22:29 +02:00
antirez 4934f93dfb Jemalloc updated to 3.0.0.
Full changelog here:

http://www.canonware.com/cgi-bin/gitweb.cgi?p=jemalloc.git;a=blob_plain;f=ChangeLog;hb=master

Notable improvements from the point of view of Redis:

1) Bugfixing.
2) Support for Valgrind.
3) Support for OSX Lion, FreeBSD.
2012-05-16 11:20:44 +02:00
Salvatore Sanfilippo 14beba7868 Merge pull request #496 from pietern/2.6-makeinstall
Fix `install` target on OSX (see #495)
2012-05-15 02:18:03 -07:00
Salvatore Sanfilippo 404223547c Merge pull request #510 from pietern/2.6-eventport
Support for Illumos event ports
2012-05-15 02:16:55 -07:00
Pieter Noordhuis d318803f94 Whitespace 2012-05-14 11:06:34 -07:00
Dave PachecoandPieter Noordhuis 72f30bcd30 use port_getn instead of port_get 2012-05-14 11:01:35 -07:00
Dave PachecoandPieter Noordhuis 2aa1efb8a5 first cut at event port support 2012-05-14 11:01:33 -07:00
antirez e67d014d9a Added time.h include in redis-cli.
redis-cli.c uses the time() function to seed the PRNG, but time.h was
not included. This was not noticed since sys/time.h is included and was
enough in most systems (but not correct). With Ubuntu 12.04 GCC
generates a warning that made us aware of the issue.
2012-05-14 17:43:31 +02:00
antirez eee680541b activeExpireCycle(): better precision in max time used.
activeExpireCycle() can consume no more than a few milliseconds per
iteration. This commit improves the precision of the check for the time
elapsed in two ways:

1) We check every 16 iterations instead of the main loop instead of 256.
2) We reset iterations at the start of the function and not every time
   we switch to the next database, so the check is correctly performed
   every 16 iterations.
2012-05-14 17:43:26 +02:00
antirez a8a981a834 Impovements for: Redis timer, hashes rehashing, keys collection.
A previous commit introduced REDIS_HZ define that changes the frequency
of calls to the serverCron() Redis function. This commit improves
different related things:

1) Software watchdog: now the minimal period can be set according to
REDIS_HZ. The minimal period is two times the timer period, that is:

    (1000/REDIS_HZ)*2 milliseconds

2) The incremental rehashing is now performed in the expires dictionary
as well.

3) The activeExpireCycle() function was improved in different ways:

- Now it checks if it already used too much time using microseconds
  instead of milliseconds for better precision.
- The time limit is now calculated correctly, in the previous version
  the division was performed before of the multiplication resulting in
  a timelimit of 0 if HZ was big enough.
- Databases with less than 1% of buckets fill in the hash table are
  skipped, because getting random keys is too expensive in this
  condition.

4) tryResizeHashTables() is now called at every timer call, we need to
   match the number of calls we do to the expired keys colleciton cycle.

5) REDIS_HZ was raised to 100.
2012-05-14 17:43:23 +02:00
antirez f7f2b2610e Redis timer interrupt frequency configurable as REDIS_HZ.
Redis uses a function called serverCron() that is very similar to the
timer interrupt of an operating system. This function is used to handle
a number of asynchronous things, like active expired keys collection,
clients timeouts, update of statistics, things related to the cluster
and replication, triggering of BGSAVE and AOF rewrite process, and so
forth.

In the past the timer was called 1 time per second. At some point it was
raised to 10 times per second, but it still was fixed and could not be
changed even at compile time, because different functions called from
serverCron() assumed a given fixed frequency.

This commmit makes the frequency configurable, so that it is simpler to
pick a good tradeoff between overhead of this function (that is usually
very small) and the responsiveness of Redis during a few critical
circumstances where a lot of work is done inside the timer.

An example of such a critical condition is mass-expire of a lot of keys
in the same second. Up to a given percentage of CPU time is used to
perform expired keys collection per expire cylce. Now changing the
REDIS_HZ macro it is possible to do less work but more times per second
in order to block the server for less time.

If this patch will work well in our tests it will enter Redis 2.6-final.
2012-05-14 17:43:07 +02:00
antirez f078d5628d Comment improved so that the code goal is more clear. Thx to @agladysh. 2012-05-12 09:33:29 +02:00
antirez 3a401464e9 More incremental active expired keys collection process.
If a large amonut of keys are all expiring about at the same time, the
"active" expired keys collection cycle used to block as far as the
percentage of already expired keys was >= 25% of the total population of
keys with an expire set.

This could block the server even for many seconds in order to reclaim
memory ASAP. The new algorithm uses at max a small amount of
milliseconds per cycle, even if this means reclaiming the memory less
promptly it also means a more responsive server.
2012-05-12 09:33:24 +02:00
antirez 064223107e If the computer running the Redis test is slow, we revert to --clients 1 to avoid false positives. 2012-05-12 09:33:19 +02:00
antirez 25496f4700 redis-cli pipe mode: handle EINTR properly as well so that SIGSTOP/SIGCONT are handled correctly. 2012-05-12 09:33:15 +02:00
antirez 346825c7ed redis-cli pipe mode: handle EAGAIN while writing to socket. 2012-05-12 09:33:11 +02:00
antirez dd4e8203b2 redis-cli --pipe for mass import. 2012-05-12 09:33:06 +02:00
antirez 91d18504c2 Fix PREFIX typo in Makefile. 2012-05-09 20:45:13 +02:00
antirez f580a3e3a0 Allow PREFIX to be overrided in Makefile. 2012-05-09 10:34:58 +02:00
antirez 8afa5b7072 Fixed typo in RC3 release notes. 2012-05-06 10:22:07 +02:00
antirez af2455be29 More complete release notes for 2.5.9 2012-05-06 10:15:40 +02:00
antirez 184b8e78c6 Redis 2.5.9 (2.6 RC3). 2012-05-06 10:11:54 +02:00
Salvatore Sanfilippo d23d73c746 Merge pull request #500 from pietern/2.6-zipfix
Compare integers in ziplist regardless of encoding
2012-05-06 01:03:15 -07:00
Pieter Noordhuis 0ef889274f Compare integers in ziplist regardless of encoding
Because of the introduction of new integer encoding types for ziplists
in the 2.6 tree, the same integer value may have a different encoding in
different versions of the ziplist implementation. This means that the
encoding can NOT be used as a fast path in comparing integers.
2012-05-04 17:26:24 -07:00
Pieter Noordhuis 1b1cf8e7bf Fix install target on OSX (see #495) 2012-05-04 11:54:25 -07:00
quiverandantirez 7f10703b33 fix several bugs of init.d scripts
- PIDFILE environ variable was not properly retrieved
- chkconfig command failed
2012-05-04 13:42:03 +02:00
antirez 0cf10e8e86 syncio.c read / write functions reworked for correctness and performance.
The new implementation start reading / writing before blocking with
aeWait(), likely the descriptor can accept writes or has buffered data
inside and we can go faster, otherwise we get an error and wait.

This change has effects on speed but also on correctness: on socket
errors when we perform non blocking connect(2) write is performed ASAP
and the error is returned ASAP before waiting.

So the practical effect is that now a Redis slave is more available if it
can not connect to the master, previously the slave continued to block on
syncWrite() trying to send SYNC, and serving commands very slowly.
2012-05-02 22:45:12 +02:00
antirez 9b43b1ef4d Remove useless trailing space in SYNC command sent to master. 2012-05-02 21:48:08 +02:00
antirez 0b08d64882 Use specific error if master is down and slave-serve-stale-data is set to no.
We used to reply -ERR ... message ..., now the reply is
instead -MASTERDOWN ... message ... so that it can be distinguished
easily by the other error conditions.
2012-05-02 17:14:45 +02:00
antirez 96d9c6cb8b Add a note abotu read-only slaves in 2.6 release notes. 2012-05-02 16:35:53 +02:00
antirez 0f07781538 Redis 2.5.8 (2.6.0 RC2). 2012-05-02 12:17:21 +02:00
antirez 1858da2faa Test "Turning off AOF kills the background writing child if any" is now more reliable. 2012-05-02 11:40:55 +02:00
Salvatore Sanfilippo 83b1092cf2 Merge pull request #488 from pietern/2.6-safekeys
Use safe dictionary iterator from KEYS (fixes #487)
2012-05-01 01:50:25 -07:00
Pieter Noordhuis 9311d2b527 Use safe dictionary iterator from KEYS
Every matched key in a KEYS call is checked for expiration. When the key
is set to expire, the call to `getExpire` will assert that the key also
exists in the main dictionary. This in turn causes a rehashing step to
be executed. Rehashing a dictionary when there is an iterator active may
result in the iterator emitting duplicate entries, or not emitting some
entries at all. By using a safe iterator, the rehash step is omitted.
2012-04-30 10:16:20 -07:00
HarmenandPieter Noordhuis 8520066d7b Show problem with 'keys' command with specific command sequence. 2012-04-30 09:51:23 -07:00
antirez 8f984bef29 Properly wait the slave to sync with master in BRPOPLPUSH test. 2012-04-30 11:32:02 +02:00
antirez dd418873db A more lightweight implementation of issue 141 regression test. 2012-04-29 17:16:47 +02:00
antirez b330de57ff Added "read-only slaves" in new features section of 2.6 release notes. 2012-04-27 23:06:02 +02:00
antirez 18759c927a yet another typo fixed in release notes. 2012-04-27 17:04:03 +02:00
antirez 8b97442c20 Fixed release notes typo 2012-04-27 17:01:44 +02:00
antirez 7c5d96d98e Redis 2.5.7 (2.6 RC1) 2012-04-27 16:40:07 +02:00
antirez 603adb2b29 memtest.c fixed to actually use v1 and v2 in memtest_fill_value(). 2012-04-27 16:28:31 +02:00
antirez 748f206e3d Release notes updated with the new 2.6 features. 2012-04-27 16:07:42 +02:00
antirez 2c0aae760f redis-cli commands description in help.h updated. 2012-04-27 15:57:17 +02:00
antirez b1aa7183d7 Update makefile dependencies. 2012-04-27 15:56:16 +02:00
antirez 77a75fdef5 Set LUA_MASKCOUNT hook more selectively. Fixes issue #480.
An user reported a crash with Redis scripting (see issue #480 on
github), inspection of the kindly provided strack trace showed that
server.lua_caller was probably set to NULL. The stack trace also slowed
that the call to the hook was originating from a point where we just
used to set/get a few global variables in the Lua state.

What was happening is that we did not set the timeout hook selectively
only when the user script was called. Now we set it more selectively,
specifically only in the context of the lua_pcall() call, and make sure
to remove the hook when the call returns. Otherwise the hook can get
called in random contexts every time we do something with the Lua
state.
2012-04-27 11:47:30 +02:00
antirez e69e76d442 Re-introduce -g -rdynamic -ggdb when linking, fixing strack traces.
A previous commit removed -g -rdynamic -ggdb as LDFLAGS, not allowing
Redis to produce a stack trace wth symbol names on crash.
This commit fixes the issue.
2012-04-27 11:47:27 +02:00
antirez 9db4cea235 Produce the stack trace in an async safe way. 2012-04-27 11:47:22 +02:00
antirez a28ab2a92b Don't use an alternative stack for SIGSEGV & co.
This commit reverts most of c575766202, in
order to use back main stack for signal handling.

The main reason is that otherwise it is completely pointless that we do
a lot of efforts to print the stack trace on crash, and the content of
the stack and registers as well. Using an alternate stack broken this
feature completely.
2012-04-27 11:47:18 +02:00
antirez b1ee7da75a Redis test: More reliable BRPOPLPUSH replication test.
Now it uses the new wait_for_condition testing primitive.
Also wait_for_condition implementation was fixed in this commit to properly
escape the expr command and its argument.
2012-04-27 11:47:15 +02:00
antirez d7bad544dc Redis test: scripting EVALSHA replication test more reliable.
A new primitive wait_for_condition was introduced in the scripting
engine that makes waiting for events simpler, so that it is simpler to
write tests that are more resistant to timing issues.
2012-04-27 11:47:12 +02:00
David Tranandantirez 8111a803cb Spelling: s/synchrnonization/synchronization 2012-04-27 11:47:08 +02:00
antirez c16bf717de Explicitly use bash for install_server.sh. Fixes issue #397 2012-04-24 19:35:40 +02:00
antirez 12a042f863 redis-check-dump now is RDB version 6 ready. 2012-04-24 19:35:36 +02:00
antirez 717145c1a3 Spurious debugging printf removed. 2012-04-24 19:35:33 +02:00
antirez dcd4efe9ef Added two new encodings to ziplist.c
1) One integer "immediate" encoding that can encode from 0 to 12 in the
encoding byte itself.
2) One 8 bit signed integer encoding that can encode 8 bit signed small
integers in a single byte.

The idea is to exploit all the not used bits we have around in a
backward compatible way.
2012-04-24 19:35:29 +02:00
antirez 62bfa66284 rdbLoad() should check REDIS_RDB_VERSION instead of hardcoded number. 2012-04-24 19:35:25 +02:00
antirez dd51571cc3 ziplist.c: added comments about the new 24 bit encoding. 2012-04-24 19:35:22 +02:00
Grisha Trubetskoyandantirez ad91404a32 Add a 24bit integer to ziplists to save one byte for ints that can
fit in 24 bits (thanks to antirez for catching and solving the two's compliment
bug).

Increment REDIS_RDB_VERSION to 6
2012-04-24 19:35:18 +02:00
antirez 9de5d4600a A few compiler warnings suppressed. 2012-04-24 19:35:03 +02:00
antirez 38b60dea54 Fix and refactoring of code used to get registers on crash.
This fixes compilation on FreeBSD (and possibly other systems) by
not using ucontext_t at all if HAVE_BACKTRACE is not defined.
Also the ifdefs to get the registers are modified to explicitly test for the
operating system in the first level, and the arch in the second level
of nesting.
2012-04-24 19:34:15 +02:00
antirez 69b30cfcb6 Ziplist encoding now tested with negative integers as well. 2012-04-24 19:34:08 +02:00
antirez 0a8a1e78dc New time limit for protocol desync test set to 30 seconds to reduce false positives. 2012-04-24 19:34:03 +02:00
antirez 537dafab84 Remove loadfile() access from the scripting engine. 2012-04-24 19:33:58 +02:00
Michael Schlenkerandantirez 7d6bf7956e Replace unnecessary calls to echo and cat
Tcl's exec can send data to stdout itself, no need to call cat/echo for
that usually.
2012-04-24 19:33:54 +02:00
antirez e337b26050 Even inside #if 0 comments are comments. 2012-04-21 21:49:34 +02:00
antirez 590d55a206 Limit memory used by big SLOWLOG entries.
Two limits are added:

1) Up to SLOWLOG_ENTRY_MAX_ARGV arguments are logged.
2) Up to SLOWLOG_ENTRY_MAX_STRING bytes per argument are logged.
3) slowlog-max-len is set to 128 by default (was 1024).

The number of remaining arguments / bytes is logged in the entry
so that the user can understand better the nature of the logged command.
2012-04-21 21:00:33 +02:00
antirez c11a01a030 redis.conf AOF section comments improved. 2012-04-21 12:17:33 +02:00
antirez 30e89410d4 README now makes clear that our support for solaris derived systems is "best effort". 2012-04-21 12:17:29 +02:00
antirez abfd08f5ad New tests related to scripts max execution time. 2012-04-19 23:49:46 +02:00
antirez ca577d162a SHUTDOWN NOSAVE now can stop a non returning script. Issue #466. 2012-04-19 23:36:04 +02:00
antirez d54943b76d Currenly not used code in dict.c commented out. 2012-04-18 23:56:15 +02:00
antirez c3312760fe Tests for scripting PRNG. 2012-04-18 23:50:27 +02:00
antirez 1d82bbd432 cr16.c removed from 2.6 branch, was not used. 2012-04-18 23:41:00 +02:00
antirez 0122cc4f42 redis-cli --bigkeys output modified to be simpler to read.. 2012-04-18 21:23:42 +02:00
antirez e10768518c redis-cli --bigkeys 2012-04-18 21:23:39 +02:00
antirez 5c45ae1f7b Test SDIFF with first set empty. 2012-04-18 21:23:35 +02:00
antirez 7a2065ef33 Test SINTER against same integer elements, but different set encoding. 2012-04-18 21:23:31 +02:00
antirez eb624e3416 Test SINTER with non existing key. 2012-04-18 21:23:15 +02:00
antirez ff5e31f74b Added an SMOVE test where src and dest key are the same. 2012-04-18 21:23:07 +02:00
antirez 24982f2bbc New hash fuzzing test. 2012-04-18 21:23:04 +02:00
antirez 60ef787efa Document mostly dead code in RPOPLPUSH implementation. 2012-04-18 21:23:00 +02:00
antirez bec200ec39 Explicit RPOP/LPOP tests. 2012-04-18 21:22:56 +02:00
antirez a00fcaa671 Test LINSERT syntax error. 2012-04-18 21:22:52 +02:00
antirez 8d12645569 Test LINDEX out of range index. 2012-04-18 21:22:48 +02:00
Salvatore Sanfilippo d00f51405b Merge pull request #455 from pietern/2.6-mkopts
Persist Makefile flags (2.6)
2012-04-18 07:26:36 -07:00
antirez ae55245d1c Marginally cleaner lookupKeyByPattern() implementation.
just fieldobj itself as sentinel of the fact a field object is used or
not, instead of using the filed length, that may be confusing both for
people and for the compiler emitting a warning.
2012-04-18 11:41:06 +02:00
antirez 212bb9ca2e More robust maxclients test. 2012-04-18 11:41:03 +02:00
antirez 727d6dd52a Two small fixes to maxclients handling.
1) Don't accept maxclients set to < 0
2) Allow maxclients < 1024, it is useful for testing.
2012-04-18 11:40:59 +02:00
antirez a1090c1193 Added test for SORT corner case: pattern ending with just "->". 2012-04-17 18:26:56 +02:00
antirez 68ee18558a lookupKeyByPattern() used by SORT GET/BY rewritten. Fixes issue #460.
lookupKeyByPattern() was implemented with a trick to speedup the lookup
process allocating two fake Redis obejcts on the stack. However now that
we propagate expires to the slave as DEL operations the lookup of the
key may result into a call to expireIfNeeded() having the stack
allocated object as argument, that may in turn use it to create the
protocol to send to the slave. But since this fake obejcts are
inherently read-only this is a problem.

As a side effect of this fix there are no longer size limits in the
pattern to be used with GET/BY option of SORT.

See https://github.com/antirez/redis/issues/460 for bug details.
2012-04-17 18:26:52 +02:00
antirez 96aeca4b9d Less false positives in maxclients test, hopefully. 2012-04-17 10:04:59 +02:00
Pieter Noordhuis d0cd262fdf Persist make settings and trigger rebuild if anything changes 2012-04-13 18:05:51 -07:00
Pieter Noordhuis 864229585a Don't set flags recursively 2012-04-13 18:05:51 -07:00
Pieter Noordhuis 3126e08757 Remove unused LIBS variable 2012-04-13 18:05:51 -07:00
Pieter Noordhuis 61e8825e89 First set defaults, then do composition 2012-04-13 18:05:51 -07:00
Pieter Noordhuis 95bc195158 Question mark assignment is not needed 2012-04-13 18:05:51 -07:00
Pieter Noordhuis cb481f432a Ignore gcov/lcov artifacts 2012-04-13 18:05:51 -07:00
Pieter Noordhuis 0b27a55f30 The lcov target shouldn't clean
This is not needed because every change in compiler/linker flags
triggers a cleanup.
2012-04-13 18:05:51 -07:00
Pieter Noordhuis 6cedb4d489 Rename ADD_*FLAGS -> REDIS_*FLAGS, REDIS_*FLAGS -> FINAL_*FLAGS
This reflects that REDIS_*FLAGS will only be used for compilation of
Redis and not for its dependencies. Similarly, that FINAL_*FLAGS are
composed of other variables and holds the options that are finally
passed to the compiler and linker.
2012-04-13 18:05:51 -07:00
antirez 59333ffd37 New test for scripting engine: DECR_IF_GT. 2012-04-13 16:23:46 +02:00
antirez 9a2dd1eff9 EVAL errors are more clear now. 2012-04-13 16:23:42 +02:00
antirez c9853f537b Use Lua tostring() before concatenation. 2012-04-13 16:23:38 +02:00
antirez d63a1716eb Tests modified to match the new global protection implementation. 2012-04-13 16:23:34 +02:00
antirez d2906893e8 mt.declared is no longer needed.
Lua global protection can now be simpified becuase we no longer have the
global() function. It's useless to occupy memory with this table, it is
also not faster because the metamethods we use are only called when a
global object does not exist or we are trying to create it from a
script.
2012-04-13 16:23:29 +02:00
antirez 97cab30993 Stop access to global vars. Not configurable.
After considering the interaction between ability to delcare globals in
scripts using the 'global' function, and the complexities related to
hanlding replication and AOF in a sane way with globals AND ability to
turn protection On and Off, we reconsidered the design. The new design
makes clear that there is only one good way to write Redis scripts, that
is not using globals. In the rare cases state must be retained across
calls a Redis key can be used.
2012-04-13 16:23:21 +02:00
antirez 6255a5ae66 Globals protection global() function modified for speed and correctness. 2012-04-13 16:22:47 +02:00
antirez e387dc52a0 Tests for lua globals protection. 2012-04-13 16:22:43 +02:00
antirez 3e6a4463e0 Scripting: globals protection can now be switched on/off. 2012-04-13 16:22:35 +02:00
antirez 430602b26c Protect globals access in Lua scripting. 2012-04-13 16:21:46 +02:00
antirez 236adc2809 A few var names fixed in Makefile.
I modified it for error in a previous commit doing search & replace.
2012-04-13 16:15:39 +02:00
antirez d298825803 Print arch bits with redis-server -v 2012-04-13 16:12:32 +02:00
antirez 206568257a memtest.c: integer overflow fixed. 2012-04-13 16:12:28 +02:00
antirez 50fb330399 Make gcov fixed. 2012-04-13 16:12:24 +02:00
antirez 69ac4d063d Makefile now introduces Redis-specific CFLAGS / LDFLAGS. Gcov target fixed. Added comments to describe how it works. 2012-04-13 16:12:04 +02:00
Erik Dubbelboerandantirez b9cd703b5b added explanation for the magic 511 backlog number 2012-04-11 18:32:10 +02:00
antirez 79e3df9d72 Makefile.dep updated. 2012-04-11 18:30:23 +02:00
antirez d5ec389585 make dep: redirect output to Makefile.dep. 2012-04-11 18:29:31 +02:00
antirez 336ba6a152 Make inline functions rioRead/Write/Tell static. This fixes issue #447. 2012-04-11 12:03:25 +02:00
antirez ef278d110f Macros ULONG_ONEZERO / ULONG_ZEROONE were inverted in #ifdef to test 32/64 bit arch. 2012-04-11 12:03:21 +02:00
Pieter Noordhuisandantirez de07849e0d Clean up Makefiles
Remove unused variables. Instead of overriding non-standard variables
such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to
override Makefile settings. Move dependencies generated by `make dep` to
a separate file.
2012-04-11 12:01:29 +02:00
Pieter Noordhuisandantirez e74bec56fa Everything x86 is little endian 2012-04-11 11:59:53 +02:00
antirez 3ba5eab774 Minor MIGRATE implementation simplification about ttl handling. 2012-04-10 20:04:21 +02:00
antirez b9aa332843 Check write(2) return value to avoid warnings, because in this context failing write is not critical. 2012-04-10 20:02:33 +02:00
antirez 3f64694e71 Version 2.5.6. 2012-04-10 16:34:51 +02:00
antirez fdf8bd4025 Test for maxclients. 2012-04-10 16:28:21 +02:00
antirez e95740392b dump/restore fixed to use the new crc64 API. 2012-04-10 16:27:36 +02:00
antirez 39d1e350d9 It is now possible to enable/disable RDB checksum computation from redis.conf or via CONFIG SET/GET. Also CONFIG SET support added for rdbcompression as well. 2012-04-10 16:27:03 +02:00
antirez 7f4f86f427 RDB files now embed a crc64 checksum. Version of RDB bumped to 5. 2012-04-10 16:26:58 +02:00
antirez 1bcb45d118 Fixed compilation of new rio.c changes (typos and so forth.) 2012-04-10 16:26:52 +02:00
antirez b4b923b04b Add checksum computation to rio.c 2012-04-10 16:26:45 +02:00
antirez bb99f42596 crc64.c modified for incremental computation. 2012-04-10 16:26:41 +02:00
antirez 9ba4d5a3bb rio.c file somewhat documented so that the casual reader can understand what's going on without reading the code. 2012-04-10 16:26:37 +02:00
antirez 08211b25d3 Added new test to check that "CONFIG appendonly no" actually kills the background AOF operation in progress if any. 2012-04-08 10:43:33 +02:00
antirez 55951f9005 For coverage testing use exit() instead of _exit() when termiating saving children. 2012-04-08 10:43:29 +02:00
antirez 2cf3f071a5 Tests for MONITOR. 2012-04-07 11:27:04 +02:00
antirez b162e6f133 New client info field added to CLIENT LIST output: multi, containing the length of the current pipeline. Test modified accordingly. 2012-04-07 11:27:00 +02:00
antirez 4f0bd607d9 Never used function stringObjectEqualsMs() removed. 2012-04-07 11:26:56 +02:00
antirez 7dc1d2ba17 Removed dead code: function rdbSaveTime() is no longer used since RDB now saves expires in milliseconds. 2012-04-07 11:26:52 +02:00
antirez eb6bc2e047 Two new tests for BGREWRTIEAOF.
Check for scheduled rewrite if a BGSAVAE is in progress.
Check for error if a rewrite is already in progress.
2012-04-07 11:26:47 +02:00
antirez 3984108474 redis.tcl: no longer leave unread replies if an error happens during a MULTI/EXEC block. 2012-04-07 11:26:42 +02:00
antirez 8d3f5ce9db New INFO field in persistence section: bgrewriteaof_scheduled. 2012-04-06 21:13:33 +02:00
antirez 8382453ecf version bumped to 2.5.5 2012-04-06 12:27:31 +02:00
jokeaandantirez a3ec16f0c2 implement aeWait using poll(2). Fixes issue #267. 2012-04-06 11:49:16 +02:00
antirez 3aad0de2e9 expireGenericCommand(): better variable names and a top-comment that describes the function's behavior. 2012-04-06 00:40:36 +02:00
Premysl Hrubyandantirez 81a28fe131 for (p)expireat use absolute time, without double recomputation 2012-04-06 00:40:31 +02:00
Premysl Hrubyandantirez b518a83525 fix mstime() ommited while comparing if key is already expired 2012-04-06 00:40:26 +02:00
Premysl Hrubyandantirez 9f899440d2 add support for generation of lcov coverage reports 2012-04-06 00:40:22 +02:00
Premysl Hrubyandantirez 937abebbe1 future-proof version comparison 2012-04-05 11:04:28 +02:00
antirez c537980d98 On slow computers, 10 seconds are not enough for this heavy replication test. 2012-04-05 11:04:23 +02:00
Premysl Hrubyandantirez c9adc81e2e allocate alternate signal stack, change of sigaction flags for sigterm 2012-04-05 11:04:19 +02:00
Premysl Hrubyandantirez b811b334ae in kill_server send the signal once, then wait for up to 5sec before sending lethal SIGKILL 2012-04-05 11:04:14 +02:00
Premysl Hrubyandantirez 4d57e44839 new option for choosing number of test clients to run 2012-04-05 11:04:09 +02:00
Premysl Hrubyandantirez 6c52d5ce27 remove mentions of VM in comments 2012-04-05 11:04:03 +02:00
antirez 09eb448746 Structure field controlling the INFO field master_link_down_since_seconds initialized correctly to avoid strange INFO output at startup when a slave has yet to connect to its master. 2012-04-04 18:33:13 +02:00
antirez fa2a27cfeb New "os" field in INFO output providing information about the operating system. 2012-04-04 18:33:09 +02:00
antirez a5b75e9a86 SLAVEOF is not a write command. 2012-04-04 18:33:04 +02:00
antirez df35d87336 Print milliseconds of the current second in log lines timestamps. Sometimes precise timing is very important for debugging. 2012-04-04 18:32:59 +02:00
antirez 3c413d59c8 redis-cli help.h updated. Script to generate it updated as well. 2012-04-03 15:29:47 +02:00
antirez e3fd3ccd72 More MIGRATE tests. 2012-04-03 15:14:03 +02:00
antirez fb8409a5c8 Another fix for MIGRATE. 2012-04-03 15:13:50 +02:00
antirez bde80cb28b Two fixed for MIGRATE: fix TTL propagation and fix transferring of data bigger than 64k. 2012-04-03 14:57:28 +02:00
antirez fbce475270 When the user-provided 'maxclients' value is too big for the max number of files we can open, at least try to search the max the OS is allowing (in steps of 256 filedes). 2012-04-03 14:55:18 +02:00
antirez 31e2156c4b redis-trib.rb removed from 2.6 branch. 2012-04-03 10:31:51 +02:00
antirez b739506936 MIGRATE test modified because the implementation changed. 2012-04-02 16:41:11 +02:00
antirez 37cc07dd41 MIGRATE now let the client distinguish I/O errors and timeouts from other erros. 2012-04-02 16:41:02 +02:00
antirez 285df7b0e0 Version bumped 2.4.5 2012-04-02 14:27:04 +02:00
antirez 8cf8974a03 DUMP, RESTORE, MIGRATE tests. 2012-04-02 14:22:07 +02:00
antirez bff31e129d New verions of DUMP, RESTORE, MIGRATE back ported from unstable to 2.6 2012-04-02 14:21:59 +02:00
antirez 179ee2db78 CRC64 implementation added to Redis code base. 2012-04-02 13:53:43 +02:00
antirez 11dae1711f Write RDB magic using a REDIS_RDB_VERSION define that is defined inside rdb.h 2012-04-02 13:52:42 +02:00
ThePicardandantirez f08aa2bf48 Fixed a typo in install_server.sh 2012-04-01 10:15:38 +02:00
antirez fa34ba3908 syncio.c calls in replication.c fixed for the new millisecond timeout API. 2012-03-31 11:26:46 +02:00
antirez 4ccf671cc6 Better syncio.c with millisecond resolution. 2012-03-31 11:26:42 +02:00
antirez 88bd32f1b7 Purely aesthetic code change. 2012-03-30 10:40:04 +02:00
Joseph Jangandantirez ae15f75089 Fixed a memory leak with replication
occurs when two or more dbs are replicated and at least one of them is >db10
2012-03-30 10:40:01 +02:00
antirez 0f51e3c564 Regression test for issue 417 (memory leak when replicating to DB with id >= 10) 2012-03-30 10:39:56 +02:00
antirez f1e38b352e Fixed typo in comment: "te" -> "the". 2012-03-29 09:34:07 +02:00
antirez ed4d4f1145 Fix for slaves chains. Force resync of slaves (simply disconnecting them) when SLAVEOF turns a master into a slave. 2012-03-29 09:34:03 +02:00
antirez ee704a0ff1 Test for redis.sha1hex(). 2012-03-29 09:33:57 +02:00
Nathan Fritzandantirez 2f2e6ad487 added redis.sha1hex(string) as lua scripting function.
(The original implementation was modified by @antirez to conform Redis
coding standards.)
2012-03-29 09:33:52 +02:00
antirez e51f7d2c17 Fixes for redisLogFromHandler(). 2012-03-28 14:57:09 +02:00
antirez be4f8cccaa Log from signal handlers is now safer. 2012-03-28 14:57:04 +02:00
antirez f3e159bc93 Redis test: regexp to check if valgrind reported errors modified. Now we no longer look at the total count because this includes "possibly lost" bytes that are not interesting for Redis (tons of false positives because of how sds.c works). 2012-03-28 13:18:32 +02:00
antirez dd896e1b8d define zlibc_free() in a way that is not shadowed by jemalloc. 2012-03-28 13:18:29 +02:00
antirez 59d884af8c Produce the watchlog warning log in a way that is safer from a signal handler. Fix a memory leak in the backtrace generation function. 2012-03-28 13:18:25 +02:00
antirez e4cd5838fc Mask SIGALRM everything but in the main thread.
This is required to ensure that the signal will be delivered to the main
thread when the watchdog timer expires.
2012-03-28 13:18:21 +02:00
antirez 1507be5b01 Correctly set the SIGARLM timer for the software watchdog. 2012-03-28 13:18:17 +02:00
antirez fc030ac7de Redis software watchdog. 2012-03-28 13:18:13 +02:00
huangz1990andantirez 085aaef325 fix typo 2012-03-27 23:04:40 +02:00
Premysl Hrubyandantirez 92dd4e4301 fix time() instead of mstime() in expireIfNeeded 2012-03-27 23:04:34 +02:00
Premysl Hrubyandantirez 56ff70f8e0 use server.unixtime instead of time(NULL) where possible (cluster.c not checked though) 2012-03-27 23:04:27 +02:00
Premysl Hrubyandantirez 38d6976c03 declare hashDictType as external too 2012-03-27 23:04:21 +02:00
Premysl Hrubyandantirez efd412f908 remove disk-store related comments 2012-03-27 23:04:16 +02:00
antirez 9ea95e6c64 SIGSEGV handler refactored so that we can reuse stack trace and current client logging functionalities in other contexts. 2012-03-27 10:33:45 +02:00
antirez d0407c2d85 CONFIG RESETSTAT resets two more fields. 2012-03-25 11:43:42 +02:00
antirez 81f32c7b65 New INFO field aof_delayed_fsync introduced.
This new field counts all the times Redis is configured with AOF enabled and
fsync policy 'everysec', but the previous fsync performed by the
background thread was not able to complete within two seconds, forcing
Redis to perform a write against the AOF file while the fsync is still
in progress (likely a blocking operation).
2012-03-25 11:43:38 +02:00
antirez 754643c1c7 Version is now 2.5.3. 2012-03-25 11:18:24 +02:00
antirez 0387e9c199 convert-zipmap-hash-on-load false positive fixed.
Apparently because the sample RDB file was not copied before every test
Redis had a chance to replace it with a newly written one, so that the
next test could fail.
2012-03-25 10:57:34 +02:00
antirez 29e1976855 When running the test in valgrind mode, pass the right flags to show memory leaks stack traces but only including the "definitely lost" items. 2012-03-24 12:06:56 +01:00
antirez 188a17ed56 Add used allocator in redis-server -v output. 2012-03-24 11:52:56 +01:00
antirez 9542d9d8d7 convert-zipmap-hash-on-load test enabled 2012-03-24 11:42:39 +01:00
antirez 2d04eef425 Contextualize comment. 2012-03-23 20:20:43 +01:00
antirez 925927fc8f Merge remote-tracking branch 'origin/2.6' into 2.6 2012-03-23 20:20:13 +01:00
antirez 27688de10e RDB load of different encodings test added. 2012-03-23 20:22:22 +01:00
antirez 9b962d10a4 Fixed memory leak in hash loading. 2012-03-23 20:21:19 +01:00
antirez ab0603812d RDB load of different encodings test added. 2012-03-23 15:23:01 +01:00
antirez b64281cc0e Big endian fix. The bug was introduced because of a typo. 2012-03-23 12:50:19 +01:00
antirez 52192552bd Replicate HINCRBYFLOAT as HSET. 2012-03-23 10:24:25 +01:00
antirez b54cdfb2b7 Code style hack. 2012-03-22 18:16:41 +01:00
antirez 7b22c44ccd Result of INCRBYFLOAT and HINCRBYFLOAT should never be in exponential form, and also should never contain trailing zeroes. This is not possible with vanilla printf() format specifiers, so we alter the output. 2012-03-22 18:14:32 +01:00
antirez 9aba884b34 Comments about security of slave-read-only in redis.coinf. 2012-03-22 10:09:58 +01:00
antirez 3f7ad83398 Correctly create shared.oomerr as an sds string. 2012-03-22 10:09:54 +01:00
antirez 38bb45223a DEBUG should not be flagged as w otherwise we can not call DEBUG DIGEST and other commands against read only slaves. 2012-03-22 10:09:49 +01:00
antirez 054061685a Support for read-only slaves. Semantical fixes.
This commit introduces support for read only slaves via redis.conf and CONFIG GET/SET commands. Also various semantical fixes are implemented here:

1) MULTI/EXEC with only read commands now work where the server is into a state where writes (or commands increasing memory usage) are not allowed. Before this patch everything inside a transaction would fail in this conditions.

2) Scripts just calling read-only commands will work against read only
slaves, when the server is out of memory, or when persistence is into an
error condition. Before the patch EVAL always failed in this condition.
2012-03-22 10:09:43 +01:00
antirez 7c1cec2525 Redis cluster stuff removed from 2.6 redis.conf file. 2012-03-20 13:07:26 +01:00
antirez 0380c13bbe redis_init_script template updated. 2012-03-20 13:06:50 +01:00
antirez 518e7202b2 Fixed typo in 2.6 release notes. 2012-03-19 22:10:18 +01:00
antirez f597910121 Merge remote-tracking branch 'origin/2.6' into 2.6 2012-03-19 19:30:30 +01:00
antirez 24b0942275 Suppress warnings compiling redis-cli with certain gcc versions. 2012-03-19 19:29:06 +01:00
antirez 6e6bbac7a5 Read-only flag removed from PUBLISH command. 2012-03-19 19:18:51 +01:00
antirez a7ef5ce1b0 Memory addressing test implemented. 2012-03-19 14:02:57 +01:00
antirez ea693f0282 More memory tests implemented. Default number of iterations lowered to a more acceptable value of 50. 2012-03-18 18:03:49 +01:00
antirez 74760d3ccd Fixed typo. 2012-03-18 17:28:03 +01:00
antirez 32f62ed6d0 Number of iteration of --test-memory is now 300 (several minutes per gigabyte). Memtest86 and Memtester links are also displayed while running the test. 2012-03-18 17:24:48 +01:00
antirez d4e6ce3e97 On crash suggest to give --test-memory a try. 2012-03-18 11:43:09 +01:00
antirez d605fdabfa Memory test function now less boring thanks to screen-wide progress bar. 2012-03-18 11:43:02 +01:00
antirez f4df22d1c5 Hem... actual memtest.c file added. 2012-03-18 11:42:56 +01:00
antirez 78d6a02b0c First implementation of --test-memory. Still a work in progress. 2012-03-18 11:42:47 +01:00
antirez f1eaf57200 Fix for issue #391.
Use a simple protocol between clientsCron() and helper functions to
understand if the client is still valind and clientsCron() should
continue processing or if the client was freed and we should continue
with the next one.
2012-03-15 20:51:39 +01:00
antirez 749817b7c3 Version bumped to 2.5.2 2012-03-14 15:37:47 +01:00
antirez 9fa9ccb04e Reclaim space from the client querybuf if needed. 2012-03-14 15:36:27 +01:00
antirez 6934832e53 sds.c: sdsAllocSize() function added. 2012-03-14 15:36:24 +01:00
antirez f9322fb8ed sds.c new function sdsRemoveFreeSpace().
The new function is used in order to resize the string allocation so
that only the minimal allocation possible is used, removing all the free
space at the end of the string normally used to improve efficiency of
concatenation operations.
2012-03-14 15:36:20 +01:00
antirez 5e473cd8fd Call all the helper functions needed by clientsCron() as clientsCronSomething() for clarity. 2012-03-14 15:36:16 +01:00
antirez 3e8fcb6d03 CLIENT LIST test modified to reflect the new output. 2012-03-13 18:08:59 +01:00
antirez cfa4b57cb0 Process async client checks like client timeouts and BLPOP timeouts incrementally using a circular list. 2012-03-13 18:08:53 +01:00
antirez 57a5e54ddc Added a qbuf-free field to CLIENT LIST output. 2012-03-13 18:08:50 +01:00
antirez 41e8e5cb8e Client creation time in redisClient structure. New age field in CLIENT LIST output. 2012-03-13 18:08:47 +01:00
antirez 56de4964ee c->bufpos initialization moved for aesthetics. 2012-03-13 18:08:43 +01:00
antirez a74ab6478c RDB hashes loading, fixed another bug in the loading of HT-encoded hashes: when the hash entry is too big for ziplist, add the field, then convert. The code used to break before the new entry was inserted, resulting into missing fields in the loaded Hash object. 2012-03-13 11:01:08 +01:00
antirez ee61a4b99e RDB hashes loading fixed removing the assertion that failed every time an HT-encoded hash was loaded. 2012-03-13 09:49:21 +01:00
quiverandantirez dfc2545470 fix typo of redis.conf 2012-03-10 13:23:31 +01:00
antirez b014c1f211 RDB4 support in redis-check-dump. 2012-03-10 12:38:59 +01:00
antirez 37180ed9cd RDB version is no 4, because small hashes are now encoded as ziplists, so older versions of Redis will not understand this format. 2012-03-10 12:35:15 +01:00
antirez 7551f2a0b1 Version is now 2.5.1, first unstable release of Redis 2.6 2012-03-10 12:29:47 +01:00
antirez 00b3336334 Build dependencies updated. 2012-03-10 12:28:14 +01:00
antirez 571e257db1 Redis 2.6 branch obtained from unstable removing all the cluster related code. 2012-03-10 12:26:37 +01:00
166 changed files with 4289 additions and 21285 deletions
+2 -7
View File
@@ -1,13 +1,8 @@
.*.swp
*.o
*.rdb
*.log
dump.rdb
redis-benchmark
redis-check-aof
redis-check-dump
redis-cli
redis-sentinel
redis-server
redis-*
doc-tools
release
misc/*
+292 -11
View File
@@ -1,16 +1,297 @@
Hello! This file is just a placeholder, since this is the "unstable" branch
of Redis, the place where all the development happens.
Redis 2.6 release notes
=======================
There is no release notes for this branch, it gets forked into another branch
every time there is a partial feature freeze in order to eventually create
a new stable release.
** IMPORTANT ** Check the 'Migrating from 2.4 to 2.6' section at the end of
this file for information about what changed between 2.4 and
2.6 and how this may affect your application.
Usually "unstable" is stable enough for you to use it in development enviromnets
however you should never use it in production environments. It is possible
to download the latest stable release here:
--------------------------------------------------------------------------------
Upgrade urgency levels:
http://download.redis.io/releases/redis-stable.tar.gz
LOW: No need to upgrade unless there are new features you want to use.
MODERATE: Program an upgrade of the server, but it's not urgent.
HIGH: There is a critical bug that may affect a subset of users. Upgrade!
CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
--------------------------------------------------------------------------------
More information is available at http://redis.io
--[ Redis 2.6.6 ]
Happy hacking!
UPGRADE URGENCY: CRITICAL if you experienced one more more crashes.
MODERATE if Redis is running fine for you.
* [BUGFIX] Jemalloc updated to 3.2.0.
--[ Redis 2.6.5 ]
UPGRADE URGENCY: MODERATE
Warning: this release of Redis introduces a different behavior in MULTI/EXEC
handling of errors. This was done because the new behavior is safer
compared to the old one, and should not break any code targeting
Redis 2.6 in a critical way.
For more information check http://redis.io/topics/transactions
* [IMPROVED] RDB/AOF childern now log amount of additional memory used
because of copy on write.
* [BUGFIX] MIGRATE non critical fixes (see commits for details).
* [BUGFIX] MULTI/EXEC: now EXEC aborts on errors before EXEC.
* [BUGFIX] Fix integer overflow in zunionInterGenericCommand resulting
into Z[INTER|UNION][STORE] commands to crash under extremely
unlikely conditions (almost impossible in real world).
* [BUGFIX] EVALSHA is now case insensitive (and will not crash).
--[ Redis 2.6.4 ]
UPGRADE URGENCY: LOW
* [IMPROVED] BSD license and copyright notice added to every .c and .h file.
--[ Redis 2.6.2 ]
UPGRADE URGENCY: LOW
* [BUGFIX] The compilation fix for RHLE5 in 2.6.1 was broken. Fixed.
* [IMPROVED] Linenoise updated, now supports Ctrl+w.
---[ Redis 2.6.1 ]
UPGRADE URGENCY: LOW
* [BUGFIX] Compilation on Linux < 2.6.17 or glibc < 2.6 fixed (RHLE5 & co).
---[ Redis 2.6.0 ]
UPGRADE URGENCY: HIGH
* [BUGFIX] Allow AUTH when server is in -BUSY state because of a slow script.
* [BUGFIX] MULTI/EXEC flow now makes sense when observed in MONITOR
* [BUGFIX] SCRIPT KILL now uses different error prefixes for different errors.
* [BUGFIX] Default memory limit for 32bit archs lowered from 3.5 to 3 GB.
* [BUGFIX] redis-check-dump is now compatible with RDB files generated by 2.6.
* [IMPROVED] New field in INFO: slave_read_only.
---[ Redis 2.5.14 (2.6 Release Candidate 8) ]
* [BUGFIX] Fixed compilation on FreeBSD.
* [IMPROVED] SRANDMEMBER <count> that returns multiple random elements.
* [IMPROVED] Sentinel backported to 2.6. It will be taken in sync with 2.8.
* [IMPROVED] Helper function for scripting to return errors and status replies.
* [IMPROVED] SORT by nosort [ASC|DESC] uses sorted set elements ordering.
* [BUGFIX] Better resistence to system clock skew.
* [IMPROVED] Warn the user when the configured maxmemory seems odd.
* [BUGFIX] Hashing function is now murmurhash2 for security purposes.
* [IMPROVED] Install script no longer uses a template but redis.conf itself.
---[ Redis 2.5.13 (2.6 Release Candidate 7) ]
UPGRADE URGENCY: HIGH
* [BUGFIX] Theoretical bug in ziplist fixed.
* [BUGFIX] Better out of memory handling (Log produced in log file).
* [BUGFIX] Incrementally flush RDB file on slave side while performing the
first synchronization with the master. This makes Redis less
blocking in environments where disk I/O is slow.
* [BUGFIX] Don't crash with Lua's redis.call() without arguments.
* [BUGFIX] Don't crash after a big number of Lua calls on 32 bit systems
because of a failed assertion.
* [BUGFIX] Fix SORT behaviour when called from scripting.
* [BUGFIX] Adjust slave PING period accordingly to REDIS_HZ define.
* [BUGFIX] BITCOUNT: fix crash on overflowing arguments.
* [BUGFIX] Return an error when SELECT argument is not an integer.
* [BUGFIX] Blocking operations on lists were completely reimplemented for
correctness. Now blocking list ops and pushes originated from
Lua scripts will play well together and will be replicated
and transmitted to the AOF correctly.
* [IMPROVED] Send async PING before starting replication to avoid blocking if
master allows us to connect but it is actually not able to reply.
* [IMPROVED] Support slave-priority for Redis Sentinel.
* [IMPROVED] Hiredis library updated.
---[ Redis 2.5.12 (2.6 Release Candidate 6) ]
UPGRADE URGENCY: MODERATE.
* [BUGFIX] Fixed a timing attack on AUTH (Issue #560).
* [BUGFIX] Don't assume that "char" is signed.
* [BUGFIX] Check that we have connection before enabling pipe mode.
* [BUGFIX] Use the optimized version of the function to convert a double to
its string representation. Compilation was disabled because of
a typo in the #if statement.
* [IMPROVED} REPLCONF internal command introduced, now INFO shows slaves with
correct port numbers. This makes 2.5.12 Redis Sentinel compatible.
* [IMPROVED] Truncate short write from the AOF for a cleaner restart. On short
writes (for instance out of space) Redis will now try to remove
the half-written data so that the next restart will work without
the need for the "redis-check-aof" utility.
* [IMPROVED] New in INFO: aof_last_bgrewrite_status
* [IMPROVED] Allow Pub/Sub in contexts where other commands are blocked.
* [BUGFIX] mark fd as writable when EPOLLERR or EPOLLHUP is returned by
epoll_wait.
---[ Redis 2.5.11 (2.6 Release Candidate 5) ]
UPGRADE URGENCY: HIGH.
* [BUGFIX] Fixed Hash corruption when loading an RDB file generated by
previous versions of Redis that encoded hashes using
a different ziplist encoding format for small integers.
All the fileds that are integers in the range 0-255 may not
be recognized, or duplicated un updates, causing a crash
when the ziplist is converted to a real hash. (Issue #547).
* [BUGFIX] Fixed the count of memory used by output buffers in the
setDeferredMultiBulkLength() function.
---[ Redis 2.5.10 (2.6 Release Candidate 4) ]
UPGRADE URGENCY: HIGH.
* [BUGFIX] Allow PREFIX to be overwritten on "make install".
* [BUGFIX] Run the test with just one client if the computer is slow.
* [BUGFIX] Event port support in our event driven libray.
* [BUGFIX] Jemalloc updated to 3.0.0. This fixes a possibly AOF rewrite issue.
See https://github.com/antirez/redis/issues/504 for info.
* [BUGFIX] Fixed issue #516: ZINTERSTORE / ZUNIONSTORE with mixed sets/zsets.
* [BUGFIX] Set fd to writable when poll(2) detects POLLERR or POLLHUP event.
* [BUGFIX] Fixed RESTORE hash failure (Issue #532).
* [IMPROVED] Allow an AOF rewrite buffer > 2GB (Related to issue #504).
* [IMPROVED] Server cron function frequency is now configurable (REDIS_HZ).
* [IMPROVED] Better, less blocking expired keys collection algorithm.
* [FEATURE] New commands: BITOP and BITCOUNT.
* [FEATURE] redis-cli --pipe for mass import.
What's new in Redis 2.5.9 (aka 2.6 Release Candidate 3)
=======================================================
UPGRADE URGENCY: critical, upgrade ASAP.
* [BUGFIX] Fix for issue #500 (https://github.com/antirez/redis/pull/500).
Redis 2.6-RC1 and RC2 may corrupt ziplist-encoded sorted sets
produced by Redis 2.4.x.
* [BUGFIX] Fixed several bugs in init.d script.
* [BUGFIX] syncio.c functions modified for speed and correctness. On osx
(and possibly other BSD-based systems) the slave would block on
replication to send the SYNC command when the master was not
available. This is fixed now, but was not affecting Linux installs.
* Now when slave-serve-stale-data is set to yes and the master is down, instead
of reporting a generic error Redis replies with -MASTERDOWN.
What's new in Redis 2.5.8 (aka 2.6 Release Candidate 2)
=======================================================
UPGRADE URGENCY: high for all the users of the KEYS command, otherwise low.
* [BUGFIX] Fix for KEYS command: if the DB contains keys with expires the KEYS
command may return the wrong output, having duplicated or missing
keys. See issue #487 and #488 on github for details.
What's new in Redis 2.5.7 (aka 2.6 Release Candidate 1)
=======================================================
UPGRADE URGENCY: upgrade not recommended because this is an RC release.
* This is the first release candidate for Redis 2.6. We are not aware of
bugs, but part of this code is young and was never tested in production
environments, so handle with care.
An overview of new features and changes in Redis 2.6.x
======================================================
* Server side Lua scripting, see http://redis.io/commands/eval
* Virtual Memory removed (was deprecated in 2.4)
* Hardcoded limits about max number of clients removed.
* AOF low level semantics is generally more sane, and especially when used
in slaves.
* Milliseconds resolution expires, also added new commands with milliseconds
precision (PEXPIRE, PTTL, ...).
* Better memory usage for "small" lists, ziplists and hashes when fields or
values contain small integers.
* Read only slaves.
* New bit opeations: BITCOUNT and BITOP commands.
* Clients max output buffer soft and hard limits. You can specifiy different
limits for different classes of clients (normal,pubsub,slave).
* More incremental (less blocking) expired keys collection algorithm, in
practical terms this means that Redis is more responsive when a very
big number of keys expire about at the same time.
* AOF is now able to rewrite aggregate data types using variadic commands,
often producing an AOF that is faster to save, load, and is smaller in size.
* Every redis.conf directive is now accepted as a command line option for the
redis-server binary, with the same name and number of arguments.
* Hash table seed randomization for protection against collisions attacks.
* Performances improved when writing large objects to Redis.
* Integrated memory test, see redis-server --test-memory.
* INCRBYFLOAT and HINCRBYFLOAT commands.
* New DUMP, RESTORE, MIGRATE commands (back ported from Redis Cluster to 2.6).
* CRC64 checksump in RDB files.
* Better MONITOR output and behavior (now commands are logged before execution).
* "Software Watchdog" feature to debug latency issues.
* Significant parts of the core refactored or rewritten. New internal APIs
and core changes allowed to develop Redis Cluster on top of the new code,
however for 2.6 all the cluster code was removed, and will be released with
Redis 3.0 when it is more complete and stable.
* Redis ASCII art logo added at startup.
* Crash report on memory violation or failed asserts improved significantly
to make debugging of hard to catch bugs simpler.
* redis-benchmark improvements: ability to run selected tests,
CSV output, faster, better help.
* redis-cli improvements: --eval for comfortable development of Lua scripts.
* SHUTDOWN now supports two optional arguments: "SAVE" and "NOSAVE".
* INFO output split into sections, the command is now able to just show
pecific sections.
* New statistics about how many time a command was called, and how much
execution time it used (INFO commandstats).
* More predictable SORT behavior in edge cases.
* Better support for big endian and *BSD systems.
* Build system improved.
Migrating from 2.4 to 2.6
=========================
Redis 2.4 is mostly a strict subset of 2.6. However there are a few things
that you should be aware of:
* You can't use .rdb and AOF files generated with 2.6 into a 2.4 instance.
* 2.6 slaves can be attached to 2.4 masters, but not the contrary, and only
for the time needed to perform the version upgrade.
There are also a few API differences, that are unlikely to cause problems,
but it is better to keep them in mind:
* SORT now will refuse to sort in numerical mode elements that can't be parsed
as numbers.
* EXPIREs now all have millisecond resolution (but this is very unlikely to
break code that was not conceived exploting the previous resolution error
in some way.)
* INFO output is a bit different now, and contains empty lines and comments
starting with '#'. All the major clients should be already fixed to work
with the new INFO format.
* Slaves are only read-only by default (but you can change this easily
setting the "slave-read-only" configuration option to "no" editing your
redis.conf or using CONFIG SET.
The following INFO fields were renamed for consistency:
changes_since_last_save -> rdb_changes_since_last_save
bgsave_in_progress -> rdb_bgsave_in_progress
last_save_time -> rdb_last_save_time
last_bgsave_status -> rdb_last_bgsave_status
bgrewriteaof_in_progress -> aof_rewrite_in_progress
bgrewriteaof_scheduled -> aof_rewrite_scheduled
The following redis.conf and CONFIG GET / SET parameters changed:
* hash-max-zipmap-entries, now replaced by hash-max-ziplist-entries
* hash-max-zipmap-value, now replaced by hash-max-ziplist-value
* glueoutputbuf option was now completely removed (was deprecated)
--------------------------------------------------------------------------------
Credits: Where not specified the implementation and design are done by
Salvatore Sanfilippo and Pieter Noordhuis. Thanks to VMware for making all
this possible. Also many thanks to all the other contributors and the amazing
community we have.
See commit messages for more credits.
Cheers,
Salvatore
+2 -3
View File
@@ -2,8 +2,7 @@ Note: by contributing code to the Redis project in any form, including sending
a pull request via Github, a code fragment or patch via private email or
public discussion groups, you agree to release your code under the terms
of the BSD license that you can find in the COPYING file included in the Redis
source distribution. You will include BSD license in the COPYING file within
each source file that you contribute.
source distribution.
# IMPORTANT: HOW TO USE REDIS GITHUB ISSUES
@@ -22,7 +21,7 @@ each source file that you contribute.
1. Drop a message to the Redis Google Group with a proposal of semantics/API.
2. If in step 1 you get an acknowledge from the project leaders, use the
2. If in steps 1 you get an acknowledge from the project leaders, use the
following procedure to submit a patch:
a. Fork Redis on github ( http://help.github.com/fork-a-repo/ )
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (c) 2006-2014, Salvatore Sanfilippo
Copyright (c) 2006-2012, Salvatore Sanfilippo
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+8 -55
View File
@@ -4,64 +4,17 @@
Redis Manifesto
===============
1 - A DSL for Abstract Data Types. Redis is a DSL (Domain Specific Language)
that manipulates abstract data types and implemented as a TCP daemon.
Commands manipulate a key space where keys are binary-safe strings and
values are different kinds of abstract data types. Every data type
represents an abstract version of a fundamental data structure. For instance
Redis Lists are an abstract representation of linked lists. In Redis, the
essence of a data type isn't just the kind of operations that the data types
support, but also the space and time complexity of the data type and the
operations performed upon it.
1 - A DSL for Abstract Data Types. Redis is a DSL (Domain Specific Language) that manipulates abstract data types and implemented as a TCP daemon. Commands manipulate a key space where keys are binary-safe strings and values are different kinds of abstract data types. Every data type represents an abstract version of a fundamental data structure. For instance Redis Lists are an abstract representation of linked lists. In Redis, the essence of a data type isn't just the kind of operations that the data types support, but also the space and time complexity of the data type and the operations performed upon it.
2 - Memory storage is #1. The Redis data set, composed of defined key-value
pairs, is primarily stored in the computer's memory. The amount of memory in
all kinds of computers, including entry-level servers, is increasing
significantly each year. Memory is fast, and allows Redis to have very
predictable performance. Datasets composed of 10k or 40 millions keys will
perform similarly. Complex data types like Redis Sorted Sets are easy to
implement and manipulate in memory with good performance, making Redis very
simple. Redis will continue to explore alternative options (where data can
be optionally stored on disk, say) but the main goal of the project remains
the development of an in-memory database.
2 - Memory storage is #1. The Redis data set, composed of defined key-value pairs, is primarily stored in the computer's memory. The amount of memory in all kinds of computers, including entry-level servers, is increasing significantly each year. Memory is fast, and allows Redis to have very predictable performance. Datasets composed of 10k or 40 millions keys will perform similarly. Complex data types like Redis Sorted Sets are easy to implement and manipulate in memory with good performance, making Redis very simple. Redis will continue to explore alternative options (where data can be optionally stored on disk, say) but the main goal of the project remains the development of an in-memory database.
3 - Fundamental data structures for a fundamental API. The Redis API is a direct
consequence of fundamental data structures. APIs can often be arbitrary but
not an API that resembles the nature of fundamental data structures. If we
ever meet intelligent life forms from another part of the universe, they'll
likely know, understand and recognize the same basic data structures we have
in our computer science books. Redis will avoid intermediate layers in API,
so that the complexity is obvious and more complex operations can be
performed as the sum of the basic operations.
3 - Fundamental data structures for a fundamental API. The Redis API is a direct consequence of fundamental data structures. APIs can often be arbitrary but not an API that resembles the nature of fundamental data structures. If we ever meet intelligent life forms from another part of the universe, they'll likely know, understand and recognize the same basic data structures we have in our computer science books. Redis will avoid intermediate layers in API, so that the complexity is obvious and more complex operations can be performed as the sum of the basic operations.
4 - Code is like a poem; it's not just something we write to reach some
practical result. Sometimes people that are far from the Redis philosophy
suggest using other code written by other authors (frequently in other
languages) in order to implement something Redis currently lacks. But to us
this is like if Shakespeare decided to end Enrico IV using the Paradiso from
the Divina Commedia. Is using any external code a bad idea? Not at all. Like
in "One Thousand and One Nights" smaller self contained stories are embedded
in a bigger story, we'll be happy to use beautiful self contained libraries
when needed. At the same time, when writing the Redis story we're trying to
write smaller stories that will fit in to other code.
4 - Code is like a poem; it's not just something we write to reach some practical result. Sometimes people that are far from the Redis philosophy suggest using other code written by other authors (frequently in other languages) in order to implement something Redis currently lacks. But to us this is like if Shakespeare decided to end Enrico IV using the Paradiso from the Divina Commedia. Is using any external code a bad idea? Not at all. Like in "One Thousand and One Nights" smaller self contained stories are embedded in a bigger story, we'll be happy to use beautiful self contained libraries when needed. At the same time, when writing the Redis story we're trying to write smaller stories that will fit in to other code.
5 - We're against complexity. We believe designing systems is a fight against
complexity. We'll accept to fight the complexity when it's worthwhile but
we'll try hard to recognize when a small feature is not worth 1000s of lines
of code. Most of the time the best way to fight complexity is by not
creating it at all.
5 - We're against complexity. We believe designing systems is a fight against complexity. We'll accept to fight the complexity when it's worthwhile but we'll try hard to recognize when a small feature is not worth 1000s of lines of code. Most of the time the best way to fight complexity is by not creating it at all.
6 - Two levels of API. The Redis API has two levels: 1) a subset of the API fits
naturally into a distributed version of Redis and 2) a more complex API that
supports multi-key operations. Both are useful if used judiciously but
there's no way to make the more complex multi-keys API distributed in an
opaque way without violating our other principles. We don't want to provide
the illusion of something that will work magically when actually it can't in
all cases. Instead we'll provide commands to quickly migrate keys from one
instance to another to perform multi-key operations and expose the tradeoffs
to the user.
6 - Two levels of API. The Redis API has two levels: 1) a subset of the API fits naturally into a distributed version of Redis and 2) a more complex API that supports multi-key operations. Both are useful if used judiciously but there's no way to make the more complex multi-keys API distributed in an opaque way without violating our other principles. We don't want to provide the illusion of something that will work magically when actually it can't in all cases. Instead we'll provide commands to quickly migrate keys from one instance to another to perform multi-key operations and expose the tradeoffs to the user.
7 - We optimize for joy. We believe writing code is a lot of hard work, and the only way it can be worth is by enjoying it. When there is no longer joy in writing code, the best thing to do is stop. To prevent this, we'll avoid taking paths that will make Redis less of a joy to develop.
7 - We optimize for joy. We believe writing code is a lot of hard work, and the
only way it can be worth is by enjoying it. When there is no longer joy in
writing code, the best thing to do is stop. To prevent this, we'll avoid
taking paths that will make Redis less of a joy to develop.
+4 -15
View File
@@ -26,20 +26,9 @@ After building Redis is a good idea to test it, using:
% make test
Fixing problems building 32 bit binaries
---------
If after building Redis with a 32 bit target you need to rebuild it
with a 64 bit target, or the other way around, you need to perform a
"make distclean" in the root directory of the Redis distribution.
In case of build errors when trying to build a 32 bit binary of Redis, try
the following steps:
* Install the packages libc6-dev-i386 (also try g++-multilib).
* Try using the following command line instead of "make 32bit":
make CFLAGS="-m32 -march=native" LDFLAGS="-m32"
NOTE: if after building Redis with a 32 bit target you need to rebuild it
with a 64 bit target you need to perform a "make clean" in the root
directory of the Redis distribution.
Allocator
---------
@@ -130,7 +119,7 @@ it the proper way for a production system, we have a script doing this
for Ubuntu and Debian systems:
% cd utils
% ./install_server.sh
% ./install_server
The script will ask you a few questions and will setup everything you need
to run Redis properly as a background daemon that will start again on
+2 -2
View File
@@ -54,8 +54,8 @@ linenoise: .make-prerequisites
.PHONY: linenoise
ifeq ($(uname_S),SunOS)
# Make isinf() available
LUA_CFLAGS= -D__C99FEATURES__=1
# Make isinf() available
LUA_CFLAGS= -D__C99FEATURES__=1
endif
LUA_CFLAGS+= -O2 -Wall -DLUA_ANSI $(CFLAGS)
-30
View File
@@ -1,33 +1,3 @@
/*
* Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __HIREDIS_AE_H__
#define __HIREDIS_AE_H__
#include <sys/types.h>
-30
View File
@@ -1,33 +1,3 @@
/*
* Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __HIREDIS_LIBEV_H__
#define __HIREDIS_LIBEV_H__
#include <stdlib.h>
-30
View File
@@ -1,33 +1,3 @@
/*
* Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __HIREDIS_LIBEVENT_H__
#define __HIREDIS_LIBEVENT_H__
#include <event.h>
+1 -1
View File
@@ -7,7 +7,7 @@
#if defined(__sun__)
#define _POSIX_C_SOURCE 200112L
#elif defined(__linux__) || defined(__OpenBSD__) || defined(__NetBSD__)
#elif defined(__linux__)
#define _XOPEN_SOURCE 600
#else
#define _XOPEN_SOURCE
+2 -2
View File
@@ -650,7 +650,7 @@ int redisReaderGetReply(redisReader *r, void **reply) {
/* Discard part of the buffer when we've consumed at least 1k, to avoid
* doing unnecessary calls to memmove() in sds.c. */
if (r->pos >= 1024) {
sdsrange(r->buf,r->pos,-1);
r->buf = sdsrange(r->buf,r->pos,-1);
r->pos = 0;
r->len = sdslen(r->buf);
}
@@ -1125,7 +1125,7 @@ int redisBufferWrite(redisContext *c, int *done) {
sdsfree(c->obuf);
c->obuf = sdsempty();
} else {
sdsrange(c->obuf,nwritten,-1);
c->obuf = sdsrange(c->obuf,nwritten,-1);
}
}
}
+2 -10
View File
@@ -215,17 +215,9 @@ int redisContextConnectTcp(redisContext *c, const char *addr, int port, struct t
hints.ai_family = AF_INET;
hints.ai_socktype = SOCK_STREAM;
/* Try with IPv6 if no IPv4 address was found. We do it in this order since
* in a Redis client you can't afford to test if you have IPv6 connectivity
* as this would add latency to every connect. Otherwise a more sensible
* route could be: Use IPv6 if both addresses are available and there is IPv6
* connectivity. */
if ((rv = getaddrinfo(addr,_port,&hints,&servinfo)) != 0) {
hints.ai_family = AF_INET6;
if ((rv = getaddrinfo(addr,_port,&hints,&servinfo)) != 0) {
__redisSetError(c,REDIS_ERR_OTHER,gai_strerror(rv));
return REDIS_ERR;
}
__redisSetError(c,REDIS_ERR_OTHER,gai_strerror(rv));
return REDIS_ERR;
}
for (p = servinfo; p != NULL; p = p->ai_next) {
if ((s = socket(p->ai_family,p->ai_socktype,p->ai_protocol)) == -1)
+109 -386
View File
@@ -1,6 +1,6 @@
/* SDSLib, A C dynamic strings library
*
* Copyright (c) 2006-2012, Salvatore Sanfilippo <antirez at gmail dot com>
* Copyright (c) 2006-2010, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -32,76 +32,52 @@
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include <assert.h>
#include "sds.h"
#include "zmalloc.h"
/* Create a new sds string with the content specified by the 'init' pointer
* and 'initlen'.
* If NULL is used for 'init' the string is initialized with zero bytes.
*
* The string is always null-termined (all the sds strings are, always) so
* even if you create an sds string with:
*
* mystring = sdsnewlen("abc",3");
*
* You can print the string with printf() as there is an implicit \0 at the
* end of the string. However the string is binary safe and can contain
* \0 characters in the middle, as the length is stored in the sds header. */
#ifdef SDS_ABORT_ON_OOM
static void sdsOomAbort(void) {
fprintf(stderr,"SDS: Out Of Memory (SDS_ABORT_ON_OOM defined)\n");
abort();
}
#endif
sds sdsnewlen(const void *init, size_t initlen) {
struct sdshdr *sh;
if (init) {
sh = zmalloc(sizeof(struct sdshdr)+initlen+1);
} else {
sh = zcalloc(sizeof(struct sdshdr)+initlen+1);
}
sh = malloc(sizeof(struct sdshdr)+initlen+1);
#ifdef SDS_ABORT_ON_OOM
if (sh == NULL) sdsOomAbort();
#else
if (sh == NULL) return NULL;
#endif
sh->len = initlen;
sh->free = 0;
if (initlen && init)
memcpy(sh->buf, init, initlen);
if (initlen) {
if (init) memcpy(sh->buf, init, initlen);
else memset(sh->buf,0,initlen);
}
sh->buf[initlen] = '\0';
return (char*)sh->buf;
}
/* Create an empty (zero length) sds string. Even in this case the string
* always has an implicit null term. */
sds sdsempty(void) {
return sdsnewlen("",0);
}
/* Create a new sds string starting from a null termined C string. */
sds sdsnew(const char *init) {
size_t initlen = (init == NULL) ? 0 : strlen(init);
return sdsnewlen(init, initlen);
}
/* Duplicate an sds string. */
sds sdsdup(const sds s) {
return sdsnewlen(s, sdslen(s));
}
/* Free an sds string. No operation is performed if 's' is NULL. */
void sdsfree(sds s) {
if (s == NULL) return;
zfree(s-sizeof(struct sdshdr));
free(s-sizeof(struct sdshdr));
}
/* Set the sds string length to the length as obtained with strlen(), so
* considering as content only up to the first null term character.
*
* This function is useful when the sds string is hacked manually in some
* way, like in the following example:
*
* s = sdsnew("foobar");
* s[2] = '\0';
* sdsupdatelen(s);
* printf("%d\n", sdslen(s));
*
* The output will be "2", but if we comment out the call to sdsupdatelen()
* the output will be "6" as the string was modified but the logical length
* remains 6 bytes. */
void sdsupdatelen(sds s) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
int reallen = strlen(s);
@@ -109,24 +85,7 @@ void sdsupdatelen(sds s) {
sh->len = reallen;
}
/* Modify an sds string on-place to make it empty (zero length).
* However all the existing buffer is not discarded but set as free space
* so that next append operations will not require allocations up to the
* number of bytes previously available. */
void sdsclear(sds s) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
sh->free += sh->len;
sh->len = 0;
sh->buf[0] = '\0';
}
/* Enlarge the free space at the end of the sds string so that the caller
* is sure that after calling this function can overwrite up to addlen
* bytes after the end of the string, plus one more byte for nul term.
*
* Note: this does not change the *length* of the sds string as returned
* by sdslen(), but only the free buffer space we have. */
sds sdsMakeRoomFor(sds s, size_t addlen) {
static sds sdsMakeRoomFor(sds s, size_t addlen) {
struct sdshdr *sh, *newsh;
size_t free = sdsavail(s);
size_t len, newlen;
@@ -134,84 +93,20 @@ sds sdsMakeRoomFor(sds s, size_t addlen) {
if (free >= addlen) return s;
len = sdslen(s);
sh = (void*) (s-(sizeof(struct sdshdr)));
newlen = (len+addlen);
if (newlen < SDS_MAX_PREALLOC)
newlen *= 2;
else
newlen += SDS_MAX_PREALLOC;
newsh = zrealloc(sh, sizeof(struct sdshdr)+newlen+1);
newlen = (len+addlen)*2;
newsh = realloc(sh, sizeof(struct sdshdr)+newlen+1);
#ifdef SDS_ABORT_ON_OOM
if (newsh == NULL) sdsOomAbort();
#else
if (newsh == NULL) return NULL;
#endif
newsh->free = newlen - len;
return newsh->buf;
}
/* Reallocate the sds string so that it has no free space at the end. The
* contained string remains not altered, but next concatenation operations
* will require a reallocation.
*
* After the call, the passed sds string is no longer valid and all the
* references must be substituted with the new pointer returned by the call. */
sds sdsRemoveFreeSpace(sds s) {
struct sdshdr *sh;
sh = (void*) (s-(sizeof(struct sdshdr)));
sh = zrealloc(sh, sizeof(struct sdshdr)+sh->len+1);
sh->free = 0;
return sh->buf;
}
/* Return the total size of the allocation of the specifed sds string,
* including:
* 1) The sds header before the pointer.
* 2) The string.
* 3) The free buffer at the end if any.
* 4) The implicit null term.
*/
size_t sdsAllocSize(sds s) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
return sizeof(*sh)+sh->len+sh->free+1;
}
/* Increment the sds length and decrements the left free space at the
* end of the string according to 'incr'. Also set the null term
* in the new end of the string.
*
* This function is used in order to fix the string length after the
* user calls sdsMakeRoomFor(), writes something after the end of
* the current string, and finally needs to set the new length.
*
* Note: it is possible to use a negative increment in order to
* right-trim the string.
*
* Usage example:
*
* Using sdsIncrLen() and sdsMakeRoomFor() it is possible to mount the
* following schema, to cat bytes coming from the kernel to the end of an
* sds string without copying into an intermediate buffer:
*
* oldlen = sdslen(s);
* s = sdsMakeRoomFor(s, BUFFER_SIZE);
* nread = read(fd, s+oldlen, BUFFER_SIZE);
* ... check for nread <= 0 and handle it ...
* sdsIncrLen(s, nread);
*/
void sdsIncrLen(sds s, int incr) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
assert(sh->free >= incr);
sh->len += incr;
sh->free -= incr;
assert(sh->free >= 0);
s[sh->len] = '\0';
}
/* Grow the sds to have the specified length. Bytes that were not part of
* the original length of the sds will be set to zero.
*
* if the specified length is smaller than the current length, no operation
* is performed. */
* the original length of the sds will be set to zero. */
sds sdsgrowzero(sds s, size_t len) {
struct sdshdr *sh = (void*)(s-(sizeof(struct sdshdr)));
size_t totlen, curlen = sh->len;
@@ -229,11 +124,6 @@ sds sdsgrowzero(sds s, size_t len) {
return s;
}
/* Append the specified binary-safe string pointed by 't' of 'len' bytes to the
* end of the specified sds string 's'.
*
* After the call, the passed sds string is no longer valid and all the
* references must be substituted with the new pointer returned by the call. */
sds sdscatlen(sds s, const void *t, size_t len) {
struct sdshdr *sh;
size_t curlen = sdslen(s);
@@ -248,25 +138,11 @@ sds sdscatlen(sds s, const void *t, size_t len) {
return s;
}
/* Append the specified null termianted C string to the sds string 's'.
*
* After the call, the passed sds string is no longer valid and all the
* references must be substituted with the new pointer returned by the call. */
sds sdscat(sds s, const char *t) {
return sdscatlen(s, t, strlen(t));
}
/* Append the specified sds 't' to the existing sds 's'.
*
* After the call, the modified sds string is no longer valid and all the
* references must be substituted with the new pointer returned by the call. */
sds sdscatsds(sds s, const sds t) {
return sdscatlen(s, t, sdslen(t));
}
/* Destructively modify the sds string 's' to hold the specified binary
* safe string pointed by 't' of length 'len' bytes. */
sds sdscpylen(sds s, const char *t, size_t len) {
sds sdscpylen(sds s, char *t, size_t len) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
size_t totlen = sh->free+sh->len;
@@ -283,52 +159,37 @@ sds sdscpylen(sds s, const char *t, size_t len) {
return s;
}
/* Like sdscpylen() but 't' must be a null-termined string so that the length
* of the string is obtained with strlen(). */
sds sdscpy(sds s, const char *t) {
sds sdscpy(sds s, char *t) {
return sdscpylen(s, t, strlen(t));
}
/* Like sdscatpritf() but gets va_list instead of being variadic. */
sds sdscatvprintf(sds s, const char *fmt, va_list ap) {
va_list cpy;
char *buf, *t;
size_t buflen = 16;
while(1) {
buf = zmalloc(buflen);
buf = malloc(buflen);
#ifdef SDS_ABORT_ON_OOM
if (buf == NULL) sdsOomAbort();
#else
if (buf == NULL) return NULL;
#endif
buf[buflen-2] = '\0';
va_copy(cpy,ap);
vsnprintf(buf, buflen, fmt, cpy);
if (buf[buflen-2] != '\0') {
zfree(buf);
free(buf);
buflen *= 2;
continue;
}
break;
}
t = sdscat(s, buf);
zfree(buf);
free(buf);
return t;
}
/* Append to the sds string 's' a string obtained using printf-alike format
* specifier.
*
* After the call, the modified sds string is no longer valid and all the
* references must be substituted with the new pointer returned by the call.
*
* Example:
*
* s = sdsempty("Sum is: ");
* s = sdscatprintf(s,"%d+%d = %d",a,b,a+b).
*
* Often you need to create a string from scratch with the printf-alike
* format. When this is the need, just use sdsempty() as the target string:
*
* s = sdscatprintf(sdsempty(), "... your format ...", args);
*/
sds sdscatprintf(sds s, const char *fmt, ...) {
va_list ap;
char *t;
@@ -338,20 +199,6 @@ sds sdscatprintf(sds s, const char *fmt, ...) {
return t;
}
/* Remove the part of the string from left and from right composed just of
* contiguous characters found in 'cset', that is a null terminted C string.
*
* After the call, the modified sds string is no longer valid and all the
* references must be substituted with the new pointer returned by the call.
*
* Example:
*
* s = sdsnew("AA...AA.a.aa.aHelloWorld :::");
* s = sdstrim(s,"A. :");
* printf("%s\n", s);
*
* Output will be just "Hello World".
*/
sds sdstrim(sds s, const char *cset) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
char *start, *end, *sp, *ep;
@@ -369,27 +216,11 @@ sds sdstrim(sds s, const char *cset) {
return s;
}
/* Turn the string into a smaller (or equal) string containing only the
* substring specified by the 'start' and 'end' indexes.
*
* start and end can be negative, where -1 means the last character of the
* string, -2 the penultimate character, and so forth.
*
* The interval is inclusive, so the start and end characters will be part
* of the resulting string.
*
* The string is modified in-place.
*
* Example:
*
* s = sdsnew("Hello World");
* sdstrim(s,1,-1); => "ello Worl"
*/
void sdsrange(sds s, int start, int end) {
sds sdsrange(sds s, int start, int end) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
size_t newlen, len = sdslen(s);
if (len == 0) return;
if (len == 0) return s;
if (start < 0) {
start = len+start;
if (start < 0) start = 0;
@@ -413,34 +244,22 @@ void sdsrange(sds s, int start, int end) {
sh->buf[newlen] = 0;
sh->free = sh->free+(sh->len-newlen);
sh->len = newlen;
return s;
}
/* Apply tolower() to every character of the sds string 's'. */
void sdstolower(sds s) {
int len = sdslen(s), j;
for (j = 0; j < len; j++) s[j] = tolower(s[j]);
}
/* Apply toupper() to every character of the sds string 's'. */
void sdstoupper(sds s) {
int len = sdslen(s), j;
for (j = 0; j < len; j++) s[j] = toupper(s[j]);
}
/* Compare two sds strings s1 and s2 with memcmp().
*
* Return value:
*
* 1 if s1 > s2.
* -1 if s1 < s2.
* 0 if s1 and s2 are exactly the same binary string.
*
* If two strings share exactly the same prefix, but one of the two has
* additional characters, the longer string is considered to be greater than
* the smaller one. */
int sdscmp(const sds s1, const sds s2) {
int sdscmp(sds s1, sds s2) {
size_t l1, l2, minlen;
int cmp;
@@ -468,15 +287,14 @@ int sdscmp(const sds s1, const sds s2) {
* requires length arguments. sdssplit() is just the
* same function but for zero-terminated strings.
*/
sds *sdssplitlen(const char *s, int len, const char *sep, int seplen, int *count) {
sds *sdssplitlen(char *s, int len, char *sep, int seplen, int *count) {
int elements = 0, slots = 5, start = 0, j;
sds *tokens;
if (seplen < 1 || len < 0) return NULL;
tokens = zmalloc(sizeof(sds)*slots);
if (tokens == NULL) return NULL;
sds *tokens = malloc(sizeof(sds)*slots);
#ifdef SDS_ABORT_ON_OOM
if (tokens == NULL) sdsOomAbort();
#endif
if (seplen < 1 || len < 0 || tokens == NULL) return NULL;
if (len == 0) {
*count = 0;
return tokens;
@@ -487,14 +305,26 @@ sds *sdssplitlen(const char *s, int len, const char *sep, int seplen, int *count
sds *newtokens;
slots *= 2;
newtokens = zrealloc(tokens,sizeof(sds)*slots);
if (newtokens == NULL) goto cleanup;
newtokens = realloc(tokens,sizeof(sds)*slots);
if (newtokens == NULL) {
#ifdef SDS_ABORT_ON_OOM
sdsOomAbort();
#else
goto cleanup;
#endif
}
tokens = newtokens;
}
/* search the separator */
if ((seplen == 1 && *(s+j) == sep[0]) || (memcmp(s+j,sep,seplen) == 0)) {
tokens[elements] = sdsnewlen(s+start,j-start);
if (tokens[elements] == NULL) goto cleanup;
if (tokens[elements] == NULL) {
#ifdef SDS_ABORT_ON_OOM
sdsOomAbort();
#else
goto cleanup;
#endif
}
elements++;
start = j+seplen;
j = j+seplen-1; /* skip the separator */
@@ -502,33 +332,35 @@ sds *sdssplitlen(const char *s, int len, const char *sep, int seplen, int *count
}
/* Add the final element. We are sure there is room in the tokens array. */
tokens[elements] = sdsnewlen(s+start,len-start);
if (tokens[elements] == NULL) goto cleanup;
if (tokens[elements] == NULL) {
#ifdef SDS_ABORT_ON_OOM
sdsOomAbort();
#else
goto cleanup;
#endif
}
elements++;
*count = elements;
return tokens;
#ifndef SDS_ABORT_ON_OOM
cleanup:
{
int i;
for (i = 0; i < elements; i++) sdsfree(tokens[i]);
zfree(tokens);
*count = 0;
free(tokens);
return NULL;
}
#endif
}
/* Free the result returned by sdssplitlen(), or do nothing if 'tokens' is NULL. */
void sdsfreesplitres(sds *tokens, int count) {
if (!tokens) return;
while(count--)
sdsfree(tokens[count]);
zfree(tokens);
free(tokens);
}
/* Create an sds string from a long long value. It is much faster than:
*
* sdscatprintf(sdsempty(),"%lld\n", value);
*/
sds sdsfromlonglong(long long value) {
char buf[32], *p;
unsigned long long v;
@@ -544,14 +376,10 @@ sds sdsfromlonglong(long long value) {
return sdsnewlen(p,32-(p-buf));
}
/* Append to the sds string "s" an escaped string representation where
* all the non-printable characters (tested with isprint()) are turned into
* escapes in the form "\n\r\a...." or "\x<hex-number>".
*
* After the call, the modified sds string is no longer valid and all the
* references must be substituted with the new pointer returned by the call. */
sds sdscatrepr(sds s, const char *p, size_t len) {
sds sdscatrepr(sds s, char *p, size_t len) {
s = sdscatlen(s,"\"",1);
if (s == NULL) return NULL;
while(len--) {
switch(*p) {
case '\\':
@@ -571,64 +399,27 @@ sds sdscatrepr(sds s, const char *p, size_t len) {
break;
}
p++;
if (s == NULL) return NULL;
}
return sdscatlen(s,"\"",1);
}
/* Helper function for sdssplitargs() that returns non zero if 'c'
* is a valid hex digit. */
int is_hex_digit(char c) {
return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') ||
(c >= 'A' && c <= 'F');
}
/* Helper function for sdssplitargs() that converts an hex digit into an
* integer from 0 to 15 */
int hex_digit_to_int(char c) {
switch(c) {
case '0': return 0;
case '1': return 1;
case '2': return 2;
case '3': return 3;
case '4': return 4;
case '5': return 5;
case '6': return 6;
case '7': return 7;
case '8': return 8;
case '9': return 9;
case 'a': case 'A': return 10;
case 'b': case 'B': return 11;
case 'c': case 'C': return 12;
case 'd': case 'D': return 13;
case 'e': case 'E': return 14;
case 'f': case 'F': return 15;
default: return 0;
}
}
/* Split a line into arguments, where every argument can be in the
* following programming-language REPL-alike form:
*
* foo bar "newline are supported\n" and "\xff\x00otherstuff"
*
* The number of arguments is stored into *argc, and an array
* of sds is returned.
*
* The caller should free the resulting array of sds strings with
* sdsfreesplitres().
* of sds is returned. The caller should sdsfree() all the returned
* strings and finally free() the array itself.
*
* Note that sdscatrepr() is able to convert back a string into
* a quoted string in the same format sdssplitargs() is able to parse.
*
* The function returns the allocated tokens on success, even when the
* input string is empty, or NULL if the input contains unbalanced
* quotes or closed quotes followed by non space characters
* as in: "foo"bar or "foo'
*/
sds *sdssplitargs(const char *line, int *argc) {
const char *p = line;
sds *sdssplitargs(char *line, int *argc) {
char *p = line;
char *current = NULL;
char **vector = NULL;
char **vector = NULL, **_vector = NULL;
*argc = 0;
while(1) {
@@ -636,24 +427,17 @@ sds *sdssplitargs(const char *line, int *argc) {
while(*p && isspace(*p)) p++;
if (*p) {
/* get a token */
int inq=0; /* set to 1 if we are in "quotes" */
int insq=0; /* set to 1 if we are in 'single quotes' */
int inq=0; /* set to 1 if we are in "quotes" */
int done=0;
if (current == NULL) current = sdsempty();
if (current == NULL) {
current = sdsempty();
if (current == NULL) goto err;
}
while(!done) {
if (inq) {
if (*p == '\\' && *(p+1) == 'x' &&
is_hex_digit(*(p+2)) &&
is_hex_digit(*(p+3)))
{
unsigned char byte;
byte = (hex_digit_to_int(*(p+2))*16)+
hex_digit_to_int(*(p+3));
current = sdscatlen(current,(char*)&byte,1);
p += 3;
} else if (*p == '\\' && *(p+1)) {
if (*p == '\\' && *(p+1)) {
char c;
p++;
@@ -667,23 +451,7 @@ sds *sdssplitargs(const char *line, int *argc) {
}
current = sdscatlen(current,&c,1);
} else if (*p == '"') {
/* closing quote must be followed by a space or
* nothing at all. */
if (*(p+1) && !isspace(*(p+1))) goto err;
done=1;
} else if (!*p) {
/* unterminated quotes */
goto err;
} else {
current = sdscatlen(current,p,1);
}
} else if (insq) {
if (*p == '\\' && *(p+1) == '\'') {
p++;
current = sdscatlen(current,"'",1);
} else if (*p == '\'') {
/* closing quote must be followed by a space or
* nothing at all. */
/* closing quote must be followed by a space */
if (*(p+1) && !isspace(*(p+1))) goto err;
done=1;
} else if (!*p) {
@@ -704,24 +472,23 @@ sds *sdssplitargs(const char *line, int *argc) {
case '"':
inq=1;
break;
case '\'':
insq=1;
break;
default:
current = sdscatlen(current,p,1);
break;
}
}
if (*p) p++;
if (current == NULL) goto err;
}
/* add the token to the vector */
vector = zrealloc(vector,((*argc)+1)*sizeof(char*));
_vector = realloc(vector,((*argc)+1)*sizeof(char*));
if (_vector == NULL) goto err;
vector = _vector;
vector[*argc] = current;
(*argc)++;
current = NULL;
} else {
/* Even on empty input string return something not NULL. */
if (vector == NULL) vector = zmalloc(sizeof(void*));
return vector;
}
}
@@ -729,55 +496,30 @@ sds *sdssplitargs(const char *line, int *argc) {
err:
while((*argc)--)
sdsfree(vector[*argc]);
zfree(vector);
if (current) sdsfree(current);
*argc = 0;
if (vector != NULL) free(vector);
if (current != NULL) sdsfree(current);
return NULL;
}
/* Modify the string substituting all the occurrences of the set of
* characters specified in the 'from' string to the corresponding character
* in the 'to' array.
*
* For instance: sdsmapchars(mystring, "ho", "01", 2)
* will have the effect of turning the string "hello" into "0ell1".
*
* The function returns the sds string pointer, that is always the same
* as the input pointer since no resize is needed. */
sds sdsmapchars(sds s, const char *from, const char *to, size_t setlen) {
size_t j, i, l = sdslen(s);
for (j = 0; j < l; j++) {
for (i = 0; i < setlen; i++) {
if (s[j] == from[i]) {
s[j] = to[i];
break;
}
}
}
return s;
}
/* Join an array of C strings using the specified separator (also a C string).
* Returns the result as an sds string. */
sds sdsjoin(char **argv, int argc, char *sep) {
sds join = sdsempty();
int j;
for (j = 0; j < argc; j++) {
join = sdscat(join, argv[j]);
if (j != argc-1) join = sdscat(join,sep);
}
return join;
}
#ifdef SDS_TEST_MAIN
#include <stdio.h>
#include "testhelp.h"
int __failed_tests = 0;
int __test_num = 0;
#define test_cond(descr,_c) do { \
__test_num++; printf("%d - %s: ", __test_num, descr); \
if(_c) printf("PASSED\n"); else {printf("FAILED\n"); __failed_tests++;} \
} while(0);
#define test_report() do { \
printf("%d tests, %d passed, %d failed\n", __test_num, \
__test_num-__failed_tests, __failed_tests); \
if (__failed_tests) { \
printf("=== WARNING === We have failed tests here...\n"); \
} \
} while(0);
int main(void) {
{
struct sdshdr *sh;
sds x = sdsnew("foo"), y;
test_cond("Create a string and obtain the length",
@@ -857,26 +599,7 @@ int main(void) {
x = sdsnew("aar");
y = sdsnew("bar");
test_cond("sdscmp(bar,bar)", sdscmp(x,y) < 0)
{
int oldfree;
sdsfree(x);
x = sdsnew("0");
sh = (void*) (x-(sizeof(struct sdshdr)));
test_cond("sdsnew() free/len buffers", sh->len == 1 && sh->free == 0);
x = sdsMakeRoomFor(x,1);
sh = (void*) (x-(sizeof(struct sdshdr)));
test_cond("sdsMakeRoomFor()", sh->len == 1 && sh->free > 0);
oldfree = sh->free;
x[1] = '1';
sdsIncrLen(x,1);
test_cond("sdsIncrLen() -- content", x[0] == '0' && x[1] == '1');
test_cond("sdsIncrLen() -- len", sh->len == 2);
test_cond("sdsIncrLen() -- free", sh->free == oldfree-1);
}
}
test_report()
return 0;
}
#endif
+8 -20
View File
@@ -31,8 +31,6 @@
#ifndef __SDS_H
#define __SDS_H
#define SDS_MAX_PREALLOC (1024*1024)
#include <sys/types.h>
#include <stdarg.h>
@@ -60,13 +58,12 @@ sds sdsempty(void);
size_t sdslen(const sds s);
sds sdsdup(const sds s);
void sdsfree(sds s);
size_t sdsavail(const sds s);
size_t sdsavail(sds s);
sds sdsgrowzero(sds s, size_t len);
sds sdscatlen(sds s, const void *t, size_t len);
sds sdscat(sds s, const char *t);
sds sdscatsds(sds s, const sds t);
sds sdscpylen(sds s, const char *t, size_t len);
sds sdscpy(sds s, const char *t);
sds sdscpylen(sds s, char *t, size_t len);
sds sdscpy(sds s, char *t);
sds sdscatvprintf(sds s, const char *fmt, va_list ap);
#ifdef __GNUC__
@@ -77,24 +74,15 @@ sds sdscatprintf(sds s, const char *fmt, ...);
#endif
sds sdstrim(sds s, const char *cset);
void sdsrange(sds s, int start, int end);
sds sdsrange(sds s, int start, int end);
void sdsupdatelen(sds s);
void sdsclear(sds s);
int sdscmp(const sds s1, const sds s2);
sds *sdssplitlen(const char *s, int len, const char *sep, int seplen, int *count);
int sdscmp(sds s1, sds s2);
sds *sdssplitlen(char *s, int len, char *sep, int seplen, int *count);
void sdsfreesplitres(sds *tokens, int count);
void sdstolower(sds s);
void sdstoupper(sds s);
sds sdsfromlonglong(long long value);
sds sdscatrepr(sds s, const char *p, size_t len);
sds *sdssplitargs(const char *line, int *argc);
sds sdsmapchars(sds s, const char *from, const char *to, size_t setlen);
sds sdsjoin(char **argv, int argc, char *sep);
/* Low level functions exposed to the user API */
sds sdsMakeRoomFor(sds s, size_t addlen);
void sdsIncrLen(sds s, int incr);
sds sdsRemoveFreeSpace(sds s);
size_t sdsAllocSize(sds s);
sds sdscatrepr(sds s, char *p, size_t len);
sds *sdssplitargs(char *line, int *argc);
#endif
-13
View File
@@ -1,13 +0,0 @@
/* Drop in replacement for zmalloc.h in order to just use libc malloc without
* any wrappering. */
#ifndef ZMALLOC_H
#define ZMALLOC_H
#define zmalloc malloc
#define zrealloc realloc
#define zcalloc(x) calloc(x,1)
#define zfree free
#define zstrdup strdup
#endif
+5
View File
@@ -2,19 +2,24 @@
/config.stamp
/config.log
/config.status
/configure
/doc/html.xsl
/doc/manpages.xsl
/doc/jemalloc.xml
/doc/jemalloc.html
/doc/jemalloc.3
/lib/
/Makefile
/include/jemalloc/internal/jemalloc_internal\.h
/include/jemalloc/internal/size_classes\.h
/include/jemalloc/jemalloc\.h
/include/jemalloc/jemalloc_defs\.h
/test/jemalloc_test\.h
/src/*.[od]
/test/*.[od]
/test/*.out
/test/[a-zA-Z_]*
!test/*.c
!test/*.exp
/VERSION
/bin/jemalloc.sh
+232 -696
View File
File diff suppressed because it is too large Load Diff
+24 -35
View File
@@ -3,44 +3,39 @@
*
* See linenoise.c for more information.
*
* ------------------------------------------------------------------------
*
* Copyright (c) 2010, Salvatore Sanfilippo <antirez at gmail dot com>
* Copyright (c) 2010, Pieter Noordhuis <pcnoordhuis at gmail dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
* met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
*
* * Redistributions in binary form must reproduce the above copyright
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __LINENOISE_H
#define __LINENOISE_H
#ifdef __cplusplus
extern "C" {
#endif
typedef struct linenoiseCompletions {
size_t len;
char **cvec;
@@ -48,19 +43,13 @@ typedef struct linenoiseCompletions {
typedef void(linenoiseCompletionCallback)(const char *, linenoiseCompletions *);
void linenoiseSetCompletionCallback(linenoiseCompletionCallback *);
void linenoiseAddCompletion(linenoiseCompletions *, const char *);
void linenoiseAddCompletion(linenoiseCompletions *, char *);
char *linenoise(const char *prompt);
int linenoiseHistoryAdd(const char *line);
int linenoiseHistorySetMaxLen(int len);
int linenoiseHistorySave(const char *filename);
int linenoiseHistoryLoad(const char *filename);
int linenoiseHistorySave(char *filename);
int linenoiseHistoryLoad(char *filename);
void linenoiseClearScreen(void);
void linenoiseSetMultiLine(int ml);
void linenoisePrintKeyCodes(void);
#ifdef __cplusplus
}
#endif
#endif /* __LINENOISE_H */
+1 -1
View File
@@ -9,7 +9,7 @@ For details and rationale, see http://www.lua.org/license.html .
===============================================================================
Copyright (C) 1994-2012 Lua.org, PUC-Rio.
Copyright (C) 1994-2008 Lua.org, PUC-Rio.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+1 -1
View File
@@ -48,7 +48,7 @@ TO_MAN= lua.1 luac.1
# Lua version and release.
V= 5.1
R= 5.1.5
R= 5.1.4
all: $(PLAT)
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 797 B

+82 -80
View File
@@ -3,10 +3,11 @@
<HEAD>
<TITLE>Lua 5.1 Reference Manual - contents</TITLE>
<LINK REL="stylesheet" TYPE="text/css" HREF="lua.css">
<META HTTP-EQUIV="content-type" CONTENT="text/html; charset=utf-8">
<META HTTP-EQUIV="content-type" CONTENT="text/html; charset=iso-8859-1">
<STYLE TYPE="text/css">
ul {
list-style-type: none ;
list-style-position: outside ;
}
</STYLE>
</HEAD>
@@ -19,13 +20,7 @@ ul {
Lua 5.1 Reference Manual
</H1>
<P>
The reference manual is the official definition of the Lua language.
For a complete introduction to Lua programming, see the book
<A HREF="http://www.lua.org/docs.html#pil">Programming in Lua</A>.
<P>
This manual is also available as a book:
This is an online version of
<BLOCKQUOTE>
<A HREF="http://www.amazon.com/exec/obidos/ASIN/8590379833/lua-indexmanual-20">
<IMG SRC="cover.png" ALT="" TITLE="buy from Amazon" BORDER=1 ALIGN="left" HSPACE=12>
@@ -34,119 +29,128 @@ This manual is also available as a book:
<BR>by R. Ierusalimschy, L. H. de Figueiredo, W. Celes
<BR>Lua.org, August 2006
<BR>ISBN 85-903798-3-3
<BR><A HREF="http://www.amazon.com/exec/obidos/ASIN/8590379833/lua-indexmanual-20">
<IMG SRC="amazon.gif" ALT="[Buy from Amazon]" BORDER=0></A>
<BR CLEAR="all">
</BLOCKQUOTE>
<P>
<A HREF="http://www.amazon.com/exec/obidos/ASIN/8590379833/lua-indexmanual-20">Buy a copy</A>
of this book and
Buy a copy of this book and
<A HREF="http://www.lua.org/donations.html">help to support</A>
the Lua project.
<P>
The reference manual is the official definition of the Lua language.
For a complete introduction to Lua programming, see the book
<A HREF="http://www.lua.org/docs.html#books">Programming in Lua</A>.
<P>
<A HREF="manual.html">start</A>
&middot;
<A HREF="#contents">contents</A>
&middot;
<A HREF="#index">index</A>
&middot;
<A HREF="http://www.lua.org/manual/">other versions</A>
<A HREF="http://www.lua.org/manual/5.1/pt/">português</A>
&middot;
<A HREF="http://www.lua.org/manual/5.1/es/">español</A>
<HR>
<SMALL>
Copyright &copy; 2006&ndash;2012 Lua.org, PUC-Rio.
Copyright &copy; 2006-2008 Lua.org, PUC-Rio.
Freely available under the terms of the
<A HREF="http://www.lua.org/license.html">Lua license</A>.
<a href="http://www.lua.org/license.html#5">Lua license</a>.
</SMALL>
<P>
<H2><A NAME="contents">Contents</A></H2>
<UL style="padding: 0">
<LI><A HREF="manual.html">1 &ndash; Introduction</A>
<LI><A HREF="manual.html">1 - Introduction</A>
<P>
<LI><A HREF="manual.html#2">2 &ndash; The Language</A>
<LI><A HREF="manual.html#2">2 - The Language</A>
<UL>
<LI><A HREF="manual.html#2.1">2.1 &ndash; Lexical Conventions</A>
<LI><A HREF="manual.html#2.2">2.2 &ndash; Values and Types</A>
<LI><A HREF="manual.html#2.1">2.1 - Lexical Conventions</A>
<LI><A HREF="manual.html#2.2">2.2 - Values and Types</A>
<UL>
<LI><A HREF="manual.html#2.2.1">2.2.1 &ndash; Coercion</A>
<LI><A HREF="manual.html#2.2.1">2.2.1 - Coercion</A>
</UL>
<LI><A HREF="manual.html#2.3">2.3 &ndash; Variables</A>
<LI><A HREF="manual.html#2.4">2.4 &ndash; Statements</A>
<LI><A HREF="manual.html#2.3">2.3 - Variables</A>
<LI><A HREF="manual.html#2.4">2.4 - Statements</A>
<UL>
<LI><A HREF="manual.html#2.4.1">2.4.1 &ndash; Chunks</A>
<LI><A HREF="manual.html#2.4.2">2.4.2 &ndash; Blocks</A>
<LI><A HREF="manual.html#2.4.3">2.4.3 &ndash; Assignment</A>
<LI><A HREF="manual.html#2.4.4">2.4.4 &ndash; Control Structures</A>
<LI><A HREF="manual.html#2.4.5">2.4.5 &ndash; For Statement</A>
<LI><A HREF="manual.html#2.4.6">2.4.6 &ndash; Function Calls as Statements</A>
<LI><A HREF="manual.html#2.4.7">2.4.7 &ndash; Local Declarations</A>
<LI><A HREF="manual.html#2.4.1">2.4.1 - Chunks</A>
<LI><A HREF="manual.html#2.4.2">2.4.2 - Blocks</A>
<LI><A HREF="manual.html#2.4.3">2.4.3 - Assignment</A>
<LI><A HREF="manual.html#2.4.4">2.4.4 - Control Structures</A>
<LI><A HREF="manual.html#2.4.5">2.4.5 - For Statement</A>
<LI><A HREF="manual.html#2.4.6">2.4.6 - Function Calls as Statements</A>
<LI><A HREF="manual.html#2.4.7">2.4.7 - Local Declarations</A>
</UL>
<LI><A HREF="manual.html#2.5">2.5 &ndash; Expressions</A>
<LI><A HREF="manual.html#2.5">2.5 - Expressions</A>
<UL>
<LI><A HREF="manual.html#2.5.1">2.5.1 &ndash; Arithmetic Operators</A>
<LI><A HREF="manual.html#2.5.2">2.5.2 &ndash; Relational Operators</A>
<LI><A HREF="manual.html#2.5.3">2.5.3 &ndash; Logical Operators</A>
<LI><A HREF="manual.html#2.5.4">2.5.4 &ndash; Concatenation</A>
<LI><A HREF="manual.html#2.5.5">2.5.5 &ndash; The Length Operator</A>
<LI><A HREF="manual.html#2.5.6">2.5.6 &ndash; Precedence</A>
<LI><A HREF="manual.html#2.5.7">2.5.7 &ndash; Table Constructors</A>
<LI><A HREF="manual.html#2.5.8">2.5.8 &ndash; Function Calls</A>
<LI><A HREF="manual.html#2.5.9">2.5.9 &ndash; Function Definitions</A>
<LI><A HREF="manual.html#2.5.1">2.5.1 - Arithmetic Operators</A>
<LI><A HREF="manual.html#2.5.2">2.5.2 - Relational Operators</A>
<LI><A HREF="manual.html#2.5.3">2.5.3 - Logical Operators</A>
<LI><A HREF="manual.html#2.5.4">2.5.4 - Concatenation</A>
<LI><A HREF="manual.html#2.5.5">2.5.5 - The Length Operator</A>
<LI><A HREF="manual.html#2.5.6">2.5.6 - Precedence</A>
<LI><A HREF="manual.html#2.5.7">2.5.7 - Table Constructors</A>
<LI><A HREF="manual.html#2.5.8">2.5.8 - Function Calls</A>
<LI><A HREF="manual.html#2.5.9">2.5.9 - Function Definitions</A>
</UL>
<LI><A HREF="manual.html#2.6">2.6 &ndash; Visibility Rules</A>
<LI><A HREF="manual.html#2.7">2.7 &ndash; Error Handling</A>
<LI><A HREF="manual.html#2.8">2.8 &ndash; Metatables</A>
<LI><A HREF="manual.html#2.9">2.9 &ndash; Environments</A>
<LI><A HREF="manual.html#2.10">2.10 &ndash; Garbage Collection</A>
<LI><A HREF="manual.html#2.6">2.6 - Visibility Rules</A>
<LI><A HREF="manual.html#2.7">2.7 - Error Handling</A>
<LI><A HREF="manual.html#2.8">2.8 - Metatables</A>
<LI><A HREF="manual.html#2.9">2.9 - Environments</A>
<LI><A HREF="manual.html#2.10">2.10 - Garbage Collection</A>
<UL>
<LI><A HREF="manual.html#2.10.1">2.10.1 &ndash; Garbage-Collection Metamethods</A>
<LI><A HREF="manual.html#2.10.2">2.10.2 &ndash; Weak Tables</A>
<LI><A HREF="manual.html#2.10.1">2.10.1 - Garbage-Collection Metamethods</A>
<LI><A HREF="manual.html#2.10.2">2.10.2 - Weak Tables</A>
</UL>
<LI><A HREF="manual.html#2.11">2.11 &ndash; Coroutines</A>
<LI><A HREF="manual.html#2.11">2.11 - Coroutines</A>
</UL>
<P>
<LI><A HREF="manual.html#3">3 &ndash; The Application Program Interface</A>
<LI><A HREF="manual.html#3">3 - The Application Program Interface</A>
<UL>
<LI><A HREF="manual.html#3.1">3.1 &ndash; The Stack</A>
<LI><A HREF="manual.html#3.2">3.2 &ndash; Stack Size</A>
<LI><A HREF="manual.html#3.3">3.3 &ndash; Pseudo-Indices</A>
<LI><A HREF="manual.html#3.4">3.4 &ndash; C Closures</A>
<LI><A HREF="manual.html#3.5">3.5 &ndash; Registry</A>
<LI><A HREF="manual.html#3.6">3.6 &ndash; Error Handling in C</A>
<LI><A HREF="manual.html#3.7">3.7 &ndash; Functions and Types</A>
<LI><A HREF="manual.html#3.8">3.8 &ndash; The Debug Interface</A>
<LI><A HREF="manual.html#3.1">3.1 - The Stack</A>
<LI><A HREF="manual.html#3.2">3.2 - Stack Size</A>
<LI><A HREF="manual.html#3.3">3.3 - Pseudo-Indices</A>
<LI><A HREF="manual.html#3.4">3.4 - C Closures</A>
<LI><A HREF="manual.html#3.5">3.5 - Registry</A>
<LI><A HREF="manual.html#3.6">3.6 - Error Handling in C</A>
<LI><A HREF="manual.html#3.7">3.7 - Functions and Types</A>
<LI><A HREF="manual.html#3.8">3.8 - The Debug Interface</A>
</UL>
<P>
<LI><A HREF="manual.html#4">4 &ndash; The Auxiliary Library</A>
<LI><A HREF="manual.html#4">4 - The Auxiliary Library</A>
<UL>
<LI><A HREF="manual.html#4.1">4.1 &ndash; Functions and Types</A>
<LI><A HREF="manual.html#4.1">4.1 - Functions and Types</A>
</UL>
<P>
<LI><A HREF="manual.html#5">5 &ndash; Standard Libraries</A>
<LI><A HREF="manual.html#5">5 - Standard Libraries</A>
<UL>
<LI><A HREF="manual.html#5.1">5.1 &ndash; Basic Functions</A>
<LI><A HREF="manual.html#5.2">5.2 &ndash; Coroutine Manipulation</A>
<LI><A HREF="manual.html#5.3">5.3 &ndash; Modules</A>
<LI><A HREF="manual.html#5.4">5.4 &ndash; String Manipulation</A>
<LI><A HREF="manual.html#5.1">5.1 - Basic Functions</A>
<LI><A HREF="manual.html#5.2">5.2 - Coroutine Manipulation</A>
<LI><A HREF="manual.html#5.3">5.3 - Modules</A>
<LI><A HREF="manual.html#5.4">5.4 - String Manipulation</A>
<UL>
<LI><A HREF="manual.html#5.4.1">5.4.1 &ndash; Patterns</A>
<LI><A HREF="manual.html#5.4.1">5.4.1 - Patterns</A>
</UL>
<LI><A HREF="manual.html#5.5">5.5 &ndash; Table Manipulation</A>
<LI><A HREF="manual.html#5.6">5.6 &ndash; Mathematical Functions</A>
<LI><A HREF="manual.html#5.7">5.7 &ndash; Input and Output Facilities</A>
<LI><A HREF="manual.html#5.8">5.8 &ndash; Operating System Facilities</A>
<LI><A HREF="manual.html#5.9">5.9 &ndash; The Debug Library</A>
<LI><A HREF="manual.html#5.5">5.5 - Table Manipulation</A>
<LI><A HREF="manual.html#5.6">5.6 - Mathematical Functions</A>
<LI><A HREF="manual.html#5.7">5.7 - Input and Output Facilities</A>
<LI><A HREF="manual.html#5.8">5.8 - Operating System Facilities</A>
<LI><A HREF="manual.html#5.9">5.9 - The Debug Library</A>
</UL>
<P>
<LI><A HREF="manual.html#6">6 &ndash; Lua Stand-alone</A>
<LI><A HREF="manual.html#6">6 - Lua Stand-alone</A>
<P>
<LI><A HREF="manual.html#7">7 &ndash; Incompatibilities with the Previous Version</A>
<LI><A HREF="manual.html#7">7 - Incompatibilities with the Previous Version</A>
<UL>
<LI><A HREF="manual.html#7.1">7.1 &ndash; Changes in the Language</A>
<LI><A HREF="manual.html#7.2">7.2 &ndash; Changes in the Libraries</A>
<LI><A HREF="manual.html#7.3">7.3 &ndash; Changes in the API</A>
<LI><A HREF="manual.html#7.1">7.1 - Changes in the Language</A>
<LI><A HREF="manual.html#7.2">7.2 - Changes in the Libraries</A>
<LI><A HREF="manual.html#7.3">7.3 - Changes in the API</A>
</UL>
<P>
<LI><A HREF="manual.html#8">8 &ndash; The Complete Syntax of Lua</A>
<LI><A HREF="manual.html#8">8 - The Complete Syntax of Lua</A>
</UL>
<H2><A NAME="index">Index</A></H2>
@@ -156,8 +160,6 @@ Freely available under the terms of the
<H3><A NAME="functions">Lua functions</A></H3>
<A HREF="manual.html#pdf-_G">_G</A><BR>
<A HREF="manual.html#pdf-_VERSION">_VERSION</A><BR>
<P>
<A HREF="manual.html#pdf-assert">assert</A><BR>
<A HREF="manual.html#pdf-collectgarbage">collectgarbage</A><BR>
<A HREF="manual.html#pdf-dofile">dofile</A><BR>
@@ -485,12 +487,12 @@ Freely available under the terms of the
<P>
<HR>
<SMALL CLASS="footer">
<SMALL>
Last update:
Mon Feb 13 18:53:32 BRST 2012
Sat Jan 19 13:24:29 BRST 2008
</SMALL>
<!--
Last change: revised for Lua 5.1.5
Last change: revised for Lua 5.1.3
-->
</BODY>
+3 -45
View File
@@ -1,37 +1,17 @@
body {
color: #000000 ;
background-color: #FFFFFF ;
font-family: Helvetica, Arial, sans-serif ;
font-family: sans-serif ;
text-align: justify ;
margin-right: 30px ;
margin-left: 30px ;
margin-right: 20px ;
margin-left: 20px ;
}
h1, h2, h3, h4 {
font-family: Verdana, Geneva, sans-serif ;
font-weight: normal ;
font-style: italic ;
}
h2 {
padding-top: 0.4em ;
padding-bottom: 0.4em ;
padding-left: 30px ;
padding-right: 30px ;
margin-left: -30px ;
background-color: #E0E0FF ;
}
h3 {
padding-left: 0.5em ;
border-left: solid #E0E0FF 1em ;
}
table h3 {
padding-left: 0px ;
border-left: none ;
}
a:link {
color: #000080 ;
background-color: inherit ;
@@ -59,25 +39,3 @@ hr {
background-color: #a0a0a0 ;
}
:target {
background-color: #F8F8F8 ;
padding: 8px ;
border: solid #a0a0a0 2px ;
}
.footer {
color: gray ;
font-size: small ;
}
input[type=text] {
border: solid #a0a0a0 2px ;
border-radius: 2em ;
-moz-border-radius: 2em ;
background-image: url('images/search.png') ;
background-repeat: no-repeat;
background-position: 4px center ;
padding-left: 20px ;
height: 2em ;
}
+2 -13
View File
@@ -1,24 +1,13 @@
h3 code {
font-family: inherit ;
font-size: inherit ;
}
pre, code {
font-size: 12pt ;
pre {
font-size: 105% ;
}
span.apii {
float: right ;
font-family: inherit ;
font-style: normal ;
font-size: small ;
color: gray ;
}
p+h1, ul+h1 {
padding-top: 0.4em ;
padding-bottom: 0.4em ;
padding-left: 30px ;
margin-left: -30px ;
background-color: #E0E0FF ;
}
+14 -17
View File
@@ -19,9 +19,9 @@ Lua 5.1 Reference Manual
by Roberto Ierusalimschy, Luiz Henrique de Figueiredo, Waldemar Celes
<p>
<small>
Copyright &copy; 2006&ndash;2012 Lua.org, PUC-Rio.
Copyright &copy; 2006-2008 Lua.org, PUC-Rio.
Freely available under the terms of the
<a href="http://www.lua.org/license.html">Lua license</a>.
<a href="http://www.lua.org/license.html#5">Lua license</a>.
</small>
<hr>
<p>
@@ -29,13 +29,11 @@ Freely available under the terms of the
<a href="contents.html#contents">contents</A>
&middot;
<a href="contents.html#index">index</A>
&middot;
<A HREF="http://www.lua.org/manual/">other versions</A>
<!-- ====================================================================== -->
<p>
<!-- $Id: manual.of,v 1.49.1.2 2012/01/13 20:23:26 roberto Exp $ -->
<!-- $Id: manual.of,v 1.48 2008/08/18 15:24:20 roberto Exp $ -->
@@ -5647,7 +5645,7 @@ Currently, Lua has the following standard libraries:
<ul>
<li>basic library, which includes the coroutine sub-library;</li>
<li>basic library,</li> which includes the coroutine sub-library;
<li>package library;</li>
@@ -5711,7 +5709,7 @@ when absent, it defaults to "assertion failed!"
<p>
<hr><h3><a name="pdf-collectgarbage"><code>collectgarbage ([opt [, arg]])</code></a></h3>
<hr><h3><a name="pdf-collectgarbage"><code>collectgarbage (opt [, arg])</code></a></h3>
<p>
@@ -5720,11 +5718,6 @@ It performs different functions according to its first argument, <code>opt</code
<ul>
<li><b>"collect":</b>
performs a full garbage-collection cycle.
This is the default option.
</li>
<li><b>"stop":</b>
stops the garbage collector.
</li>
@@ -5733,6 +5726,10 @@ stops the garbage collector.
restarts the garbage collector.
</li>
<li><b>"collect":</b>
performs a full garbage-collection cycle.
</li>
<li><b>"count":</b>
returns the total memory in use by Lua (in Kbytes).
</li>
@@ -5763,7 +5760,7 @@ Returns the previous value for <em>step</em>.
<p>
<hr><h3><a name="pdf-dofile"><code>dofile ([filename])</code></a></h3>
<hr><h3><a name="pdf-dofile"><code>dofile (filename)</code></a></h3>
Opens the named file and executes its contents as a Lua chunk.
When called without arguments,
<code>dofile</code> executes the contents of the standard input (<code>stdin</code>).
@@ -8424,7 +8421,7 @@ Otherwise, it returns the name of the upvalue.
<p>
<hr><h3><a name="pdf-debug.traceback"><code>debug.traceback ([thread,] [message [, level]])</code></a></h3>
<hr><h3><a name="pdf-debug.traceback"><code>debug.traceback ([thread,] [message] [, level])</code></a></h3>
<p>
@@ -8792,12 +8789,12 @@ Here is the complete syntax of Lua in extended BNF.
<HR>
<SMALL CLASS="footer">
<SMALL>
Last update:
Mon Feb 13 18:54:19 BRST 2012
Mon Aug 18 13:25:46 BRT 2008
</SMALL>
<!--
Last change: revised for Lua 5.1.5
Last change: revised for Lua 5.1.4
-->
</body></html>
+2 -2
View File
@@ -12,7 +12,7 @@
Documentation
</H1>
This is the documentation included in the source distribution of Lua 5.1.5.
This is the documentation included in the source distribution of Lua 5.1.4.
<UL>
<LI><A HREF="contents.html">Reference manual</A>
@@ -33,7 +33,7 @@ especially the
<HR>
<SMALL>
Last update:
Fri Feb 3 09:44:42 BRST 2012
Tue Aug 12 14:46:07 BRT 2008
</SMALL>
</BODY>
+1 -1
View File
@@ -5,7 +5,7 @@
# grep '^V=' ../Makefile
V= 5.1
# grep '^R=' ../Makefile
R= 5.1.5
R= 5.1.4
# grep '^INSTALL_.*=' ../Makefile | sed 's/INSTALL_TOP/prefix/'
prefix= /usr/local
+2 -2
View File
@@ -7,7 +7,7 @@
# Your platform. See PLATS for possible values.
PLAT= none
CC?= gcc
CC= gcc
CFLAGS= -O2 -Wall $(MYCFLAGS)
AR= ar rcu
RANLIB= ranlib
@@ -48,7 +48,7 @@ o: $(ALL_O)
a: $(ALL_A)
$(LUA_A): $(CORE_O) $(LIB_O)
$(AR) $@ $(CORE_O) $(LIB_O) # DLL needs all object files
$(AR) $@ $?
$(RANLIB) $@
$(LUA_T): $(LUA_O) $(LUA_A)
+1 -1
View File
@@ -631,7 +631,7 @@ static void base_open (lua_State *L) {
luaL_register(L, "_G", base_funcs);
lua_pushliteral(L, LUA_VERSION);
lua_setglobal(L, "_VERSION"); /* set global _VERSION */
/* `ipairs' and `pairs' need auxiliary functions as upvalues */
/* `ipairs' and `pairs' need auxliliary functions as upvalues */
auxopen(L, "ipairs", luaB_ipairs, ipairsaux);
auxopen(L, "pairs", luaB_pairs, luaB_next);
/* `newproxy' needs a weaktable as upvalue */
+9 -1
View File
@@ -1,5 +1,5 @@
/*
** $Id: lcode.c,v 2.25.1.5 2011/01/31 14:53:16 roberto Exp $
** $Id: lcode.c,v 2.25.1.3 2007/12/28 15:32:23 roberto Exp $
** Code generator for Lua
** See Copyright Notice in lua.h
*/
@@ -544,6 +544,10 @@ void luaK_goiftrue (FuncState *fs, expdesc *e) {
pc = NO_JUMP; /* always true; do nothing */
break;
}
case VFALSE: {
pc = luaK_jump(fs); /* always jump */
break;
}
case VJMP: {
invertjump(fs, e);
pc = e->u.s.info;
@@ -568,6 +572,10 @@ static void luaK_goiffalse (FuncState *fs, expdesc *e) {
pc = NO_JUMP; /* always false; do nothing */
break;
}
case VTRUE: {
pc = luaK_jump(fs); /* always jump */
break;
}
case VJMP: {
pc = e->u.s.info;
break;
+1 -2
View File
@@ -1,5 +1,5 @@
/*
** $Id: ldblib.c,v 1.104.1.4 2009/08/04 18:50:18 roberto Exp $
** $Id: ldblib.c,v 1.104.1.3 2008/01/21 13:11:21 roberto Exp $
** Interface from Lua to its debug API
** See Copyright Notice in lua.h
*/
@@ -45,7 +45,6 @@ static int db_setmetatable (lua_State *L) {
static int db_getfenv (lua_State *L) {
luaL_checkany(L, 1);
lua_getfenv(L, 1);
return 1;
}
+1 -2
View File
@@ -1,5 +1,5 @@
/*
** $Id: ldo.c,v 2.38.1.4 2012/01/18 02:27:10 roberto Exp $
** $Id: ldo.c,v 2.38.1.3 2008/01/18 22:31:22 roberto Exp $
** Stack and Call structure of Lua
** See Copyright Notice in lua.h
*/
@@ -217,7 +217,6 @@ static StkId adjust_varargs (lua_State *L, Proto *p, int actual) {
int nvar = actual - nfixargs; /* number of extra arguments */
lua_assert(p->is_vararg & VARARG_HASARG);
luaC_checkGC(L);
luaD_checkstack(L, p->maxstacksize);
htab = luaH_new(L, nvar, 1); /* create `arg' table */
for (i=0; i<nvar; i++) /* put extra arguments into `arg' table */
setobj2n(L, luaH_setnum(L, htab, i+1), L->top - nvar + i);
+2 -1
View File
@@ -1,5 +1,5 @@
/*
** $Id: lgc.c,v 2.38.1.2 2011/03/18 18:05:38 roberto Exp $
** $Id: lgc.c,v 2.38.1.1 2007/12/27 13:02:25 roberto Exp $
** Garbage Collector
** See Copyright Notice in lua.h
*/
@@ -627,6 +627,7 @@ void luaC_step (lua_State *L) {
}
}
else {
lua_assert(g->totalbytes >= g->estimate);
setthreshold(g);
}
}
+2 -5
View File
@@ -1,5 +1,5 @@
/*
** $Id: liolib.c,v 2.73.1.4 2010/05/14 15:33:51 roberto Exp $
** $Id: liolib.c,v 2.73.1.3 2008/01/18 17:47:43 roberto Exp $
** Standard I/O (and system) library
** See Copyright Notice in lua.h
*/
@@ -276,10 +276,7 @@ static int read_number (lua_State *L, FILE *f) {
lua_pushnumber(L, d);
return 1;
}
else {
lua_pushnil(L); /* "result" to be removed */
return 0; /* read fails */
}
else return 0; /* read fails */
}
+2 -4
View File
@@ -1,5 +1,5 @@
/*
** $Id: llex.c,v 2.20.1.2 2009/11/23 14:58:22 roberto Exp $
** $Id: llex.c,v 2.20.1.1 2007/12/27 13:02:25 roberto Exp $
** Lexical Analyzer
** See Copyright Notice in lua.h
*/
@@ -118,10 +118,8 @@ TString *luaX_newstring (LexState *ls, const char *str, size_t l) {
lua_State *L = ls->L;
TString *ts = luaS_newlstr(L, str, l);
TValue *o = luaH_setstr(L, ls->fs->h, ts); /* entry for `str' */
if (ttisnil(o)) {
if (ttisnil(o))
setbvalue(o, 1); /* make sure `str' will not be collected */
luaC_checkGC(L);
}
return ts;
}
+2 -2
View File
@@ -1,5 +1,5 @@
/*
** $Id: loadlib.c,v 1.52.1.4 2009/09/09 13:17:16 roberto Exp $
** $Id: loadlib.c,v 1.52.1.3 2008/08/06 13:29:28 roberto Exp $
** Dynamic library loader for Lua
** See Copyright Notice in lua.h
**
@@ -639,7 +639,7 @@ LUALIB_API int luaopen_package (lua_State *L) {
lua_pushvalue(L, -1);
lua_replace(L, LUA_ENVIRONINDEX);
/* create `loaders' table */
lua_createtable(L, sizeof(loaders)/sizeof(loaders[0]) - 1, 0);
lua_createtable(L, 0, sizeof(loaders)/sizeof(loaders[0]) - 1);
/* fill it with pre-defined loaders */
for (i=0; loaders[i] != NULL; i++) {
lua_pushcfunction(L, loaders[i]);
+2 -2
View File
@@ -1,5 +1,5 @@
/*
** $Id: lparser.c,v 2.42.1.4 2011/10/21 19:31:42 roberto Exp $
** $Id: lparser.c,v 2.42.1.3 2007/12/28 15:32:23 roberto Exp $
** Lua Parser
** See Copyright Notice in lua.h
*/
@@ -374,9 +374,9 @@ static void close_func (LexState *ls) {
lua_assert(luaG_checkcode(f));
lua_assert(fs->bl == NULL);
ls->fs = fs->prev;
L->top -= 2; /* remove table and prototype from the stack */
/* last token read was anchored in defunct function; must reanchor it */
if (fs) anchor_token(ls);
L->top -= 2; /* remove table and prototype from the stack */
}
+2 -4
View File
@@ -1,5 +1,5 @@
/*
** $Id: lstrlib.c,v 1.132.1.5 2010/05/14 15:34:19 roberto Exp $
** $Id: lstrlib.c,v 1.132.1.4 2008/07/11 17:27:21 roberto Exp $
** Standard library for string operations and pattern-matching
** See Copyright Notice in lua.h
*/
@@ -754,7 +754,6 @@ static void addintlen (char *form) {
static int str_format (lua_State *L) {
int top = lua_gettop(L);
int arg = 1;
size_t sfl;
const char *strfrmt = luaL_checklstring(L, arg, &sfl);
@@ -769,8 +768,7 @@ static int str_format (lua_State *L) {
else { /* format item */
char form[MAX_FORMAT]; /* to store the format (`%...') */
char buff[MAX_ITEM]; /* to store the formatted item */
if (++arg > top)
luaL_argerror(L, arg, "no value");
arg++;
strfrmt = scanformat(L, strfrmt, form);
switch (*strfrmt++) {
case 'c': {
+4 -4
View File
@@ -1,5 +1,5 @@
/*
** $Id: lua.h,v 1.218.1.7 2012/01/13 20:36:20 roberto Exp $
** $Id: lua.h,v 1.218.1.5 2008/08/06 13:30:12 roberto Exp $
** Lua - An Extensible Extension Language
** Lua.org, PUC-Rio, Brazil (http://www.lua.org)
** See Copyright Notice at the end of this file
@@ -17,9 +17,9 @@
#define LUA_VERSION "Lua 5.1"
#define LUA_RELEASE "Lua 5.1.5"
#define LUA_RELEASE "Lua 5.1.4"
#define LUA_VERSION_NUM 501
#define LUA_COPYRIGHT "Copyright (C) 1994-2012 Lua.org, PUC-Rio"
#define LUA_COPYRIGHT "Copyright (C) 1994-2008 Lua.org, PUC-Rio"
#define LUA_AUTHORS "R. Ierusalimschy, L. H. de Figueiredo & W. Celes"
@@ -362,7 +362,7 @@ struct lua_Debug {
/******************************************************************************
* Copyright (C) 1994-2012 Lua.org, PUC-Rio. All rights reserved.
* Copyright (C) 1994-2008 Lua.org, PUC-Rio. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
+1 -1
View File
@@ -374,7 +374,7 @@ static int table_is_an_array(lua_State *L) {
while(lua_next(L,-2)) {
/* Stack: ... key value */
lua_pop(L,1); /* Stack: ... key */
if (lua_type(L,-1) != LUA_TNUMBER) goto not_array;
if (!lua_isnumber(L,-1)) goto not_array;
n = lua_tonumber(L,-1);
idx = n;
if (idx != n || idx < 1) goto not_array;
+52 -119
View File
@@ -1,7 +1,18 @@
#include <assert.h>
#include <ctype.h>
#include <limits.h>
#include <string.h>
#include "lua.h"
#include "lauxlib.h"
/*
** {======================================================
** Library for packing/unpacking structures.
** $Id: struct.c,v 1.4 2012/07/04 18:54:29 roberto Exp $
** $Id: struct.c,v 1.2 2008/04/18 20:06:01 roberto Exp $
** See Copyright Notice at the end of this file
** =======================================================
*/
@@ -14,7 +25,6 @@
** b/B - signed/unsigned byte
** h/H - signed/unsigned short
** l/L - signed/unsigned long
** T - size_t
** i/In - signed/unsigned integer with size `n' (default is size of int)
** cn - sequence of `n' chars (from/to a string); when packing, n==0 means
the whole string; when unpacking, n==0 means use the previous
@@ -26,38 +36,6 @@
*/
#include <assert.h>
#include <ctype.h>
#include <limits.h>
#include <stddef.h>
#include <string.h>
#include "lua.h"
#include "lauxlib.h"
#if (LUA_VERSION_NUM >= 502)
#define luaL_register(L,n,f) luaL_newlib(L,f)
#endif
/* basic integer type */
#if !defined(STRUCT_INT)
#define STRUCT_INT long
#endif
typedef STRUCT_INT Inttype;
/* corresponding unsigned version */
typedef unsigned STRUCT_INT Uinttype;
/* maximum size (in bytes) for integral types */
#define MAXINTSIZE 32
/* is 'x' a power of 2? */
#define isp2(x) ((x) > 0 && ((x) & ((x) - 1)) == 0)
@@ -89,11 +67,11 @@ typedef struct Header {
} Header;
static int getnum (const char **fmt, int df) {
static size_t getnum (const char **fmt, size_t df) {
if (!isdigit(**fmt)) /* no number? */
return df; /* return default value */
else {
int a = 0;
size_t a = 0;
do {
a = a*10 + *((*fmt)++) - '0';
} while (isdigit(**fmt));
@@ -111,40 +89,33 @@ static size_t optsize (lua_State *L, char opt, const char **fmt) {
case 'B': case 'b': return sizeof(char);
case 'H': case 'h': return sizeof(short);
case 'L': case 'l': return sizeof(long);
case 'T': return sizeof(size_t);
case 'f': return sizeof(float);
case 'd': return sizeof(double);
case 'x': return 1;
case 'c': return getnum(fmt, 1);
case 's': case ' ': case '<': case '>': case '!': return 0;
case 'i': case 'I': {
int sz = getnum(fmt, sizeof(int));
if (sz > MAXINTSIZE)
luaL_error(L, "integral size %d is larger than limit of %d",
sz, MAXINTSIZE);
if (!isp2(sz))
luaL_error(L, "integral size %d is not a power of 2", sz);
return sz;
}
default: return 0; /* other cases do not need alignment */
default: {
const char *msg = lua_pushfstring(L, "invalid format option [%c]", opt);
return luaL_argerror(L, 1, msg);
}
}
}
/*
** return number of bytes needed to align an element of size 'size'
** at current position 'len'
*/
static int gettoalign (size_t len, Header *h, int opt, size_t size) {
if (size == 0 || opt == 'c') return 0;
if (size > (size_t)h->align)
size = h->align; /* respect max. alignment */
return (size - (len & (size - 1))) & (size - 1);
if (size > (size_t)h->align) size = h->align; /* respect max. alignment */
return (size - (len & (size - 1))) & (size - 1);
}
/*
** options to control endianess and alignment
*/
static void controloptions (lua_State *L, int opt, const char **fmt,
Header *h) {
static void commoncases (lua_State *L, int opt, const char **fmt, Header *h) {
switch (opt) {
case ' ': return; /* ignore white spaces */
case '>': h->endian = BIG; return;
@@ -156,10 +127,7 @@ static void controloptions (lua_State *L, int opt, const char **fmt,
h->align = a;
return;
}
default: {
const char *msg = lua_pushfstring(L, "invalid format option '%c'", opt);
luaL_argerror(L, 1, msg);
}
default: assert(0);
}
}
@@ -167,27 +135,21 @@ static void controloptions (lua_State *L, int opt, const char **fmt,
static void putinteger (lua_State *L, luaL_Buffer *b, int arg, int endian,
int size) {
lua_Number n = luaL_checknumber(L, arg);
Uinttype value;
char buff[MAXINTSIZE];
if (n < 0)
value = (Uinttype)(Inttype)n;
unsigned long value;
if (n < (lua_Number)LONG_MAX)
value = (long)n;
else
value = (Uinttype)n;
value = (unsigned long)n;
if (endian == LITTLE) {
int i;
for (i = 0; i < size; i++) {
buff[i] = (value & 0xff);
value >>= 8;
}
for (i = 0; i < size; i++)
luaL_addchar(b, (value >> 8*i) & 0xff);
}
else {
int i;
for (i = size - 1; i >= 0; i--) {
buff[i] = (value & 0xff);
value >>= 8;
}
for (i = size - 1; i >= 0; i--)
luaL_addchar(b, (value >> 8*i) & 0xff);
}
luaL_addlstring(b, buff, size);
}
@@ -217,15 +179,15 @@ static int b_pack (lua_State *L) {
size_t size = optsize(L, opt, &fmt);
int toalign = gettoalign(totalsize, &h, opt, size);
totalsize += toalign;
while (toalign-- > 0) luaL_addchar(&b, '\0');
while (toalign-- > 0) luaL_putchar(&b, '\0');
switch (opt) {
case 'b': case 'B': case 'h': case 'H':
case 'l': case 'L': case 'T': case 'i': case 'I': { /* integer types */
case 'l': case 'L': case 'i': case 'I': { /* integer types */
putinteger(L, &b, arg++, h.endian, size);
break;
}
case 'x': {
luaL_addchar(&b, '\0');
luaL_putchar(&b, '\0');
break;
}
case 'f': {
@@ -247,12 +209,12 @@ static int b_pack (lua_State *L) {
luaL_argcheck(L, l >= (size_t)size, arg, "string too short");
luaL_addlstring(&b, s, size);
if (opt == 's') {
luaL_addchar(&b, '\0'); /* add zero at the end */
luaL_putchar(&b, '\0'); /* add zero at the end */
size++;
}
break;
}
default: controloptions(L, opt, &fmt, &h);
default: commoncases(L, opt, &fmt, &h);
}
totalsize += size;
}
@@ -263,27 +225,24 @@ static int b_pack (lua_State *L) {
static lua_Number getinteger (const char *buff, int endian,
int issigned, int size) {
Uinttype l = 0;
int i;
unsigned long l = 0;
if (endian == BIG) {
for (i = 0; i < size; i++) {
l <<= 8;
l |= (Uinttype)(unsigned char)buff[i];
}
int i;
for (i = 0; i < size; i++)
l |= (unsigned long)(unsigned char)buff[size - i - 1] << (i*8);
}
else {
for (i = size - 1; i >= 0; i--) {
l <<= 8;
l |= (Uinttype)(unsigned char)buff[i];
}
int i;
for (i = 0; i < size; i++)
l |= (unsigned long)(unsigned char)buff[i] << (i*8);
}
if (!issigned)
return (lua_Number)l;
else { /* signed format */
Uinttype mask = (Uinttype)(~((Uinttype)0)) << (size*8 - 1);
unsigned long mask = ~(0UL) << (size*8 - 1);
if (l & mask) /* negative value? */
l |= mask; /* signal extension */
return (lua_Number)(Inttype)l;
return (lua_Number)(long)l;
}
}
@@ -301,10 +260,9 @@ static int b_unpack (lua_State *L) {
size_t size = optsize(L, opt, &fmt);
pos += gettoalign(pos, &h, opt, size);
luaL_argcheck(L, pos+size <= ld, 2, "data string too short");
luaL_checkstack(L, 1, "too many results");
switch (opt) {
case 'b': case 'B': case 'h': case 'H':
case 'l': case 'L': case 'T': case 'i': case 'I': { /* integer types */
case 'l': case 'L': case 'i': case 'I': { /* integer types */
int issigned = islower(opt);
lua_Number res = getinteger(data+pos, h.endian, issigned, size);
lua_pushnumber(L, res);
@@ -346,7 +304,7 @@ static int b_unpack (lua_State *L) {
lua_pushlstring(L, data+pos, size - 1);
break;
}
default: controloptions(L, opt, &fmt, &h);
default: commoncases(L, opt, &fmt, &h);
}
pos += size;
}
@@ -354,50 +312,26 @@ static int b_unpack (lua_State *L) {
return lua_gettop(L) - 2;
}
static int b_size (lua_State *L) {
Header h;
const char *fmt = luaL_checkstring(L, 1);
size_t pos = 0;
defaultoptions(&h);
while (*fmt) {
int opt = *fmt++;
size_t size = optsize(L, opt, &fmt);
pos += gettoalign(pos, &h, opt, size);
if (opt == 's')
luaL_argerror(L, 1, "option 's' has no fixed size");
else if (opt == 'c' && size == 0)
luaL_argerror(L, 1, "option 'c0' has no fixed size");
if (!isalnum(opt))
controloptions(L, opt, &fmt, &h);
pos += size;
}
lua_pushinteger(L, pos);
return 1;
}
/* }====================================================== */
static const struct luaL_Reg thislib[] = {
static const struct luaL_reg thislib[] = {
{"pack", b_pack},
{"unpack", b_unpack},
{"size", b_size},
{NULL, NULL}
};
LUALIB_API int luaopen_struct (lua_State *L);
LUALIB_API int luaopen_struct (lua_State *L) {
luaL_register(L, "struct", thislib);
return 1;
}
/******************************************************************************
* Copyright (C) 2010-2012 Lua.org, PUC-Rio. All rights reserved.
* Copyright (C) 2010 Lua.org, PUC-Rio. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
@@ -418,4 +352,3 @@ LUALIB_API int luaopen_struct (lua_State *L) {
* TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
* SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
******************************************************************************/
+2 -6
View File
@@ -1,5 +1,5 @@
/*
** $Id: lvm.c,v 2.63.1.5 2011/08/17 20:43:11 roberto Exp $
** $Id: lvm.c,v 2.63.1.3 2007/12/28 15:32:23 roberto Exp $
** Lua virtual machine
** See Copyright Notice in lua.h
*/
@@ -133,7 +133,6 @@ void luaV_gettable (lua_State *L, const TValue *t, TValue *key, StkId val) {
void luaV_settable (lua_State *L, const TValue *t, TValue *key, StkId val) {
int loop;
TValue temp;
for (loop = 0; loop < MAXTAGLOOP; loop++) {
const TValue *tm;
if (ttistable(t)) { /* `t' is a table? */
@@ -142,7 +141,6 @@ void luaV_settable (lua_State *L, const TValue *t, TValue *key, StkId val) {
if (!ttisnil(oldval) || /* result is no nil? */
(tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) { /* or no TM? */
setobj2t(L, oldval, val);
h->flags = 0;
luaC_barriert(L, h, val);
return;
}
@@ -154,9 +152,7 @@ void luaV_settable (lua_State *L, const TValue *t, TValue *key, StkId val) {
callTM(L, tm, t, key, val);
return;
}
/* else repeat with `tm' */
setobj(L, &temp, tm); /* avoid pointing inside table (may rehash) */
t = &temp;
t = tm; /* else repeat with `tm' */
}
luaG_runerror(L, "loop in settable");
}
+52 -280
View File
@@ -12,26 +12,6 @@
#
# units are case insensitive so 1GB 1Gb 1gB are all the same.
################################## INCLUDES ###################################
# Include one or more other config files here. This is useful if you
# have a standard template that goes to all Redis server but also need
# to customize a few per-server settings. Include files can include
# other files, so use this wisely.
#
# Notice option "include" won't be rewritten by command "CONFIG REWRITE"
# from admin or Redis Sentinel. Since Redis always uses the last processed
# line as value of a configuration directive, you'd better put includes
# at the beginning of this file to avoid overwriting config change at runtime.
#
# If instead you are interested in using includes to override configuration
# options, it is better to use include as the last line.
#
# include /path/to/local.conf
# include /path/to/other.conf
################################ GENERAL #####################################
# By default Redis does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
daemonize no
@@ -44,26 +24,12 @@ pidfile /var/run/redis.pid
# If port 0 is specified Redis will not listen on a TCP socket.
port 6379
# TCP listen() backlog.
# If you want you can bind a single interface, if the bind option is not
# specified all the interfaces will listen for incoming connections.
#
# In high requests-per-second environments you need an high backlog in order
# to avoid slow clients connections issues. Note that the Linux kernel
# will silently truncate it to the value of /proc/sys/net/core/somaxconn so
# make sure to raise both the value of somaxconn and tcp_max_syn_backlog
# in order to get the desired effect.
tcp-backlog 511
# By default Redis listens for connections from all the network interfaces
# available on the server. It is possible to listen to just one or multiple
# interfaces using the "bind" configuration directive, followed by one or
# more IP addresses.
#
# Examples:
#
# bind 192.168.1.100 10.0.0.1
# bind 127.0.0.1
# Specify the path for the Unix socket that will be used to listen for
# Specify the path for the unix socket that will be used to listen for
# incoming connections. There is no default, so Redis will not listen
# on a unix socket when not specified.
#
@@ -73,34 +39,18 @@ tcp-backlog 511
# Close the connection after a client is idle for N seconds (0 to disable)
timeout 0
# TCP keepalive.
#
# If non-zero, use SO_KEEPALIVE to send TCP ACKs to clients in absence
# of communication. This is useful for two reasons:
#
# 1) Detect dead peers.
# 2) Take the connection alive from the point of view of network
# equipment in the middle.
#
# On Linux, the specified value (in seconds) is the period used to send ACKs.
# Note that to close the connection the double of the time is needed.
# On other kernels the period depends on the kernel configuration.
#
# A reasonable value for this option is 60 seconds.
tcp-keepalive 0
# Specify the server verbosity level.
# This can be one of:
# Set server verbosity to 'debug'
# it can be one of:
# debug (a lot of information, useful for development/testing)
# verbose (many rarely useful info, but not a mess like the debug level)
# notice (moderately verbose, what you want in production probably)
# warning (only very important / critical messages are logged)
loglevel notice
# Specify the log file name. Also the empty string can be used to force
# Specify the log file name. Also 'stdout' can be used to force
# Redis to log on the standard output. Note that if you use standard
# output for logging but daemonize, logs will be sent to /dev/null
logfile ""
logfile stdout
# To enable logging to the system logger, just set 'syslog-enabled' to yes,
# and optionally update the other syslog parameters to suit your needs.
@@ -109,7 +59,7 @@ logfile ""
# Specify the syslog identity.
# syslog-ident redis
# Specify the syslog facility. Must be USER or between LOCAL0-LOCAL7.
# Specify the syslog facility. Must be USER or between LOCAL0-LOCAL7.
# syslog-facility local0
# Set the number of databases. The default database is DB 0, you can select
@@ -117,7 +67,7 @@ logfile ""
# dbid is a number between 0 and 'databases'-1
databases 16
################################ SNAPSHOTTING ################################
################################ SNAPSHOTTING #################################
#
# Save the DB on disk:
#
@@ -145,16 +95,16 @@ save 60 10000
# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# This will make the user aware (in an hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
# distater will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# continue to work as usually even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes
@@ -164,7 +114,7 @@ stop-writes-on-bgsave-error yes
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes
# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# Since verison 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
@@ -181,7 +131,7 @@ dbfilename dump.rdb
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
# Also the Append Only File will be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir ./
@@ -202,14 +152,14 @@ dir ./
#
# masterauth <master-password>
# When a slave loses its connection with the master, or when the replication
# When a slave lost the connection with the master, or when the replication
# is still in progress, the slave can act in two different ways:
#
# 1) if slave-serve-stale-data is set to 'yes' (the default) the slave will
# still reply to client requests, possibly with out of date data, or the
# data set may just be empty if this is the first synchronization.
#
# 2) if slave-serve-stale-data is set to 'no' the slave will reply with
# 2) if slave-serve-stale data is set to 'no' the slave will reply with
# an error "SYNC with master in progress" to all the kind of commands
# but to INFO and SLAVEOF.
#
@@ -226,7 +176,7 @@ slave-serve-stale-data yes
# Note: read only slaves are not designed to be exposed to untrusted clients
# on the internet. It's just a protection layer against misuse of the instance.
# Still a read only slave exports by default all the administrative commands
# such as CONFIG, DEBUG, and so forth. To a limited extent you can improve
# such as CONFIG, DEBUG, and so forth. To a limited extend you can improve
# security of read only slaves using 'rename-command' to shadow all the
# administrative / dangerous commands.
slave-read-only yes
@@ -237,11 +187,8 @@ slave-read-only yes
#
# repl-ping-slave-period 10
# The following option sets the replication timeout for:
#
# 1) Bulk transfer I/O during SYNC, from the point of view of slave.
# 2) Master timeout from the point of view of slaves (data, pings).
# 3) Slave timeout from the point of view of masters (REPLCONF ACK pings).
# The following option sets a timeout for both Bulk transfer I/O timeout and
# master data or ping response timeout. The default value is 60 seconds.
#
# It is important to make sure that this value is greater than the value
# specified for repl-ping-slave-period otherwise a timeout will be detected
@@ -249,50 +196,13 @@ slave-read-only yes
#
# repl-timeout 60
# Disable TCP_NODELAY on the slave socket after SYNC?
#
# If you select "yes" Redis will use a smaller number of TCP packets and
# less bandwidth to send data to slaves. But this can add a delay for
# the data to appear on the slave side, up to 40 milliseconds with
# Linux kernels using a default configuration.
#
# If you select "no" the delay for data to appear on the slave side will
# be reduced but more bandwidth will be used for replication.
#
# By default we optimize for low latency, but in very high traffic conditions
# or when the master and slaves are many hops away, turning this to "yes" may
# be a good idea.
repl-disable-tcp-nodelay no
# Set the replication backlog size. The backlog is a buffer that accumulates
# slave data when slaves are disconnected for some time, so that when a slave
# wants to reconnect again, often a full resync is not needed, but a partial
# resync is enough, just passing the portion of data the slave missed while
# disconnected.
#
# The biggest the replication backlog, the longer the time the slave can be
# disconnected and later be able to perform a partial resynchronization.
#
# The backlog is only allocated once there is at least a slave connected.
#
# repl-backlog-size 1mb
# After a master has no longer connected slaves for some time, the backlog
# will be freed. The following option configures the amount of seconds that
# need to elapse, starting from the time the last slave disconnected, for
# the backlog buffer to be freed.
#
# A value of 0 means to never release the backlog.
#
# repl-backlog-ttl 3600
# The slave priority is an integer number published by Redis in the INFO output.
# It is used by Redis Sentinel in order to select a slave to promote into a
# master if the master is no longer working correctly.
#
# A slave with a low priority number is considered better for promotion, so
# for instance if there are three slaves with priority 10, 100, 25 Sentinel will
# pick the one with priority 10, that is the lowest.
# pick the one wtih priority 10, that is the lowest.
#
# However a special priority of 0 marks the slave as not able to perform the
# role of master, so a slave with priority of 0 will never be selected by
@@ -301,28 +211,6 @@ repl-disable-tcp-nodelay no
# By default the priority is 100.
slave-priority 100
# It is possible for a master to stop accepting writes if there are less than
# N slaves connected, having a lag less or equal than M seconds.
#
# The N slaves need to be in "online" state.
#
# The lag in seconds, that must be <= the specified value, is calculated from
# the last ping received from the slave, that is usually sent every second.
#
# This option does not GUARANTEES that N replicas will accept the write, but
# will limit the window of exposure for lost writes in case not enough slaves
# are available, to the specified number of seconds.
#
# For example to require at least 3 slaves with a lag <= 10 seconds use:
#
# min-slaves-to-write 3
# min-slaves-max-lag 10
#
# Setting one or the other to 0 disables the feature.
#
# By default min-slaves-to-write is set to 0 (feature disabled) and
# min-slaves-max-lag is set to 10.
################################## SECURITY ###################################
# Require clients to issue AUTH <PASSWORD> before processing any other
@@ -342,26 +230,23 @@ slave-priority 100
#
# It is possible to change the name of dangerous commands in a shared
# environment. For instance the CONFIG command may be renamed into something
# hard to guess so that it will still be available for internal-use tools
# but not available for general clients.
# of hard to guess so that it will be still available for internal-use
# tools but not available for general clients.
#
# Example:
#
# rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
#
# It is also possible to completely kill a command by renaming it into
# It is also possible to completely kill a command renaming it into
# an empty string:
#
# rename-command CONFIG ""
#
# Please note that changing the name of commands that are logged into the
# AOF file or transmitted to slaves may cause problems.
################################### LIMITS ####################################
# Set the max number of connected clients at the same time. By default
# this limit is set to 10000 clients, however if the Redis server is not
# able to configure the process file limit to allow for the specified limit
# able ot configure the process file limit to allow for the specified limit
# the max number of allowed clients is set to the current file limit
# minus 32 (as Redis reserves a few file descriptors for internal uses).
#
@@ -372,7 +257,7 @@ slave-priority 100
# Don't use more memory than the specified amount of bytes.
# When the memory limit is reached Redis will try to remove keys
# according to the eviction policy selected (see maxmemory-policy).
# accordingly to the eviction policy selected (see maxmemmory-policy).
#
# If Redis can't remove keys according to the policy, or if the policy is
# set to 'noeviction', Redis will start to reply with errors to commands
@@ -380,7 +265,7 @@ slave-priority 100
# to reply to read-only commands like GET.
#
# This option is usually useful when using Redis as an LRU cache, or to set
# a hard memory limit for an instance (using the 'noeviction' policy).
# an hard memory limit for an instance (using the 'noeviction' policy).
#
# WARNING: If you have slaves attached to an instance with maxmemory on,
# the size of the output buffers needed to feed the slaves are subtracted
@@ -396,7 +281,7 @@ slave-priority 100
# maxmemory <bytes>
# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
# is reached. You can select among five behaviors:
# is reached? You can select among five behavior:
#
# volatile-lru -> remove the key with an expire set using an LRU algorithm
# allkeys-lru -> remove any key accordingly to the LRU algorithm
@@ -405,7 +290,7 @@ slave-priority 100
# volatile-ttl -> remove the key with the nearest expire time (minor TTL)
# noeviction -> don't expire at all, just return an error on write operations
#
# Note: with any of the above policies, Redis will return an error on write
# Note: with all the kind of policies, Redis will return an error on write
# operations, when there are not suitable keys for eviction.
#
# At the date of writing this commands are: set setnx setex append
@@ -416,18 +301,15 @@ slave-priority 100
#
# The default is:
#
# maxmemory-policy noeviction
# maxmemory-policy volatile-lru
# LRU and minimal TTL algorithms are not precise algorithms but approximated
# algorithms (in order to save memory), so you can tune it for speed or
# accuracy. For default Redis will check five keys and pick the one that was
# used less recently, you can change the sample size using the following
# configuration directive.
# algorithms (in order to save memory), so you can select as well the sample
# size to check. For instance for default Redis will check three keys and
# pick the one that was used less recently, you can change the sample size
# using the following configuration directive.
#
# The default of 5 produces good enough results. 10 Approximates very closely
# true LRU but costs a bit more CPU. 3 is very fast but not very accurate.
#
# maxmemory-samples 5
# maxmemory-samples 3
############################## APPEND ONLY MODE ###############################
@@ -452,8 +334,7 @@ slave-priority 100
appendonly no
# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"
# appendfilename appendonly.aof
# The fsync() call tells the Operating System to actually write data on disk
# instead to wait for more data in the output buffer. Some OS will really flush
@@ -465,7 +346,7 @@ appendfilename "appendonly.aof"
# always: fsync after every write to the append only log . Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# The default is "everysec" that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
@@ -493,22 +374,21 @@ appendfsync everysec
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# This means that while another child is saving the durability of Redis is
# the same as "appendfsync none", that in practical terms means that it is
# possible to lost up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
no-appendfsync-on-rewrite no
# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
# BGREWRITEAOF when the AOF log size will growth by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# latest rewrite (or if no rewrite happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
@@ -541,50 +421,6 @@ auto-aof-rewrite-min-size 64mb
# Set it to 0 or a negative value for unlimited execution without warnings.
lua-time-limit 5000
################################ REDIS CLUSTER ###############################
#
# Normal Redis instances can't be part of a Redis Cluster; only nodes that are
# started as cluster nodes can. In order to start a Redis instance as a
# cluster node enable the cluster support uncommenting the following:
#
# cluster-enabled yes
# Every cluster node has a cluster configuration file. This file is not
# intended to be edited by hand. It is created and updated by Redis nodes.
# Every Redis Cluster node requires a different cluster configuration file.
# Make sure that instances running in the same system does not have
# overlapping cluster configuration file names.
#
# cluster-config-file nodes-6379.conf
# Cluster node timeout is the amount of milliseconds a node must be unreachable
# for it to be considered in failure state.
# Most other internal time limits are multiple of the node timeout.
#
# cluster-node-timeout 15000
# Cluster slaves are able to migrate to orphaned masters, that are masters
# that are left without working slaves. This improves the cluster ability
# to resist to failures as otherwise an orphaned master can't be failed over
# in case of failure if it has no working slaves.
#
# Slaves migrate to orphaned masters only if there are still at least a
# given number of other working slaves for their old master. This number
# is the "migration barrier". A migration barrier of 1 means that a slave
# will migrate only if there is at least 1 other working slave for its master
# and so forth. It usually reflects the number of slaves you want for every
# master in your cluster.
#
# Default is 1 (slaves migrate only if their masters remain with at least
# one slave). To disable migration just set it to a very large value.
# A value of 0 can be set but is useful only for debugging and dangerous
# in production.
#
# cluster-migration-barrier 1
# In order to setup your cluster make sure to read the documentation
# available at http://redis.io web site.
################################## SLOW LOG ###################################
# The Redis Slow Log is a system to log queries that exceeded a specified
@@ -609,52 +445,6 @@ slowlog-log-slower-than 10000
# You can reclaim memory used by the slow log with SLOWLOG RESET.
slowlog-max-len 128
############################# Event notification ##############################
# Redis can notify Pub/Sub clients about events happening in the key space.
# This feature is documented at http://redis.io/topics/keyspace-events
#
# For instance if keyspace events notification is enabled, and a client
# performs a DEL operation on key "foo" stored in the Database 0, two
# messages will be published via Pub/Sub:
#
# PUBLISH __keyspace@0__:foo del
# PUBLISH __keyevent@0__:del foo
#
# It is possible to select the events that Redis will notify among a set
# of classes. Every class is identified by a single character:
#
# K Keyspace events, published with __keyspace@<db>__ prefix.
# E Keyevent events, published with __keyevent@<db>__ prefix.
# g Generic commands (non-type specific) like DEL, EXPIRE, RENAME, ...
# $ String commands
# l List commands
# s Set commands
# h Hash commands
# z Sorted set commands
# x Expired events (events generated every time a key expires)
# e Evicted events (events generated when a key is evicted for maxmemory)
# A Alias for g$lshzxe, so that the "AKE" string means all the events.
#
# The "notify-keyspace-events" takes as argument a string that is composed
# by zero or multiple characters. The empty string means that notifications
# are disabled at all.
#
# Example: to enable list and generic events, from the point of view of the
# event name, use:
#
# notify-keyspace-events Elg
#
# Example 2: to get the stream of the expired keys subscribing to channel
# name __keyevent@0__:expired use:
#
# notify-keyspace-events Ex
#
# By default all notifications are disabled because most users don't need
# this feature and the feature has some overhead. Note that if you don't
# specify at least one of K or E, no events will be delivered.
notify-keyspace-events ""
############################### ADVANCED CONFIG ###############################
# Hashes are encoded using a memory efficient data structure when they have a
@@ -685,7 +475,7 @@ zset-max-ziplist-value 64
# Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in
# order to help rehashing the main Redis hash table (the one mapping top-level
# keys to values). The hash table implementation Redis uses (see dict.c)
# performs a lazy rehashing: the more operation you run into a hash table
# performs a lazy rehashing: the more operation you run into an hash table
# that is rehashing, the more rehashing "steps" are performed, so if the
# server is idle the rehashing is never complete and some more memory is used
# by the hash table.
@@ -711,7 +501,7 @@ activerehashing yes
#
# normal -> normal clients
# slave -> slave clients and MONITOR clients
# pubsub -> clients subscribed to at least one pubsub channel or pattern
# pubsub -> clients subcribed to at least one pubsub channel or pattern
#
# The syntax of every client-output-buffer-limit directive is the following:
#
@@ -734,35 +524,17 @@ activerehashing yes
# Instead there is a default limit for pubsub and slave clients, since
# subscribers and slaves receive data in a push fashion.
#
# Both the hard or the soft limit can be disabled by setting them to zero.
# Both the hard or the soft limit can be disabled just setting it to zero.
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
# Redis calls an internal function to perform many background tasks, like
# closing connections of clients in timeout, purging expired keys that are
# never requested, and so forth.
#
# Not all tasks are performed with the same frequency, but Redis checks for
# tasks to perform accordingly to the specified "hz" value.
#
# By default "hz" is set to 10. Raising the value will use more CPU when
# Redis is idle, but at the same time will make Redis more responsive when
# there are many keys expiring at the same time, and timeouts may be
# handled with more precision.
#
# The range is between 1 and 500, however a value over 100 is usually not
# a good idea. Most users should use the default of 10 and raise this up to
# 100 only in environments where very low latency is required.
hz 10
################################## INCLUDES ###################################
# When a child rewrites the AOF file, if the following option is enabled
# the file will be fsync-ed every 32 MB of data generated. This is useful
# in order to commit the file to the disk more incrementally and avoid
# big latency spikes.
aof-rewrite-incremental-fsync yes
# Transparently compress string objects in memory using LZF.
# For default this is set to no since it may affect performances, however
# when caching things like HTML fragments this may result into a big win.
memcompression no
# Include one or more other config files here. This is useful if you
# have a standard template that goes to all Redis server but also need
# to customize a few per-server settings. Include files can include
# other files, so use this wisely.
#
# include /path/to/local.conf
# include /path/to/other.conf
+5 -10
View File
@@ -1,14 +1,9 @@
#!/bin/sh
TCL_VERSIONS="8.5 8.6"
TCLSH=""
for VERSION in $TCL_VERSIONS; do
TCL=`which tclsh$VERSION 2>/dev/null` && TCLSH=$TCL
done
if [ -z $TCLSH ]
TCL=tclsh8.5
which $TCL
if [ "$?" != "0" ]
then
echo "You need tcl 8.5 or newer in order to run the Redis test"
echo "You need '$TCL' in order to run the Redis test"
exit 1
fi
$TCLSH tests/test_helper.tcl $*
$TCL tests/test_helper.tcl $*
-14
View File
@@ -1,14 +0,0 @@
#!/bin/sh
TCL_VERSIONS="8.5 8.6"
TCLSH=""
for VERSION in $TCL_VERSIONS; do
TCL=`which tclsh$VERSION 2>/dev/null` && TCLSH=$TCL
done
if [ -z $TCLSH ]
then
echo "You need tcl 8.5 or newer in order to run the Redis Sentinel test"
exit 1
fi
$TCLSH tests/sentinel.tcl $*
+36 -32
View File
@@ -6,13 +6,9 @@ port 26379
# sentinel monitor <master-name> <ip> <redis-port> <quorum>
#
# Tells Sentinel to monitor this master, and to consider it in O_DOWN
# Tells Sentinel to monitor this slave, and to consider it in O_DOWN
# (Objectively Down) state only if at least <quorum> sentinels agree.
#
# Note that whatever is the ODOWN quorum, a Sentinel will require to
# be elected by the majority of the known Sentinels in order to
# start a failover, so no failover can be performed in minority.
#
# Note: master name should not include special characters or spaces.
# The valid charset is A-z 0-9 and the three characters ".-_".
sentinel monitor mymaster 127.0.0.1 6379 2
@@ -46,6 +42,11 @@ sentinel monitor mymaster 127.0.0.1 6379 2
# Default is 30 seconds.
sentinel down-after-milliseconds mymaster 30000
# sentinel can-failover <master-name> <yes|no>
#
# Specify if this Sentinel can start the failover for this master.
sentinel can-failover mymaster yes
# sentinel parallel-syncs <master-name> <numslaves>
#
# How many slaves we can reconfigure to point to the new slave simultaneously
@@ -56,40 +57,31 @@ sentinel parallel-syncs mymaster 1
# sentinel failover-timeout <master-name> <milliseconds>
#
# Specifies the failover timeout in milliseconds. It is used in many ways:
# Specifies the failover timeout in milliseconds. When this time has elapsed
# without any progress in the failover process, it is considered concluded by
# the sentinel even if not all the attached slaves were correctly configured
# to replicate with the new master (however a "best effort" SLAVEOF command
# is sent to all the slaves before).
#
# - The time needed to re-start a failover after a previous failover was
# already tried against the same master by a given Sentinel, is two
# times the failover timeout.
# Also when 25% of this time has elapsed without any advancement, and there
# is a leader switch (the sentinel did not started the failover but is now
# elected as leader), the sentinel will continue the failover doing a
# "takeover".
#
# - The time needed for a slave replicating to a wrong master according
# to a Sentinel current configuration, to be forced to replicate
# with the right master, is exactly the failover timeout (counting since
# the moment a Sentinel detected the misconfiguration).
#
# - The time needed to cancel a failover that is already in progress but
# did not produced any configuration change (SLAVEOF NO ONE yet not
# acknowledged by the promoted slave).
#
# - The maximum time a failover in progress waits for all the slaves to be
# reconfigured as slaves of the new master. However even after this time
# the slaves will be reconfigured by the Sentinels anyway, but not with
# the exact parallel-syncs progression as specified.
#
# Default is 3 minutes.
sentinel failover-timeout mymaster 180000
# Default is 15 minutes.
sentinel failover-timeout mymaster 900000
# SCRIPTS EXECUTION
# SCRIPTS EXECTION
#
# sentinel notification-script and sentinel reconfig-script are used in order
# to configure scripts that are called to notify the system administrator
# or to reconfigure clients after a failover. The scripts are executed
# with the following rules for error handling:
#
# If script exits with "1" the execution is retried later (up to a maximum
# If script exists with "1" the execution is retried later (up to a maximum
# number of times currently set to 10).
#
# If script exits with "2" (or an higher value) the script execution is
# If script exists with "2" (or an higher value) the script execution is
# not retried.
#
# If script terminates because it receives a signal the behavior is the same
@@ -102,7 +94,7 @@ sentinel failover-timeout mymaster 180000
#
# sentinel notification-script <master-name> <script-path>
#
# Call the specified notification script for any sentinel event that is
# Call the specified notification script for any sentienl event that is
# generated in the WARNING level (for instance -sdown, -odown, and so forth).
# This script should notify the system administrator via email, SMS, or any
# other messaging system, that there is something wrong with the monitored
@@ -122,20 +114,32 @@ sentinel failover-timeout mymaster 180000
#
# sentinel client-reconfig-script <master-name> <script-path>
#
# When the master changed because of a failover a script can be called in
# When the failover starts, ends, or is aborted, a script can be called in
# order to perform application-specific tasks to notify the clients that the
# configuration has changed and the master is at a different address.
#
# The script is called in the following cases:
#
# Failover started (a slave is already promoted)
# Failover finished (all the additional slaves already reconfigured)
# Failover aborted (in that case the script was previously called when the
# failover started, and now gets called again with swapped
# addresses).
#
# The following arguments are passed to the script:
#
# <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
#
# <state> is currently always "failover"
# <state> is "start", "end" or "abort"
# <role> is either "leader" or "observer"
#
# The arguments from-ip, from-port, to-ip, to-port are used to communicate
# the old address of the master and the new address of the elected slave
# (now a master).
# (now a master) in the case state is "start" or "end".
#
# For abort instead the "from" is the address of the promoted slave and
# "to" is the address of the original master address, since the failover
# was aborted.
#
# This script should be resistant to multiple invocations.
#
+40 -56
View File
@@ -18,80 +18,72 @@ OPTIMIZATION?=-O2
DEPENDENCY_TARGETS=hiredis linenoise lua
# Default settings
STD=-std=c99 -pedantic
WARN=-Wall
OPT=$(OPTIMIZATION)
PREFIX?=/usr/local
INSTALL_BIN=$(PREFIX)/bin
INSTALL=install
STD= -std=c99 -pedantic
WARN= -Wall
OPT= $(OPTIMIZATION)
# Default allocator
ifeq ($(uname_S),Linux)
MALLOC=jemalloc
MALLOC=jemalloc
else
MALLOC=libc
MALLOC=libc
endif
# Backwards compatibility for selecting an allocator
ifeq ($(USE_TCMALLOC),yes)
MALLOC=tcmalloc
MALLOC=tcmalloc
endif
ifeq ($(USE_TCMALLOC_MINIMAL),yes)
MALLOC=tcmalloc_minimal
MALLOC=tcmalloc_minimal
endif
ifeq ($(USE_JEMALLOC),yes)
MALLOC=jemalloc
MALLOC=jemalloc
endif
# Override default settings if possible
-include .make-settings
FINAL_CFLAGS=$(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS)
FINAL_LDFLAGS=$(LDFLAGS) $(REDIS_LDFLAGS) $(DEBUG)
FINAL_LIBS=-lm
DEBUG=-g -ggdb
ifeq ($(uname_S),SunOS)
# SunOS
INSTALL=cp -pf
FINAL_CFLAGS+= -D__EXTENSIONS__ -D_XPG6
FINAL_LIBS+= -ldl -lnsl -lsocket -lpthread
FINAL_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS) -D__EXTENSIONS__ -D_XPG6
FINAL_LDFLAGS= $(LDFLAGS) $(REDIS_LDFLAGS) -g -ggdb
FINAL_LIBS= -ldl -lnsl -lsocket -lm -lpthread
DEBUG= -g -ggdb
else
ifeq ($(uname_S),Darwin)
# Darwin (nothing to do)
else
# All the other OSes (notably Linux)
FINAL_LDFLAGS+= -rdynamic
FINAL_LIBS+= -pthread
endif
FINAL_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS)
FINAL_LDFLAGS= $(LDFLAGS) $(REDIS_LDFLAGS) -g -rdynamic -ggdb
FINAL_LIBS= -lm -pthread
DEBUG= -g -rdynamic -ggdb
endif
# Include paths to dependencies
FINAL_CFLAGS+= -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src
ifeq ($(MALLOC),tcmalloc)
FINAL_CFLAGS+= -DUSE_TCMALLOC
FINAL_LIBS+= -ltcmalloc
FINAL_CFLAGS+= -DUSE_TCMALLOC
FINAL_LIBS+= -ltcmalloc
endif
ifeq ($(MALLOC),tcmalloc_minimal)
FINAL_CFLAGS+= -DUSE_TCMALLOC
FINAL_LIBS+= -ltcmalloc_minimal
FINAL_CFLAGS+= -DUSE_TCMALLOC
FINAL_LIBS+= -ltcmalloc_minimal
endif
ifeq ($(MALLOC),jemalloc)
DEPENDENCY_TARGETS+= jemalloc
FINAL_CFLAGS+= -DUSE_JEMALLOC -I../deps/jemalloc/include
FINAL_LIBS+= ../deps/jemalloc/lib/libjemalloc.a -ldl
DEPENDENCY_TARGETS+= jemalloc
FINAL_CFLAGS+= -DUSE_JEMALLOC -I../deps/jemalloc/include
FINAL_LIBS+= ../deps/jemalloc/lib/libjemalloc.a -ldl
endif
REDIS_CC=$(QUIET_CC)$(CC) $(FINAL_CFLAGS)
REDIS_LD=$(QUIET_LINK)$(CC) $(FINAL_LDFLAGS)
REDIS_INSTALL=$(QUIET_INSTALL)$(INSTALL)
PREFIX?=/usr/local
INSTALL_BIN= $(PREFIX)/bin
INSTALL= cp -pf
CCCOLOR="\033[34m"
LINKCOLOR="\033[34;1m"
SRCCOLOR="\033[33m"
@@ -105,17 +97,17 @@ QUIET_LINK = @printf ' %b %b\n' $(LINKCOLOR)LINK$(ENDCOLOR) $(BINCOLOR)$@$(EN
QUIET_INSTALL = @printf ' %b %b\n' $(LINKCOLOR)INSTALL$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR) 1>&2;
endif
REDIS_SERVER_NAME=redis-server
REDIS_SENTINEL_NAME=redis-sentinel
REDIS_SERVER_OBJ=adlist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o
REDIS_CLI_NAME=redis-cli
REDIS_CLI_OBJ=anet.o sds.o adlist.o redis-cli.o zmalloc.o release.o anet.o ae.o crc64.o
REDIS_BENCHMARK_NAME=redis-benchmark
REDIS_BENCHMARK_OBJ=ae.o anet.o redis-benchmark.o sds.o adlist.o zmalloc.o redis-benchmark.o
REDIS_CHECK_DUMP_NAME=redis-check-dump
REDIS_CHECK_DUMP_OBJ=redis-check-dump.o lzf_c.o lzf_d.o crc64.o
REDIS_CHECK_AOF_NAME=redis-check-aof
REDIS_CHECK_AOF_OBJ=redis-check-aof.o
REDIS_SERVER_NAME= redis-server
REDIS_SENTINEL_NAME= redis-sentinel
REDIS_SERVER_OBJ= adlist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o migrate.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o
REDIS_CLI_NAME= redis-cli
REDIS_CLI_OBJ= anet.o sds.o adlist.o redis-cli.o zmalloc.o release.o anet.o ae.o
REDIS_BENCHMARK_NAME= redis-benchmark
REDIS_BENCHMARK_OBJ= ae.o anet.o redis-benchmark.o sds.o adlist.o zmalloc.o redis-benchmark.o
REDIS_CHECK_DUMP_NAME= redis-check-dump
REDIS_CHECK_DUMP_OBJ= redis-check-dump.o lzf_c.o lzf_d.o crc64.o
REDIS_CHECK_AOF_NAME= redis-check-aof
REDIS_CHECK_AOF_OBJ= redis-check-aof.o
all: $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_DUMP_NAME) $(REDIS_CHECK_AOF_NAME)
@echo ""
@@ -204,11 +196,6 @@ distclean: clean
test: $(REDIS_SERVER_NAME) $(REDIS_CHECK_AOF_NAME)
@(cd ..; ./runtest)
test-sentinel: $(REDIS_SENTINEL_NAME)
@(cd ..; ./runtest-sentinel)
check: test
lcov:
$(MAKE) gcov
@(set -e; cd ..; ./runtest --clients 1)
@@ -230,16 +217,13 @@ gcov:
$(MAKE) REDIS_CFLAGS="-fprofile-arcs -ftest-coverage -DCOVERAGE_TEST" REDIS_LDFLAGS="-fprofile-arcs -ftest-coverage"
noopt:
$(MAKE) OPTIMIZATION="-O0"
valgrind:
$(MAKE) OPTIMIZATION="-O0" MALLOC="libc"
$(MAKE) OPT="-O0"
src/help.h:
@../utils/generate-command-help.rb > help.h
install: all
@mkdir -p $(INSTALL_BIN)
mkdir -p $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_BENCHMARK_NAME) $(INSTALL_BIN)
$(REDIS_INSTALL) $(REDIS_CLI_NAME) $(INSTALL_BIN)
+63 -88
View File
@@ -1,126 +1,101 @@
adlist.o: adlist.c adlist.h zmalloc.h
ae.o: ae.c ae.h zmalloc.h config.h ae_kqueue.c ae_epoll.c ae_select.c ae_evport.c
ae.o: ae.c ae.h zmalloc.h config.h ae_kqueue.c
ae_epoll.o: ae_epoll.c
ae_evport.o: ae_evport.c
ae_kqueue.o: ae_kqueue.c
ae_select.o: ae_select.c
anet.o: anet.c fmacros.h anet.h
aof.o: aof.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h bio.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h bio.h
bio.o: bio.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h bio.h
bitops.o: bitops.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
blocked.o: blocked.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
cluster.o: cluster.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h cluster.h endianconv.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h bio.h
config.o: config.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h cluster.h
crc16.o: crc16.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
crc64.o: crc64.c
db.o: db.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h cluster.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
debug.o: debug.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h sha1.h crc64.h bio.h
dict.o: dict.c fmacros.h dict.h zmalloc.h redisassert.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h sha1.h
dict.o: dict.c fmacros.h dict.h zmalloc.h
endianconv.o: endianconv.c
hyperloglog.o: hyperloglog.c redis.h fmacros.h config.h \
../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h rdb.h \
rio.h
intset.o: intset.c intset.h zmalloc.h endianconv.h config.h
intset.o: intset.c intset.h zmalloc.h endianconv.h
lzf_c.o: lzf_c.c lzfP.h
lzf_d.o: lzf_d.c lzfP.h
memtest.o: memtest.c config.h
memtest.o: memtest.c
migrate.o: migrate.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h endianconv.h
multi.o: multi.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
networking.o: networking.c redis.h fmacros.h config.h \
../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h rdb.h \
rio.h
notify.o: notify.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h rdb.h \
rio.h
object.o: object.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
pqsort.o: pqsort.c
pubsub.o: pubsub.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
rand.o: rand.c
rdb.o: rdb.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h lzf.h zipmap.h \
endianconv.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h lzf.h zipmap.h \
endianconv.h
redis-benchmark.o: redis-benchmark.c fmacros.h ae.h \
../deps/hiredis/hiredis.h sds.h adlist.h zmalloc.h
../deps/hiredis/hiredis.h sds.h adlist.h zmalloc.h
redis-check-aof.o: redis-check-aof.c fmacros.h config.h
redis-check-dump.o: redis-check-dump.c lzf.h crc64.h
redis-check-dump.o: redis-check-dump.c lzf.h
redis-cli.o: redis-cli.c fmacros.h version.h ../deps/hiredis/hiredis.h \
sds.h zmalloc.h ../deps/linenoise/linenoise.h help.h anet.h ae.h
sds.h zmalloc.h ../deps/linenoise/linenoise.h help.h
redis.o: redis.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h cluster.h slowlog.h \
bio.h asciilogo.h
release.o: release.c release.h version.h crc64.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h slowlog.h bio.h \
asciilogo.h
release.o: release.c release.h
replication.o: replication.c redis.h fmacros.h config.h \
../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h rdb.h \
rio.h
rio.o: rio.c fmacros.h rio.h sds.h util.h crc64.h config.h redis.h \
../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h dict.h adlist.h \
zmalloc.h anet.h ziplist.h intset.h version.h rdb.h
../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h rdb.h \
rio.h
rio.o: rio.c fmacros.h rio.h sds.h util.h
scripting.o: scripting.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h sha1.h rand.h \
../deps/lua/src/lauxlib.h ../deps/lua/src/lua.h ../deps/lua/src/lualib.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h sha1.h rand.h \
../deps/lua/src/lauxlib.h ../deps/lua/src/lua.h \
../deps/lua/src/lualib.h
sds.o: sds.c sds.h zmalloc.h
sentinel.o: sentinel.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h \
../deps/hiredis/hiredis.h ../deps/hiredis/async.h \
../deps/hiredis/hiredis.h
setproctitle.o: setproctitle.c
sha1.o: sha1.c sha1.h config.h
slowlog.o: slowlog.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h slowlog.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h slowlog.h
sort.o: sort.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h pqsort.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h pqsort.h
syncio.o: syncio.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
t_hash.o: t_hash.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
t_list.o: t_list.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
t_set.o: t_set.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
t_string.o: t_string.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
t_zset.o: t_zset.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
util.o: util.c fmacros.h util.h sds.h
ziplist.o: ziplist.c zmalloc.h util.h sds.h ziplist.h endianconv.h \
config.h redisassert.h
zipmap.o: zipmap.c zmalloc.h endianconv.h config.h
../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
ziplist.h intset.h version.h util.h rdb.h rio.h
util.o: util.c fmacros.h util.h
ziplist.o: ziplist.c zmalloc.h util.h ziplist.h endianconv.h
zipmap.o: zipmap.c zmalloc.h endianconv.h
zmalloc.o: zmalloc.c config.h zmalloc.h
+3 -3
View File
@@ -97,7 +97,7 @@ list *listAddNodeHead(list *list, void *value)
return list;
}
/* Add a new node to the list, to tail, containing the specified 'value'
/* Add a new node to the list, to tail, contaning the specified 'value'
* pointer as value.
*
* On error, NULL is returned and no operation is performed (i.e. the
@@ -308,7 +308,7 @@ listNode *listSearchKey(list *list, void *key)
/* Return the element at the specified zero-based index
* where 0 is the head, 1 is the element next to head
* and so on. Negative integers are used in order to count
* from the tail, -1 is the last element, -2 the penultimate
* from the tail, -1 is the last element, -2 the penultimante
* and so on. If the index is out of range NULL is returned. */
listNode *listIndex(list *list, long index) {
listNode *n;
@@ -330,7 +330,7 @@ void listRotate(list *list) {
if (listLength(list) <= 1) return;
/* Detach current tail */
/* Detatch current tail */
list->tail = tail->prev;
list->tail->next = NULL;
/* Move it as head */
+4 -38
View File
@@ -38,7 +38,6 @@
#include <poll.h>
#include <string.h>
#include <time.h>
#include <errno.h>
#include "ae.h"
#include "zmalloc.h"
@@ -91,36 +90,6 @@ err:
return NULL;
}
/* Return the current set size. */
int aeGetSetSize(aeEventLoop *eventLoop) {
return eventLoop->setsize;
}
/* Resize the maximum set size of the event loop.
* If the requested set size is smaller than the current set size, but
* there is already a file descriptor in use that is >= the requested
* set size minus one, AE_ERR is returned and the operation is not
* performed at all.
*
* Otherwise AE_OK is returned and the operation is successful. */
int aeResizeSetSize(aeEventLoop *eventLoop, int setsize) {
int i;
if (setsize == eventLoop->setsize) return AE_OK;
if (eventLoop->maxfd >= setsize) return AE_ERR;
if (aeApiResize(eventLoop,setsize) == -1) return AE_ERR;
eventLoop->events = zrealloc(eventLoop->events,sizeof(aeFileEvent)*setsize);
eventLoop->fired = zrealloc(eventLoop->fired,sizeof(aeFiredEvent)*setsize);
eventLoop->setsize = setsize;
/* Make sure that if we created new slots, they are initialized with
* an AE_NONE mask. */
for (i = eventLoop->maxfd+1; i < setsize; i++)
eventLoop->events[i].mask = AE_NONE;
return AE_OK;
}
void aeDeleteEventLoop(aeEventLoop *eventLoop) {
aeApiFree(eventLoop);
zfree(eventLoop->events);
@@ -135,10 +104,7 @@ void aeStop(aeEventLoop *eventLoop) {
int aeCreateFileEvent(aeEventLoop *eventLoop, int fd, int mask,
aeFileProc *proc, void *clientData)
{
if (fd >= eventLoop->setsize) {
errno = ERANGE;
return AE_ERR;
}
if (fd >= eventLoop->setsize) return AE_ERR;
aeFileEvent *fe = &eventLoop->events[fd];
if (aeApiAddEvent(eventLoop, fd, mask) == -1)
@@ -339,7 +305,7 @@ static int processTimeEvents(aeEventLoop *eventLoop) {
/* Process every pending time event, then every pending file event
* (that may be registered by time event callbacks just processed).
* Without special flags the function sleeps until some file event
* fires, or when the next time event occurs (if any).
* fires, or when the next time event occurrs (if any).
*
* If flags is 0, the function does nothing and returns.
* if flags has AE_ALL_EVENTS set, all the kind of events are processed.
@@ -386,7 +352,7 @@ int aeProcessEvents(aeEventLoop *eventLoop, int flags)
if (tvp->tv_usec < 0) tvp->tv_usec = 0;
} else {
/* If we have to check for events but need to return
* ASAP because of AE_DONT_WAIT we need to set the timeout
* ASAP because of AE_DONT_WAIT we need to se the timeout
* to zero */
if (flags & AE_DONT_WAIT) {
tv.tv_sec = tv.tv_usec = 0;
@@ -425,7 +391,7 @@ int aeProcessEvents(aeEventLoop *eventLoop, int flags)
return processed; /* return the number of processed file/time events */
}
/* Wait for milliseconds until the given file descriptor becomes
/* Wait for millseconds until the given file descriptor becomes
* writable/readable/exception */
int aeWait(int fd, int mask, long long milliseconds) {
struct pollfd pfd;
-2
View File
@@ -114,7 +114,5 @@ int aeWait(int fd, int mask, long long milliseconds);
void aeMain(aeEventLoop *eventLoop);
char *aeGetApiName(void);
void aeSetBeforeSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *beforesleep);
int aeGetSetSize(aeEventLoop *eventLoop);
int aeResizeSetSize(aeEventLoop *eventLoop, int setsize);
#endif
+1 -8
View File
@@ -45,7 +45,7 @@ static int aeApiCreate(aeEventLoop *eventLoop) {
zfree(state);
return -1;
}
state->epfd = epoll_create(1024); /* 1024 is just a hint for the kernel */
state->epfd = epoll_create(1024); /* 1024 is just an hint for the kernel */
if (state->epfd == -1) {
zfree(state->events);
zfree(state);
@@ -55,13 +55,6 @@ static int aeApiCreate(aeEventLoop *eventLoop) {
return 0;
}
static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
aeApiState *state = eventLoop->apidata;
state->events = zrealloc(state->events, sizeof(struct epoll_event)*setsize);
return 0;
}
static void aeApiFree(aeEventLoop *eventLoop) {
aeApiState *state = eventLoop->apidata;
+6 -11
View File
@@ -50,15 +50,15 @@ static int evport_debug = 0;
* aeApiPoll, the corresponding file descriptors become dissociated from the
* port. This is necessary because poll events are level-triggered, so if the
* fd didn't become dissociated, it would immediately fire another event since
* the underlying state hasn't changed yet. We must re-associate the file
* the underlying state hasn't changed yet. We must reassociate the file
* descriptor, but only after we know that our caller has actually read from it.
* The ae API does not tell us exactly when that happens, but we do know that
* it must happen by the time aeApiPoll is called again. Our solution is to
* keep track of the last fds returned by aeApiPoll and re-associate them next
* keep track of the last fds returned by aeApiPoll and reassociate them next
* time aeApiPoll is invoked.
*
* To summarize, in this module, each fd association is EITHER (a) represented
* only via the in-kernel association OR (b) represented by pending_fds and
* only via the in-kernel assocation OR (b) represented by pending_fds and
* pending_masks. (b) is only true for the last fds we returned from aeApiPoll,
* and only until we enter aeApiPoll again (at which point we restore the
* in-kernel association).
@@ -94,11 +94,6 @@ static int aeApiCreate(aeEventLoop *eventLoop) {
return 0;
}
static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
/* Nothing to resize here. */
return 0;
}
static void aeApiFree(aeEventLoop *eventLoop) {
aeApiState *state = eventLoop->apidata;
@@ -169,7 +164,7 @@ static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask) {
* This fd was recently returned from aeApiPoll. It should be safe to
* assume that the consumer has processed that poll event, but we play
* it safer by simply updating pending_mask. The fd will be
* re-associated as usual when aeApiPoll is called again.
* reassociated as usual when aeApiPoll is called again.
*/
if (evport_debug)
fprintf(stderr, "aeApiAddEvent: adding to pending fd %d\n", fd);
@@ -233,7 +228,7 @@ static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int mask) {
* ENOMEM is a potentially transient condition, but the kernel won't
* generally return it unless things are really bad. EAGAIN indicates
* we've reached an resource limit, for which it doesn't make sense to
* retry (counter-intuitively). All other errors indicate a bug. In any
* retry (counterintuitively). All other errors indicate a bug. In any
* of these cases, the best we can do is to abort.
*/
abort(); /* will not return */
@@ -248,7 +243,7 @@ static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {
port_event_t event[MAX_EVENT_BATCHSZ];
/*
* If we've returned fd events before, we must re-associate them with the
* If we've returned fd events before, we must reassociate them with the
* port now, before calling port_get(). See the block comment at the top of
* this file for an explanation of why.
*/
+1 -7
View File
@@ -54,16 +54,10 @@ static int aeApiCreate(aeEventLoop *eventLoop) {
return -1;
}
eventLoop->apidata = state;
return 0;
}
static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
aeApiState *state = eventLoop->apidata;
state->events = zrealloc(state->events, sizeof(struct kevent)*setsize);
return 0;
}
static void aeApiFree(aeEventLoop *eventLoop) {
aeApiState *state = eventLoop->apidata;
-6
View File
@@ -48,12 +48,6 @@ static int aeApiCreate(aeEventLoop *eventLoop) {
return 0;
}
static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
/* Just ensure we have enough room in the fd_set type. */
if (setsize >= FD_SETSIZE) return -1;
return 0;
}
static void aeApiFree(aeEventLoop *eventLoop) {
zfree(eventLoop->apidata);
}
+87 -269
View File
@@ -61,7 +61,7 @@ int anetNonBlock(char *err, int fd)
{
int flags;
/* Set the socket non-blocking.
/* Set the socket nonblocking.
* Note that fcntl(2) for F_GETFL and F_SETFL can't be
* interrupted by a signal. */
if ((flags = fcntl(fd, F_GETFL)) == -1) {
@@ -75,56 +75,10 @@ int anetNonBlock(char *err, int fd)
return ANET_OK;
}
/* Set TCP keep alive option to detect dead peers. The interval option
* is only used for Linux as we are using Linux-specific APIs to set
* the probe send time, interval, and count. */
int anetKeepAlive(char *err, int fd, int interval)
int anetTcpNoDelay(char *err, int fd)
{
int val = 1;
if (setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &val, sizeof(val)) == -1)
{
anetSetError(err, "setsockopt SO_KEEPALIVE: %s", strerror(errno));
return ANET_ERR;
}
#ifdef __linux__
/* Default settings are more or less garbage, with the keepalive time
* set to 7200 by default on Linux. Modify settings to make the feature
* actually useful. */
/* Send first probe after interval. */
val = interval;
if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPIDLE, &val, sizeof(val)) < 0) {
anetSetError(err, "setsockopt TCP_KEEPIDLE: %s\n", strerror(errno));
return ANET_ERR;
}
/* Send next probes after the specified interval. Note that we set the
* delay as interval / 3, as we send three probes before detecting
* an error (see the next setsockopt call). */
val = interval/3;
if (val == 0) val = 1;
if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPINTVL, &val, sizeof(val)) < 0) {
anetSetError(err, "setsockopt TCP_KEEPINTVL: %s\n", strerror(errno));
return ANET_ERR;
}
/* Consider the socket in error state after three we send three ACK
* probes without getting a reply. */
val = 3;
if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPCNT, &val, sizeof(val)) < 0) {
anetSetError(err, "setsockopt TCP_KEEPCNT: %s\n", strerror(errno));
return ANET_ERR;
}
#endif
return ANET_OK;
}
static int anetSetTcpNoDelay(char *err, int fd, int val)
{
if (setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &val, sizeof(val)) == -1)
int yes = 1;
if (setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &yes, sizeof(yes)) == -1)
{
anetSetError(err, "setsockopt TCP_NODELAY: %s", strerror(errno));
return ANET_ERR;
@@ -132,17 +86,6 @@ static int anetSetTcpNoDelay(char *err, int fd, int val)
return ANET_OK;
}
int anetEnableTcpNoDelay(char *err, int fd)
{
return anetSetTcpNoDelay(err, fd, 1);
}
int anetDisableTcpNoDelay(char *err, int fd)
{
return anetSetTcpNoDelay(err, fd, 0);
}
int anetSetSendBuffer(char *err, int fd, int buffsize)
{
if (setsockopt(fd, SOL_SOCKET, SO_SNDBUF, &buffsize, sizeof(buffsize)) == -1)
@@ -163,70 +106,36 @@ int anetTcpKeepAlive(char *err, int fd)
return ANET_OK;
}
/* anetGenericResolve() is called by anetResolve() and anetResolveIP() to
* do the actual work. It resolves the hostname "host" and set the string
* representation of the IP address into the buffer pointed by "ipbuf".
*
* If flags is set to ANET_IP_ONLY the function only resolves hostnames
* that are actually already IPv4 or IPv6 addresses. This turns the function
* into a validating / normalizing function. */
int anetGenericResolve(char *err, char *host, char *ipbuf, size_t ipbuf_len,
int flags)
int anetResolve(char *err, char *host, char *ipbuf)
{
struct addrinfo hints, *info;
int rv;
struct sockaddr_in sa;
memset(&hints,0,sizeof(hints));
if (flags & ANET_IP_ONLY) hints.ai_flags = AI_NUMERICHOST;
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM; /* specify socktype to avoid dups */
sa.sin_family = AF_INET;
if (inet_aton(host, &sa.sin_addr) == 0) {
struct hostent *he;
if ((rv = getaddrinfo(host, NULL, &hints, &info)) != 0) {
anetSetError(err, "%s", gai_strerror(rv));
return ANET_ERR;
}
if (info->ai_family == AF_INET) {
struct sockaddr_in *sa = (struct sockaddr_in *)info->ai_addr;
inet_ntop(AF_INET, &(sa->sin_addr), ipbuf, ipbuf_len);
} else {
struct sockaddr_in6 *sa = (struct sockaddr_in6 *)info->ai_addr;
inet_ntop(AF_INET6, &(sa->sin6_addr), ipbuf, ipbuf_len);
}
freeaddrinfo(info);
return ANET_OK;
}
int anetResolve(char *err, char *host, char *ipbuf, size_t ipbuf_len) {
return anetGenericResolve(err,host,ipbuf,ipbuf_len,ANET_NONE);
}
int anetResolveIP(char *err, char *host, char *ipbuf, size_t ipbuf_len) {
return anetGenericResolve(err,host,ipbuf,ipbuf_len,ANET_IP_ONLY);
}
static int anetSetReuseAddr(char *err, int fd) {
int yes = 1;
/* Make sure connection-intensive things like the redis benckmark
* will be able to close/open sockets a zillion of times */
if (setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes)) == -1) {
anetSetError(err, "setsockopt SO_REUSEADDR: %s", strerror(errno));
return ANET_ERR;
he = gethostbyname(host);
if (he == NULL) {
anetSetError(err, "can't resolve: %s", host);
return ANET_ERR;
}
memcpy(&sa.sin_addr, he->h_addr, sizeof(struct in_addr));
}
strcpy(ipbuf,inet_ntoa(sa.sin_addr));
return ANET_OK;
}
static int anetCreateSocket(char *err, int domain) {
int s;
int s, on = 1;
if ((s = socket(domain, SOCK_STREAM, 0)) == -1) {
anetSetError(err, "creating socket: %s", strerror(errno));
return ANET_ERR;
}
/* Make sure connection-intensive things like the redis benchmark
/* Make sure connection-intensive things like the redis benckmark
* will be able to close/open sockets a zillion of times */
if (anetSetReuseAddr(err,s) == ANET_ERR) {
close(s);
if (setsockopt(s, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) == -1) {
anetSetError(err, "setsockopt SO_REUSEADDR: %s", strerror(errno));
return ANET_ERR;
}
return s;
@@ -234,89 +143,51 @@ static int anetCreateSocket(char *err, int domain) {
#define ANET_CONNECT_NONE 0
#define ANET_CONNECT_NONBLOCK 1
static int anetTcpGenericConnect(char *err, char *addr, int port,
char *source_addr, int flags)
static int anetTcpGenericConnect(char *err, char *addr, int port, int flags)
{
int s = ANET_ERR, rv;
char portstr[6]; /* strlen("65535") + 1; */
struct addrinfo hints, *servinfo, *bservinfo, *p, *b;
int s;
struct sockaddr_in sa;
snprintf(portstr,sizeof(portstr),"%d",port);
memset(&hints,0,sizeof(hints));
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
if ((s = anetCreateSocket(err,AF_INET)) == ANET_ERR)
return ANET_ERR;
if ((rv = getaddrinfo(addr,portstr,&hints,&servinfo)) != 0) {
anetSetError(err, "%s", gai_strerror(rv));
sa.sin_family = AF_INET;
sa.sin_port = htons(port);
if (inet_aton(addr, &sa.sin_addr) == 0) {
struct hostent *he;
he = gethostbyname(addr);
if (he == NULL) {
anetSetError(err, "can't resolve: %s", addr);
close(s);
return ANET_ERR;
}
memcpy(&sa.sin_addr, he->h_addr, sizeof(struct in_addr));
}
if (flags & ANET_CONNECT_NONBLOCK) {
if (anetNonBlock(err,s) != ANET_OK)
return ANET_ERR;
}
if (connect(s, (struct sockaddr*)&sa, sizeof(sa)) == -1) {
if (errno == EINPROGRESS &&
flags & ANET_CONNECT_NONBLOCK)
return s;
anetSetError(err, "connect: %s", strerror(errno));
close(s);
return ANET_ERR;
}
for (p = servinfo; p != NULL; p = p->ai_next) {
/* Try to create the socket and to connect it.
* If we fail in the socket() call, or on connect(), we retry with
* the next entry in servinfo. */
if ((s = socket(p->ai_family,p->ai_socktype,p->ai_protocol)) == -1)
continue;
if (anetSetReuseAddr(err,s) == ANET_ERR) goto error;
if (flags & ANET_CONNECT_NONBLOCK && anetNonBlock(err,s) != ANET_OK)
goto error;
if (source_addr) {
int bound = 0;
/* Using getaddrinfo saves us from self-determining IPv4 vs IPv6 */
if ((rv = getaddrinfo(source_addr, NULL, &hints, &bservinfo)) != 0) {
anetSetError(err, "%s", gai_strerror(rv));
goto end;
}
for (b = bservinfo; b != NULL; b = b->ai_next) {
if (bind(s,b->ai_addr,b->ai_addrlen) != -1) {
bound = 1;
break;
}
}
if (!bound) {
anetSetError(err, "bind: %s", strerror(errno));
goto end;
}
}
if (connect(s,p->ai_addr,p->ai_addrlen) == -1) {
/* If the socket is non-blocking, it is ok for connect() to
* return an EINPROGRESS error here. */
if (errno == EINPROGRESS && flags & ANET_CONNECT_NONBLOCK)
goto end;
close(s);
s = ANET_ERR;
continue;
}
/* If we ended an iteration of the for loop without errors, we
* have a connected socket. Let's return to the caller. */
goto end;
}
if (p == NULL)
anetSetError(err, "creating socket: %s", strerror(errno));
error:
if (s != ANET_ERR) {
close(s);
s = ANET_ERR;
}
end:
freeaddrinfo(servinfo);
return s;
}
int anetTcpConnect(char *err, char *addr, int port)
{
return anetTcpGenericConnect(err,addr,port,NULL,ANET_CONNECT_NONE);
return anetTcpGenericConnect(err,addr,port,ANET_CONNECT_NONE);
}
int anetTcpNonBlockConnect(char *err, char *addr, int port)
{
return anetTcpGenericConnect(err,addr,port,NULL,ANET_CONNECT_NONBLOCK);
}
int anetTcpNonBlockBindConnect(char *err, char *addr, int port, char *source_addr)
{
return anetTcpGenericConnect(err,addr,port,source_addr,ANET_CONNECT_NONBLOCK);
return anetTcpGenericConnect(err,addr,port,ANET_CONNECT_NONBLOCK);
}
int anetUnixGenericConnect(char *err, char *path, int flags)
@@ -385,14 +256,17 @@ int anetWrite(int fd, char *buf, int count)
return totlen;
}
static int anetListen(char *err, int s, struct sockaddr *sa, socklen_t len, int backlog) {
static int anetListen(char *err, int s, struct sockaddr *sa, socklen_t len) {
if (bind(s,sa,len) == -1) {
anetSetError(err, "bind: %s", strerror(errno));
close(s);
return ANET_ERR;
}
if (listen(s, backlog) == -1) {
/* Use a backlog of 512 entries. We pass 511 to the listen() call because
* the kernel does: backlogsize = roundup_pow_of_two(backlogsize + 1);
* which will thus give us a backlog of 512 entries */
if (listen(s, 511) == -1) {
anetSetError(err, "listen: %s", strerror(errno));
close(s);
return ANET_ERR;
@@ -400,64 +274,29 @@ static int anetListen(char *err, int s, struct sockaddr *sa, socklen_t len, int
return ANET_OK;
}
static int anetV6Only(char *err, int s) {
int yes = 1;
if (setsockopt(s,IPPROTO_IPV6,IPV6_V6ONLY,&yes,sizeof(yes)) == -1) {
anetSetError(err, "setsockopt: %s", strerror(errno));
int anetTcpServer(char *err, int port, char *bindaddr)
{
int s;
struct sockaddr_in sa;
if ((s = anetCreateSocket(err,AF_INET)) == ANET_ERR)
return ANET_ERR;
memset(&sa,0,sizeof(sa));
sa.sin_family = AF_INET;
sa.sin_port = htons(port);
sa.sin_addr.s_addr = htonl(INADDR_ANY);
if (bindaddr && inet_aton(bindaddr, &sa.sin_addr) == 0) {
anetSetError(err, "invalid bind address");
close(s);
return ANET_ERR;
}
return ANET_OK;
}
static int _anetTcpServer(char *err, int port, char *bindaddr, int af, int backlog)
{
int s, rv;
char _port[6]; /* strlen("65535") */
struct addrinfo hints, *servinfo, *p;
snprintf(_port,6,"%d",port);
memset(&hints,0,sizeof(hints));
hints.ai_family = af;
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_PASSIVE; /* No effect if bindaddr != NULL */
if ((rv = getaddrinfo(bindaddr,_port,&hints,&servinfo)) != 0) {
anetSetError(err, "%s", gai_strerror(rv));
if (anetListen(err,s,(struct sockaddr*)&sa,sizeof(sa)) == ANET_ERR)
return ANET_ERR;
}
for (p = servinfo; p != NULL; p = p->ai_next) {
if ((s = socket(p->ai_family,p->ai_socktype,p->ai_protocol)) == -1)
continue;
if (af == AF_INET6 && anetV6Only(err,s) == ANET_ERR) goto error;
if (anetSetReuseAddr(err,s) == ANET_ERR) goto error;
if (anetListen(err,s,p->ai_addr,p->ai_addrlen,backlog) == ANET_ERR) goto error;
goto end;
}
if (p == NULL) {
anetSetError(err, "unable to bind socket");
goto error;
}
error:
s = ANET_ERR;
end:
freeaddrinfo(servinfo);
return s;
}
int anetTcpServer(char *err, int port, char *bindaddr, int backlog)
{
return _anetTcpServer(err, port, bindaddr, AF_INET, backlog);
}
int anetTcp6Server(char *err, int port, char *bindaddr, int backlog)
{
return _anetTcpServer(err, port, bindaddr, AF_INET6, backlog);
}
int anetUnixServer(char *err, char *path, mode_t perm, int backlog)
int anetUnixServer(char *err, char *path, mode_t perm)
{
int s;
struct sockaddr_un sa;
@@ -468,7 +307,7 @@ int anetUnixServer(char *err, char *path, mode_t perm, int backlog)
memset(&sa,0,sizeof(sa));
sa.sun_family = AF_LOCAL;
strncpy(sa.sun_path,path,sizeof(sa.sun_path)-1);
if (anetListen(err,s,(struct sockaddr*)&sa,sizeof(sa),backlog) == ANET_ERR)
if (anetListen(err,s,(struct sockaddr*)&sa,sizeof(sa)) == ANET_ERR)
return ANET_ERR;
if (perm)
chmod(sa.sun_path, perm);
@@ -492,22 +331,15 @@ static int anetGenericAccept(char *err, int s, struct sockaddr *sa, socklen_t *l
return fd;
}
int anetTcpAccept(char *err, int s, char *ip, size_t ip_len, int *port) {
int anetTcpAccept(char *err, int s, char *ip, int *port) {
int fd;
struct sockaddr_storage sa;
struct sockaddr_in sa;
socklen_t salen = sizeof(sa);
if ((fd = anetGenericAccept(err,s,(struct sockaddr*)&sa,&salen)) == ANET_ERR)
return ANET_ERR;
if (sa.ss_family == AF_INET) {
struct sockaddr_in *s = (struct sockaddr_in *)&sa;
if (ip) inet_ntop(AF_INET,(void*)&(s->sin_addr),ip,ip_len);
if (port) *port = ntohs(s->sin_port);
} else {
struct sockaddr_in6 *s = (struct sockaddr_in6 *)&sa;
if (ip) inet_ntop(AF_INET6,(void*)&(s->sin6_addr),ip,ip_len);
if (port) *port = ntohs(s->sin6_port);
}
if (ip) strcpy(ip,inet_ntoa(sa.sin_addr));
if (port) *port = ntohs(sa.sin_port);
return fd;
}
@@ -521,46 +353,32 @@ int anetUnixAccept(char *err, int s) {
return fd;
}
int anetPeerToString(int fd, char *ip, size_t ip_len, int *port) {
struct sockaddr_storage sa;
int anetPeerToString(int fd, char *ip, int *port) {
struct sockaddr_in sa;
socklen_t salen = sizeof(sa);
if (getpeername(fd,(struct sockaddr*)&sa,&salen) == -1) {
if (port) *port = 0;
*port = 0;
ip[0] = '?';
ip[1] = '\0';
return -1;
}
if (sa.ss_family == AF_INET) {
struct sockaddr_in *s = (struct sockaddr_in *)&sa;
if (ip) inet_ntop(AF_INET,(void*)&(s->sin_addr),ip,ip_len);
if (port) *port = ntohs(s->sin_port);
} else {
struct sockaddr_in6 *s = (struct sockaddr_in6 *)&sa;
if (ip) inet_ntop(AF_INET6,(void*)&(s->sin6_addr),ip,ip_len);
if (port) *port = ntohs(s->sin6_port);
}
if (ip) strcpy(ip,inet_ntoa(sa.sin_addr));
if (port) *port = ntohs(sa.sin_port);
return 0;
}
int anetSockName(int fd, char *ip, size_t ip_len, int *port) {
struct sockaddr_storage sa;
int anetSockName(int fd, char *ip, int *port) {
struct sockaddr_in sa;
socklen_t salen = sizeof(sa);
if (getsockname(fd,(struct sockaddr*)&sa,&salen) == -1) {
if (port) *port = 0;
*port = 0;
ip[0] = '?';
ip[1] = '\0';
return -1;
}
if (sa.ss_family == AF_INET) {
struct sockaddr_in *s = (struct sockaddr_in *)&sa;
if (ip) inet_ntop(AF_INET,(void*)&(s->sin_addr),ip,ip_len);
if (port) *port = ntohs(s->sin_port);
} else {
struct sockaddr_in6 *s = (struct sockaddr_in6 *)&sa;
if (ip) inet_ntop(AF_INET6,(void*)&(s->sin6_addr),ip,ip_len);
if (port) *port = ntohs(s->sin6_port);
}
if (ip) strcpy(ip,inet_ntoa(sa.sin_addr));
if (port) *port = ntohs(sa.sin_port);
return 0;
}
+6 -16
View File
@@ -35,34 +35,24 @@
#define ANET_ERR -1
#define ANET_ERR_LEN 256
/* Flags used with certain functions. */
#define ANET_NONE 0
#define ANET_IP_ONLY (1<<0)
#if defined(__sun)
#define AF_LOCAL AF_UNIX
#endif
int anetTcpConnect(char *err, char *addr, int port);
int anetTcpNonBlockConnect(char *err, char *addr, int port);
int anetTcpNonBlockBindConnect(char *err, char *addr, int port, char *source_addr);
int anetUnixConnect(char *err, char *path);
int anetUnixNonBlockConnect(char *err, char *path);
int anetRead(int fd, char *buf, int count);
int anetResolve(char *err, char *host, char *ipbuf, size_t ipbuf_len);
int anetResolveIP(char *err, char *host, char *ipbuf, size_t ipbuf_len);
int anetTcpServer(char *err, int port, char *bindaddr, int backlog);
int anetTcp6Server(char *err, int port, char *bindaddr, int backlog);
int anetUnixServer(char *err, char *path, mode_t perm, int backlog);
int anetTcpAccept(char *err, int serversock, char *ip, size_t ip_len, int *port);
int anetResolve(char *err, char *host, char *ipbuf);
int anetTcpServer(char *err, int port, char *bindaddr);
int anetUnixServer(char *err, char *path, mode_t perm);
int anetTcpAccept(char *err, int serversock, char *ip, int *port);
int anetUnixAccept(char *err, int serversock);
int anetWrite(int fd, char *buf, int count);
int anetNonBlock(char *err, int fd);
int anetEnableTcpNoDelay(char *err, int fd);
int anetDisableTcpNoDelay(char *err, int fd);
int anetTcpNoDelay(char *err, int fd);
int anetTcpKeepAlive(char *err, int fd);
int anetPeerToString(int fd, char *ip, size_t ip_len, int *port);
int anetKeepAlive(char *err, int fd, int interval);
int anetSockName(int fd, char *ip, size_t ip_len, int *port);
int anetPeerToString(int fd, char *ip, int *port);
#endif
+30 -86
View File
@@ -126,7 +126,7 @@ void aofRewriteBufferAppend(unsigned char *s, unsigned long len) {
}
/* Write the buffer (possibly composed of multiple blocks) into the specified
* fd. If a short write or any other error happens -1 is returned,
* fd. If no short write or any other error happens -1 is returned,
* otherwise the number of bytes written is returned. */
ssize_t aofRewriteBufferWrite(int fd) {
listNode *ln;
@@ -177,7 +177,7 @@ void stopAppendOnly(void) {
redisLog(REDIS_NOTICE,"Killing running AOF rewrite child: %ld",
(long) server.aof_child_pid);
if (kill(server.aof_child_pid,SIGUSR1) != -1)
if (kill(server.aof_child_pid,SIGKILL) != -1)
wait3(&statloc,0,NULL);
/* reset the buffer accumulating changes while the child saves */
aofRewriteBufferReset();
@@ -226,7 +226,6 @@ int startAppendOnly(void) {
*
* However if force is set to 1 we'll write regardless of the background
* fsync. */
#define AOF_WRITE_LOG_ERROR_RATE 30 /* Seconds between errors logging. */
void flushAppendOnlyFile(int force) {
ssize_t nwritten;
int sync_in_progress = 0;
@@ -268,76 +267,27 @@ void flushAppendOnlyFile(int force) {
* or alike */
nwritten = write(server.aof_fd,server.aof_buf,sdslen(server.aof_buf));
if (nwritten != (signed)sdslen(server.aof_buf)) {
static time_t last_write_error_log = 0;
int can_log = 0;
/* Limit logging rate to 1 line per AOF_WRITE_LOG_ERROR_RATE seconds. */
if ((server.unixtime - last_write_error_log) > AOF_WRITE_LOG_ERROR_RATE) {
can_log = 1;
last_write_error_log = server.unixtime;
}
/* Lof the AOF write error and record the error code. */
/* Ooops, we are in troubles. The best thing to do for now is
* aborting instead of giving the illusion that everything is
* working as expected. */
if (nwritten == -1) {
if (can_log) {
redisLog(REDIS_WARNING,"Error writing to the AOF file: %s",
strerror(errno));
server.aof_last_write_errno = errno;
}
redisLog(REDIS_WARNING,"Exiting on error writing to the append-only file: %s",strerror(errno));
} else {
if (can_log) {
redisLog(REDIS_WARNING,"Short write while writing to "
"the AOF file: (nwritten=%lld, "
"expected=%lld)",
(long long)nwritten,
(long long)sdslen(server.aof_buf));
}
redisLog(REDIS_WARNING,"Exiting on short write while writing to "
"the append-only file: %s (nwritten=%ld, "
"expected=%ld)",
strerror(errno),
(long)nwritten,
(long)sdslen(server.aof_buf));
if (ftruncate(server.aof_fd, server.aof_current_size) == -1) {
if (can_log) {
redisLog(REDIS_WARNING, "Could not remove short write "
"from the append-only file. Redis may refuse "
"to load the AOF the next time it starts. "
"ftruncate: %s", strerror(errno));
}
} else {
/* If the ftrunacate() succeeded we can set nwritten to
* -1 since there is no longer partial data into the AOF. */
nwritten = -1;
redisLog(REDIS_WARNING, "Could not remove short write "
"from the append-only file. Redis may refuse "
"to load the AOF the next time it starts. "
"ftruncate: %s", strerror(errno));
}
server.aof_last_write_errno = ENOSPC;
}
/* Handle the AOF write error. */
if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
/* We can't recover when the fsync policy is ALWAYS since the
* reply for the client is already in the output buffers, and we
* have the contract with the user that on acknowledged write data
* is synched on disk. */
redisLog(REDIS_WARNING,"Can't recover from AOF write error when the AOF fsync policy is 'always'. Exiting...");
exit(1);
} else {
/* Recover from failed write leaving data into the buffer. However
* set an error to stop accepting writes as long as the error
* condition is not cleared. */
server.aof_last_write_status = REDIS_ERR;
/* Trim the sds buffer if there was a partial write, and there
* was no way to undo it with ftruncate(2). */
if (nwritten > 0) {
server.aof_current_size += nwritten;
sdsrange(server.aof_buf,nwritten,-1);
}
return; /* We'll try again on the next call... */
}
} else {
/* Successful write(2). If AOF was in error state, restore the
* OK state and log the event. */
if (server.aof_last_write_status == REDIS_ERR) {
redisLog(REDIS_WARNING,
"AOF write error looks solved, Redis can write again.");
server.aof_last_write_status = REDIS_OK;
}
exit(1);
}
server.aof_current_size += nwritten;
@@ -435,7 +385,7 @@ void feedAppendOnlyFile(struct redisCommand *cmd, int dictid, robj **argv, int a
sds buf = sdsempty();
robj *tmpargv[3];
/* The DB this command was targeting is not the same as the last command
/* The DB this command was targetting is not the same as the last command
* we appendend. To issue a SELECT command is needed. */
if (dictid != server.aof_selected_db) {
char seldb[64];
@@ -492,14 +442,12 @@ struct redisClient *createFakeClient(void) {
selectDb(c,0);
c->fd = -1;
c->name = NULL;
c->querybuf = sdsempty();
c->querybuf_peak = 0;
c->argc = 0;
c->argv = NULL;
c->bufpos = 0;
c->flags = 0;
c->btype = REDIS_BLOCKED_NONE;
/* We set the fake client as a slave waiting for the synchronization
* so that Redis will not try to send replies to this client. */
c->replstate = REDIS_REPL_WAIT_BGSAVE_START;
@@ -507,7 +455,7 @@ struct redisClient *createFakeClient(void) {
c->reply_bytes = 0;
c->obuf_soft_limit_reached_time = 0;
c->watched_keys = listCreate();
listSetFreeMethod(c->reply,decrRefCountVoid);
listSetFreeMethod(c->reply,decrRefCount);
listSetDupMethod(c->reply,dupClientReplyValue);
initClientMultiState(c);
return c;
@@ -587,7 +535,7 @@ int loadAppendOnlyFile(char *filename) {
/* Command lookup */
cmd = lookupCommand(argv[0]->ptr);
if (!cmd) {
redisLog(REDIS_WARNING,"Unknown command '%s' reading the append only file", (char*)argv[0]->ptr);
redisLog(REDIS_WARNING,"Unknown command '%s' reading the append only file", argv[0]->ptr);
exit(1);
}
/* Run the command in the context of a fake client */
@@ -642,7 +590,7 @@ int rioWriteBulkObject(rio *r, robj *obj) {
* in a child process when this function is called). */
if (obj->encoding == REDIS_ENCODING_INT) {
return rioWriteBulkLongLong(r,(long)obj->ptr);
} else if (sdsEncodedObject(obj)) {
} else if (obj->encoding == REDIS_ENCODING_RAW) {
return rioWriteBulkString(r,obj->ptr,sdslen(obj->ptr));
} else {
redisPanic("Unknown string encoding");
@@ -822,7 +770,7 @@ int rewriteSortedSetObject(rio *r, robj *key, robj *o) {
return 1;
}
/* Write either the key or the value of the currently selected item of a hash.
/* Write either the key or the value of the currently selected item of an hash.
* The 'hi' argument passes a valid Redis hash iterator.
* The 'what' filed specifies if to write a key or a value and can be
* either REDIS_HASH_KEY or REDIS_HASH_VALUE.
@@ -906,8 +854,6 @@ int rewriteAppendOnlyFile(char *filename) {
}
rioInitWithFile(&aof,fp);
if (server.aof_rewrite_incremental_fsync)
rioSetAutoSync(&aof,REDIS_AOF_AUTOSYNC_BYTES);
for (j = 0; j < server.dbnum; j++) {
char selectcmd[] = "*2\r\n$6\r\nSELECT\r\n";
redisDb *db = server.db+j;
@@ -935,9 +881,6 @@ int rewriteAppendOnlyFile(char *filename) {
expiretime = getExpire(db,&key);
/* If this key is already expired skip it */
if (expiretime != -1 && expiretime < now) continue;
/* Save the key and associated value */
if (o->type == REDIS_STRING) {
/* Emit a SET command */
@@ -960,6 +903,8 @@ int rewriteAppendOnlyFile(char *filename) {
/* Save the expire time */
if (expiretime != -1) {
char cmd[]="*3\r\n$9\r\nPEXPIREAT\r\n";
/* If this key is already expired skip it */
if (expiretime < now) continue;
if (rioWrite(&aof,cmd,sizeof(cmd)-1) == 0) goto werr;
if (rioWriteBulkObject(&aof,&key) == 0) goto werr;
if (rioWriteBulkLongLong(&aof,expiretime) == 0) goto werr;
@@ -969,9 +914,9 @@ int rewriteAppendOnlyFile(char *filename) {
}
/* Make sure data will not remain on the OS's output buffers */
if (fflush(fp) == EOF) goto werr;
if (aof_fsync(fileno(fp)) == -1) goto werr;
if (fclose(fp) == EOF) goto werr;
fflush(fp);
aof_fsync(fileno(fp));
fclose(fp);
/* Use RENAME to make sure the DB file is changed atomically only
* if the generate DB file is ok. */
@@ -1013,15 +958,15 @@ int rewriteAppendOnlyFileBackground(void) {
char tmpfile[256];
/* Child */
closeListeningSockets(0);
redisSetProcTitle("redis-aof-rewrite");
if (server.ipfd > 0) close(server.ipfd);
if (server.sofd > 0) close(server.sofd);
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) getpid());
if (rewriteAppendOnlyFile(tmpfile) == REDIS_OK) {
size_t private_dirty = zmalloc_get_private_dirty();
if (private_dirty) {
redisLog(REDIS_NOTICE,
"AOF rewrite: %zu MB of memory used by copy-on-write",
"AOF rewrite: %lu MB of memory used by copy-on-write",
private_dirty/(1024*1024));
}
exitFromChild(0);
@@ -1048,7 +993,6 @@ int rewriteAppendOnlyFileBackground(void) {
* accumulated by the parent into server.aof_rewrite_buf will start
* with a SELECT statement and it will be safe to merge. */
server.aof_selected_db = -1;
replicationScriptCacheFlush();
return REDIS_OK;
}
return REDIS_OK; /* unreached */
+51 -24
View File
@@ -61,7 +61,6 @@
#include "redis.h"
#include "bio.h"
static pthread_t bio_threads[REDIS_BIO_NUM_OPS];
static pthread_mutex_t bio_mutex[REDIS_BIO_NUM_OPS];
static pthread_cond_t bio_condvar[REDIS_BIO_NUM_OPS];
static list *bio_jobs[REDIS_BIO_NUM_OPS];
@@ -74,7 +73,7 @@ static list *bio_jobs[REDIS_BIO_NUM_OPS];
static unsigned long long bio_pending[REDIS_BIO_NUM_OPS];
/* This structure represents a background Job. It is only used locally to this
* file as the API does not expose the internals at all. */
* file as the API deos not expose the internals at all. */
struct bio_job {
time_t time; /* Time at which the job was created. */
/* Job specific arguments pointers. If we need to pass more than three
@@ -119,7 +118,6 @@ void bioInit(void) {
redisLog(REDIS_WARNING,"Fatal: Can't initialize Background Jobs.");
exit(1);
}
bio_threads[j] = thread;
}
}
@@ -142,11 +140,7 @@ void *bioProcessBackgroundJobs(void *arg) {
unsigned long type = (unsigned long) arg;
sigset_t sigset;
/* Make the thread killable at any time, so that bioKillThreads()
* can work reliably. */
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
pthread_detach(pthread_self());
pthread_mutex_lock(&bio_mutex[type]);
/* Block SIGALRM so we are sure that only the main thread will
* receive the watchdog signal. */
@@ -198,23 +192,56 @@ unsigned long long bioPendingJobsOfType(int type) {
return val;
}
/* Kill the running bio threads in an unclean way. This function should be
* used only when it's critical to stop the threads for some reason.
* Currently Redis does this only on crash (for instance on SIGSEGV) in order
* to perform a fast memory check without other threads messing with memory. */
void bioKillThreads(void) {
int err, j;
#if 0 /* We don't use the following code for now, and bioWaitPendingJobsLE
probably needs a rewrite using conditional variables instead of the
current implementation. */
for (j = 0; j < REDIS_BIO_NUM_OPS; j++) {
if (pthread_cancel(bio_threads[j]) == 0) {
if ((err = pthread_join(bio_threads[j],NULL)) != 0) {
redisLog(REDIS_WARNING,
"Bio thread for job type #%d can be joined: %s",
j, strerror(err));
} else {
redisLog(REDIS_WARNING,
"Bio thread for job type #%d terminated",j);
}
/* Wait until the number of pending jobs of the specified type are
* less or equal to the specified number.
*
* This function may block for long time, it should only be used to perform
* the following tasks:
*
* 1) To avoid that the main thread is pushing jobs of a given time so fast
* that the background thread can't process them at the same speed.
* So before creating a new job of a given type the main thread should
* call something like: bioWaitPendingJobsLE(job_type,10000);
* 2) In order to perform special operations that make it necessary to be sure
* no one is touching shared resourced in the background.
*/
void bioWaitPendingJobsLE(int type, unsigned long long num) {
unsigned long long iteration = 0;
/* We poll the jobs queue aggressively to start, and gradually relax
* the polling speed if it is going to take too much time. */
while(1) {
iteration++;
if (iteration > 1000 && iteration <= 10000) {
usleep(100);
} else if (iteration > 10000) {
usleep(1000);
}
if (bioPendingJobsOfType(type) <= num) break;
}
}
/* Return the older job of the specified type. */
time_t bioOlderJobOfType(int type) {
time_t time;
listNode *ln;
struct bio_job *job;
pthread_mutex_lock(&bio_mutex[type]);
ln = listFirst(bio_jobs[type]);
if (ln == NULL) {
pthread_mutex_unlock(&bio_mutex[type]);
return 0;
}
job = ln->value;
time = job->time;
pthread_mutex_unlock(&bio_mutex[type]);
return time;
}
#endif
-1
View File
@@ -33,7 +33,6 @@ void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3);
unsigned long long bioPendingJobsOfType(int type);
void bioWaitPendingJobsLE(int type, unsigned long long num);
time_t bioOlderJobOfType(int type);
void bioKillThreads(void);
/* Background job opcodes */
#define REDIS_BIO_CLOSE_FILE 0 /* Deferred close(2) syscall. */
+20 -200
View File
@@ -34,7 +34,7 @@
* Helpers and low level bit functions.
* -------------------------------------------------------------------------- */
/* This helper function used by GETBIT / SETBIT parses the bit offset argument
/* This helper function used by GETBIT / SETBIT parses the bit offset arguemnt
* making sure an error is returned if it is negative or if it overflows
* Redis 512 MB limit for the string value. */
static int getBitOffsetFromArgument(redisClient *c, robj *o, size_t *offset) {
@@ -58,20 +58,13 @@ static int getBitOffsetFromArgument(redisClient *c, robj *o, size_t *offset) {
/* Count number of bits set in the binary array pointed by 's' and long
* 'count' bytes. The implementation of this function is required to
* work with a input string length up to 512 MB. */
size_t redisPopcount(void *s, long count) {
size_t bits = 0;
unsigned char *p = s;
uint32_t *p4;
long popcount(void *s, long count) {
long bits = 0;
unsigned char *p;
uint32_t *p4 = s;
static const unsigned char bitsinbyte[256] = {0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,4,5,5,6,5,6,6,7,5,6,6,7,6,7,7,8};
/* Count initial bytes not aligned to 32 bit. */
while((unsigned long)p & 3 && count) {
bits += bitsinbyte[*p++];
count--;
}
/* Count bits 16 bytes at a time */
p4 = (uint32_t*)p;
while(count>=16) {
uint32_t aux1, aux2, aux3, aux4;
@@ -94,99 +87,12 @@ size_t redisPopcount(void *s, long count) {
((((aux3 + (aux3 >> 4)) & 0x0F0F0F0F) * 0x01010101) >> 24) +
((((aux4 + (aux4 >> 4)) & 0x0F0F0F0F) * 0x01010101) >> 24);
}
/* Count the remaining bytes. */
/* Count the remaining bytes */
p = (unsigned char*)p4;
while(count--) bits += bitsinbyte[*p++];
return bits;
}
/* Return the position of the first bit set to one (if 'bit' is 1) or
* zero (if 'bit' is 0) in the bitmap starting at 's' and long 'count' bytes.
*
* The function is guaranteed to return a value >= 0 if 'bit' is 0 since if
* no zero bit is found, it returns count*8 assuming the string is zero
* padded on the right. However if 'bit' is 1 it is possible that there is
* not a single set bit in the bitmap. In this special case -1 is returned. */
long redisBitpos(void *s, long count, int bit) {
unsigned long *l;
unsigned char *c;
unsigned long skipval, word = 0, one;
long pos = 0; /* Position of bit, to return to the caller. */
int j;
/* Process whole words first, seeking for first word that is not
* all ones or all zeros respectively if we are lookig for zeros
* or ones. This is much faster with large strings having contiguous
* blocks of 1 or 0 bits compared to the vanilla bit per bit processing.
*
* Note that if we start from an address that is not aligned
* to sizeof(unsigned long) we consume it byte by byte until it is
* aligned. */
/* Skip initial bits not aligned to sizeof(unsigned long) byte by byte. */
skipval = bit ? 0 : UCHAR_MAX;
c = (unsigned char*) s;
while((unsigned long)c & (sizeof(*l)-1) && count) {
if (*c != skipval) break;
c++;
count--;
pos += 8;
}
/* Skip bits with full word step. */
skipval = bit ? 0 : ULONG_MAX;
l = (unsigned long*) c;
while (count >= sizeof(*l)) {
if (*l != skipval) break;
l++;
count -= sizeof(*l);
pos += sizeof(*l)*8;
}
/* Load bytes into "word" considering the first byte as the most significant
* (we basically consider it as written in big endian, since we consider the
* string as a set of bits from left to right, with the first bit at position
* zero.
*
* Note that the loading is designed to work even when the bytes left
* (count) are less than a full word. We pad it with zero on the right. */
c = (unsigned char*)l;
for (j = 0; j < sizeof(*l); j++) {
word <<= 8;
if (count) {
word |= *c;
c++;
count--;
}
}
/* Special case:
* If bits in the string are all zero and we are looking for one,
* return -1 to signal that there is not a single "1" in the whole
* string. This can't happen when we are looking for "0" as we assume
* that the right of the string is zero padded. */
if (bit == 1 && word == 0) return -1;
/* Last word left, scan bit by bit. The first thing we need is to
* have a single "1" set in the most significant position in an
* unsigned long. We don't know the size of the long so we use a
* simple trick. */
one = ULONG_MAX; /* All bits set to 1.*/
one >>= 1; /* All bits set to 1 but the MSB. */
one = ~one; /* All bits set to 0 but the MSB. */
while(one) {
if (((one & word) != 0) == bit) return pos;
pos++;
one >>= 1;
}
/* If we reached this point, there is a bug in the algorithm, since
* the case of no match is handled as a special case before. */
redisPanic("End of redisBitpos() reached.");
return 0; /* Just to avoid warnings. */
}
/* -----------------------------------------------------------------------------
* Bits related string commands: GETBIT, SETBIT, BITCOUNT, BITOP.
* -------------------------------------------------------------------------- */
@@ -223,7 +129,14 @@ void setbitCommand(redisClient *c) {
dbAdd(c->db,c->argv[1],o);
} else {
if (checkType(c,o,REDIS_STRING)) return;
o = dbUnshareStringValue(c->db,c->argv[1],o);
/* Create a copy when the object is shared or encoded. */
if (o->refcount != 1 || o->encoding != REDIS_ENCODING_RAW) {
robj *decoded = getDecodedObject(o);
o = createStringObject(decoded->ptr, sdslen(decoded->ptr));
decrRefCount(decoded);
dbOverwrite(c->db,c->argv[1],o);
}
}
/* Grow sds value to the right length if necessary */
@@ -240,7 +153,6 @@ void setbitCommand(redisClient *c) {
byteval |= ((on & 0x1) << bit);
((uint8_t*)o->ptr)[byte] = byteval;
signalModifiedKey(c->db,c->argv[1]);
notifyKeyspaceEvent(REDIS_NOTIFY_STRING,"setbit",c->argv[1],c->db->id);
server.dirty++;
addReply(c, bitval ? shared.cone : shared.czero);
}
@@ -261,13 +173,12 @@ void getbitCommand(redisClient *c) {
byte = bitoffset >> 3;
bit = 7 - (bitoffset & 0x7);
if (lzfEncodedObject(o)) o = dbUnshareStringValue(c->db,c->argv[1],o);
if (sdsEncodedObject(o)) {
if (byte < sdslen(o->ptr))
bitval = ((uint8_t*)o->ptr)[byte] & (1 << bit);
} else {
if (o->encoding != REDIS_ENCODING_RAW) {
if (byte < (size_t)ll2string(llbuf,sizeof(llbuf),(long)o->ptr))
bitval = llbuf[byte] & (1 << bit);
} else {
if (byte < sdslen(o->ptr))
bitval = ((uint8_t*)o->ptr)[byte] & (1 << bit);
}
addReply(c, bitval ? shared.cone : shared.czero);
@@ -278,7 +189,7 @@ void bitopCommand(redisClient *c) {
char *opname = c->argv[1]->ptr;
robj *o, *targetkey = c->argv[2];
long op, j, numkeys;
robj **objects; /* Array of source objects. */
robj **objects; /* Array of soruce objects. */
unsigned char **src; /* Array of source strings pointers. */
long *len, maxlen = 0; /* Array of length of src strings, and max len. */
long minlen = 0; /* Min len among the input keys. */
@@ -435,11 +346,9 @@ void bitopCommand(redisClient *c) {
if (maxlen) {
o = createObject(REDIS_STRING,res);
setKey(c->db,targetkey,o);
notifyKeyspaceEvent(REDIS_NOTIFY_STRING,"set",targetkey,c->db->id);
decrRefCount(o);
} else if (dbDelete(c->db,targetkey)) {
signalModifiedKey(c->db,targetkey);
notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,"del",targetkey,c->db->id);
}
server.dirty++;
addReplyLongLong(c,maxlen); /* Return the output string length in bytes. */
@@ -458,7 +367,6 @@ void bitcountCommand(redisClient *c) {
/* Set the 'p' pointer to the string, that can be just a stack allocated
* array if our string was integer encoded. */
if (lzfEncodedObject(o)) o = dbUnshareStringValue(c->db,c->argv[1],o);
if (o->encoding == REDIS_ENCODING_INT) {
p = (unsigned char*) llbuf;
strlen = ll2string(llbuf,sizeof(llbuf),(long)o->ptr);
@@ -496,94 +404,6 @@ void bitcountCommand(redisClient *c) {
} else {
long bytes = end-start+1;
addReplyLongLong(c,redisPopcount(p+start,bytes));
}
}
/* BITPOS key bit [start [end]] */
void bitposCommand(redisClient *c) {
robj *o;
long bit, start, end, strlen;
unsigned char *p;
char llbuf[32];
int end_given = 0;
/* Parse the bit argument to understand what we are looking for, set
* or clear bits. */
if (getLongFromObjectOrReply(c,c->argv[2],&bit,NULL) != REDIS_OK)
return;
if (bit != 0 && bit != 1) {
addReplyError(c, "The bit argument must be 1 or 0.");
return;
}
/* If the key does not exist, from our point of view it is an infinite
* array of 0 bits. If the user is looking for the fist clear bit return 0,
* If the user is looking for the first set bit, return -1. */
if ((o = lookupKeyRead(c->db,c->argv[1])) == NULL) {
addReplyLongLong(c, bit ? -1 : 0);
return;
}
if (checkType(c,o,REDIS_STRING)) return;
/* Set the 'p' pointer to the string, that can be just a stack allocated
* array if our string was integer encoded. */
if (lzfEncodedObject(o)) o = dbUnshareStringValue(c->db,c->argv[1],o);
if (o->encoding == REDIS_ENCODING_INT) {
p = (unsigned char*) llbuf;
strlen = ll2string(llbuf,sizeof(llbuf),(long)o->ptr);
} else {
p = (unsigned char*) o->ptr;
strlen = sdslen(o->ptr);
}
/* Parse start/end range if any. */
if (c->argc == 4 || c->argc == 5) {
if (getLongFromObjectOrReply(c,c->argv[3],&start,NULL) != REDIS_OK)
return;
if (c->argc == 5) {
if (getLongFromObjectOrReply(c,c->argv[4],&end,NULL) != REDIS_OK)
return;
end_given = 1;
} else {
end = strlen-1;
}
/* Convert negative indexes */
if (start < 0) start = strlen+start;
if (end < 0) end = strlen+end;
if (start < 0) start = 0;
if (end < 0) end = 0;
if (end >= strlen) end = strlen-1;
} else if (c->argc == 3) {
/* The whole string. */
start = 0;
end = strlen-1;
} else {
/* Syntax error. */
addReply(c,shared.syntaxerr);
return;
}
/* For empty ranges (start > end) we return -1 as an empty range does
* not contain a 0 nor a 1. */
if (start > end) {
addReplyLongLong(c, -1);
} else {
long bytes = end-start+1;
long pos = redisBitpos(p+start,bytes,bit);
/* If we are looking for clear bits, and the user specified an exact
* range with start-end, we can't consider the right of the range as
* zero padded (as we do when no explicit end is given).
*
* So if redisBitpos() returns the first bit outside the range,
* we return -1 to the caller, to mean, in the specified range there
* is not a single "0" bit. */
if (end_given && bit == 0 && pos == bytes*8) {
addReplyLongLong(c,-1);
return;
}
if (pos != -1) pos += start*8; /* Adjust for the bytes we skipped. */
addReplyLongLong(c,pos);
addReplyLongLong(c,popcount(p+start,bytes));
}
}
-158
View File
@@ -1,158 +0,0 @@
/* blocked.c - generic support for blocking operations like BLPOP & WAIT.
*
* Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*
* ---------------------------------------------------------------------------
*
* API:
*
* getTimeoutFromObjectOrReply() is just an utility function to parse a
* timeout argument since blocking operations usually require a timeout.
*
* blockClient() set the REDIS_BLOCKED flag in the client, and set the
* specified block type 'btype' filed to one of REDIS_BLOCKED_* macros.
*
* unblockClient() unblocks the client doing the following:
* 1) It calls the btype-specific function to cleanup the state.
* 2) It unblocks the client by unsetting the REDIS_BLOCKED flag.
* 3) It puts the client into a list of just unblocked clients that are
* processed ASAP in the beforeSleep() event loop callback, so that
* if there is some query buffer to process, we do it. This is also
* required because otherwise there is no 'readable' event fired, we
* already read the pending commands. We also set the REDIS_UNBLOCKED
* flag to remember the client is in the unblocked_clients list.
*
* processUnblockedClients() is called inside the beforeSleep() function
* to process the query buffer from unblocked clients and remove the clients
* from the blocked_clients queue.
*
* replyToBlockedClientTimedOut() is called by the cron function when
* a client blocked reaches the specified timeout (if the timeout is set
* to 0, no timeout is processed).
* It usually just needs to send a reply to the client.
*
* When implementing a new type of blocking opeation, the implementation
* should modify unblockClient() and replyToBlockedClientTimedOut() in order
* to handle the btype-specific behavior of this two functions.
*/
#include "redis.h"
/* Get a timeout value from an object and store it into 'timeout'.
* The final timeout is always stored as milliseconds as a time where the
* timeout will expire, however the parsing is performed according to
* the 'unit' that can be seconds or milliseconds.
*
* Note that if the timeout is zero (usually from the point of view of
* commands API this means no timeout) the value stored into 'timeout'
* is zero. */
int getTimeoutFromObjectOrReply(redisClient *c, robj *object, mstime_t *timeout, int unit) {
long long tval;
if (getLongLongFromObjectOrReply(c,object,&tval,
"timeout is not an integer or out of range") != REDIS_OK)
return REDIS_ERR;
if (tval < 0) {
addReplyError(c,"timeout is negative");
return REDIS_ERR;
}
if (tval > 0) {
if (unit == UNIT_SECONDS) tval *= 1000;
tval += mstime();
}
*timeout = tval;
return REDIS_OK;
}
/* Block a client for the specific operation type. Once the REDIS_BLOCKED
* flag is set client query buffer is not longer processed, but accumulated,
* and will be processed when the client is unblocked. */
void blockClient(redisClient *c, int btype) {
c->flags |= REDIS_BLOCKED;
c->btype = btype;
server.bpop_blocked_clients++;
}
/* This function is called in the beforeSleep() function of the event loop
* in order to process the pending input buffer of clients that were
* unblocked after a blocking operation. */
void processUnblockedClients(void) {
listNode *ln;
redisClient *c;
while (listLength(server.unblocked_clients)) {
ln = listFirst(server.unblocked_clients);
redisAssert(ln != NULL);
c = ln->value;
listDelNode(server.unblocked_clients,ln);
c->flags &= ~REDIS_UNBLOCKED;
c->btype = REDIS_BLOCKED_NONE;
/* Process remaining data in the input buffer. */
if (c->querybuf && sdslen(c->querybuf) > 0) {
server.current_client = c;
processInputBuffer(c);
server.current_client = NULL;
}
}
}
/* Unblock a client calling the right function depending on the kind
* of operation the client is blocking for. */
void unblockClient(redisClient *c) {
if (c->btype == REDIS_BLOCKED_LIST) {
unblockClientWaitingData(c);
} else if (c->btype == REDIS_BLOCKED_WAIT) {
unblockClientWaitingReplicas(c);
} else {
redisPanic("Unknown btype in unblockClient().");
}
/* Clear the flags, and put the client in the unblocked list so that
* we'll process new commands in its query buffer ASAP. */
c->flags &= ~REDIS_BLOCKED;
c->flags |= REDIS_UNBLOCKED;
c->btype = REDIS_BLOCKED_NONE;
server.bpop_blocked_clients--;
listAddNodeTail(server.unblocked_clients,c);
}
/* This function gets called when a blocked client timed out in order to
* send it a reply of some kind. */
void replyToBlockedClientTimedOut(redisClient *c) {
if (c->btype == REDIS_BLOCKED_LIST) {
addReply(c,shared.nullmultibulk);
} else if (c->btype == REDIS_BLOCKED_WAIT) {
addReplyLongLong(c,replicationCountAcksByOffset(c->bpop.reploffset));
} else {
redisPanic("Unknown btype in replyToBlockedClientTimedOut().");
}
}
-4182
View File
File diff suppressed because it is too large Load Diff
-237
View File
@@ -1,237 +0,0 @@
#ifndef __REDIS_CLUSTER_H
#define __REDIS_CLUSTER_H
/*-----------------------------------------------------------------------------
* Redis cluster data structures, defines, exported API.
*----------------------------------------------------------------------------*/
#define REDIS_CLUSTER_SLOTS 16384
#define REDIS_CLUSTER_OK 0 /* Everything looks ok */
#define REDIS_CLUSTER_FAIL 1 /* The cluster can't work */
#define REDIS_CLUSTER_NAMELEN 40 /* sha1 hex length */
#define REDIS_CLUSTER_PORT_INCR 10000 /* Cluster port = baseport + PORT_INCR */
/* The following defines are amunt of time, sometimes expressed as
* multiplicators of the node timeout value (when ending with MULT). */
#define REDIS_CLUSTER_DEFAULT_NODE_TIMEOUT 15000
#define REDIS_CLUSTER_FAIL_REPORT_VALIDITY_MULT 2 /* Fail report validity. */
#define REDIS_CLUSTER_FAIL_UNDO_TIME_MULT 2 /* Undo fail if master is back. */
#define REDIS_CLUSTER_FAIL_UNDO_TIME_ADD 10 /* Some additional time. */
#define REDIS_CLUSTER_SLAVE_VALIDITY_MULT 10 /* Slave data validity. */
#define REDIS_CLUSTER_FAILOVER_DELAY 5 /* Seconds */
#define REDIS_CLUSTER_DEFAULT_MIGRATION_BARRIER 1
#define REDIS_CLUSTER_MF_TIMEOUT 5000 /* Milliseconds to do a manual failover. */
#define REDIS_CLUSTER_MF_PAUSE_MULT 2 /* Master pause manual failover mult. */
/* Redirection errors returned by getNodeByQuery(). */
#define REDIS_CLUSTER_REDIR_NONE 0 /* Node can serve the request. */
#define REDIS_CLUSTER_REDIR_CROSS_SLOT 1 /* Keys in different slots. */
#define REDIS_CLUSTER_REDIR_UNSTABLE 2 /* Keys in slot resharding. */
#define REDIS_CLUSTER_REDIR_ASK 3 /* -ASK redirection required. */
#define REDIS_CLUSTER_REDIR_MOVED 4 /* -MOVED redirection required. */
struct clusterNode;
/* clusterLink encapsulates everything needed to talk with a remote node. */
typedef struct clusterLink {
mstime_t ctime; /* Link creation time */
int fd; /* TCP socket file descriptor */
sds sndbuf; /* Packet send buffer */
sds rcvbuf; /* Packet reception buffer */
struct clusterNode *node; /* Node related to this link if any, or NULL */
} clusterLink;
/* Cluster node flags and macros. */
#define REDIS_NODE_MASTER 1 /* The node is a master */
#define REDIS_NODE_SLAVE 2 /* The node is a slave */
#define REDIS_NODE_PFAIL 4 /* Failure? Need acknowledge */
#define REDIS_NODE_FAIL 8 /* The node is believed to be malfunctioning */
#define REDIS_NODE_MYSELF 16 /* This node is myself */
#define REDIS_NODE_HANDSHAKE 32 /* We have still to exchange the first ping */
#define REDIS_NODE_NOADDR 64 /* We don't know the address of this node */
#define REDIS_NODE_MEET 128 /* Send a MEET message to this node */
#define REDIS_NODE_PROMOTED 256 /* Master was a slave propoted by failover */
#define REDIS_NODE_NULL_NAME "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"
#define nodeIsMaster(n) ((n)->flags & REDIS_NODE_MASTER)
#define nodeIsSlave(n) ((n)->flags & REDIS_NODE_SLAVE)
#define nodeInHandshake(n) ((n)->flags & REDIS_NODE_HANDSHAKE)
#define nodeHasAddr(n) (!((n)->flags & REDIS_NODE_NOADDR))
#define nodeWithoutAddr(n) ((n)->flags & REDIS_NODE_NOADDR)
#define nodeTimedOut(n) ((n)->flags & REDIS_NODE_PFAIL)
#define nodeFailed(n) ((n)->flags & REDIS_NODE_FAIL)
/* This structure represent elements of node->fail_reports. */
struct clusterNodeFailReport {
struct clusterNode *node; /* Node reporting the failure condition. */
mstime_t time; /* Time of the last report from this node. */
} typedef clusterNodeFailReport;
struct clusterNode {
mstime_t ctime; /* Node object creation time. */
char name[REDIS_CLUSTER_NAMELEN]; /* Node name, hex string, sha1-size */
int flags; /* REDIS_NODE_... */
uint64_t configEpoch; /* Last configEpoch observed for this node */
unsigned char slots[REDIS_CLUSTER_SLOTS/8]; /* slots handled by this node */
int numslots; /* Number of slots handled by this node */
int numslaves; /* Number of slave nodes, if this is a master */
struct clusterNode **slaves; /* pointers to slave nodes */
struct clusterNode *slaveof; /* pointer to the master node */
mstime_t ping_sent; /* Unix time we sent latest ping */
mstime_t pong_received; /* Unix time we received the pong */
mstime_t fail_time; /* Unix time when FAIL flag was set */
mstime_t voted_time; /* Last time we voted for a slave of this master */
mstime_t repl_offset_time; /* Unix time we received offset for this node */
long long repl_offset; /* Last known repl offset for this node. */
char ip[REDIS_IP_STR_LEN]; /* Latest known IP address of this node */
int port; /* Latest known port of this node */
clusterLink *link; /* TCP/IP link with this node */
list *fail_reports; /* List of nodes signaling this as failing */
};
typedef struct clusterNode clusterNode;
typedef struct clusterState {
clusterNode *myself; /* This node */
uint64_t currentEpoch;
int state; /* REDIS_CLUSTER_OK, REDIS_CLUSTER_FAIL, ... */
int size; /* Num of master nodes with at least one slot */
dict *nodes; /* Hash table of name -> clusterNode structures */
dict *nodes_black_list; /* Nodes we don't re-add for a few seconds. */
clusterNode *migrating_slots_to[REDIS_CLUSTER_SLOTS];
clusterNode *importing_slots_from[REDIS_CLUSTER_SLOTS];
clusterNode *slots[REDIS_CLUSTER_SLOTS];
zskiplist *slots_to_keys;
/* The following fields are used to take the slave state on elections. */
mstime_t failover_auth_time; /* Time of previous or next election. */
int failover_auth_count; /* Number of votes received so far. */
int failover_auth_sent; /* True if we already asked for votes. */
int failover_auth_rank; /* This slave rank for current auth request. */
uint64_t failover_auth_epoch; /* Epoch of the current election. */
/* Manual failover state in common. */
mstime_t mf_end; /* Manual failover time limit (ms unixtime).
It is zero if there is no MF in progress. */
/* Manual failover state of master. */
clusterNode *mf_slave; /* Slave performing the manual failover. */
/* Manual failover state of slave. */
long long mf_master_offset; /* Master offset the slave needs to start MF
or zero if stil not received. */
int mf_can_start; /* If non-zero signal that the manual failover
can start requesting masters vote. */
/* The followign fields are uesd by masters to take state on elections. */
uint64_t lastVoteEpoch; /* Epoch of the last vote granted. */
int todo_before_sleep; /* Things to do in clusterBeforeSleep(). */
long long stats_bus_messages_sent; /* Num of msg sent via cluster bus. */
long long stats_bus_messages_received; /* Num of msg rcvd via cluster bus.*/
} clusterState;
/* clusterState todo_before_sleep flags. */
#define CLUSTER_TODO_HANDLE_FAILOVER (1<<0)
#define CLUSTER_TODO_UPDATE_STATE (1<<1)
#define CLUSTER_TODO_SAVE_CONFIG (1<<2)
#define CLUSTER_TODO_FSYNC_CONFIG (1<<3)
/* Redis cluster messages header */
/* Note that the PING, PONG and MEET messages are actually the same exact
* kind of packet. PONG is the reply to ping, in the exact format as a PING,
* while MEET is a special PING that forces the receiver to add the sender
* as a node (if it is not already in the list). */
#define CLUSTERMSG_TYPE_PING 0 /* Ping */
#define CLUSTERMSG_TYPE_PONG 1 /* Pong (reply to Ping) */
#define CLUSTERMSG_TYPE_MEET 2 /* Meet "let's join" message */
#define CLUSTERMSG_TYPE_FAIL 3 /* Mark node xxx as failing */
#define CLUSTERMSG_TYPE_PUBLISH 4 /* Pub/Sub Publish propagation */
#define CLUSTERMSG_TYPE_FAILOVER_AUTH_REQUEST 5 /* May I failover? */
#define CLUSTERMSG_TYPE_FAILOVER_AUTH_ACK 6 /* Yes, you have my vote */
#define CLUSTERMSG_TYPE_UPDATE 7 /* Another node slots configuration */
#define CLUSTERMSG_TYPE_MFSTART 8 /* Pause clients for manual failover */
/* Initially we don't know our "name", but we'll find it once we connect
* to the first node, using the getsockname() function. Then we'll use this
* address for all the next messages. */
typedef struct {
char nodename[REDIS_CLUSTER_NAMELEN];
uint32_t ping_sent;
uint32_t pong_received;
char ip[REDIS_IP_STR_LEN]; /* IP address last time it was seen */
uint16_t port; /* port last time it was seen */
uint16_t flags;
uint32_t notused; /* for 64 bit alignment */
} clusterMsgDataGossip;
typedef struct {
char nodename[REDIS_CLUSTER_NAMELEN];
} clusterMsgDataFail;
typedef struct {
uint32_t channel_len;
uint32_t message_len;
unsigned char bulk_data[8]; /* defined as 8 just for alignment concerns. */
} clusterMsgDataPublish;
typedef struct {
uint64_t configEpoch; /* Config epoch of the specified instance. */
char nodename[REDIS_CLUSTER_NAMELEN]; /* Name of the slots owner. */
unsigned char slots[REDIS_CLUSTER_SLOTS/8]; /* Slots bitmap. */
} clusterMsgDataUpdate;
union clusterMsgData {
/* PING, MEET and PONG */
struct {
/* Array of N clusterMsgDataGossip structures */
clusterMsgDataGossip gossip[1];
} ping;
/* FAIL */
struct {
clusterMsgDataFail about;
} fail;
/* PUBLISH */
struct {
clusterMsgDataPublish msg;
} publish;
/* UPDATE */
struct {
clusterMsgDataUpdate nodecfg;
} update;
};
typedef struct {
char sig[4]; /* Siganture "RCmb" (Redis Cluster message bus). */
uint32_t totlen; /* Total length of this message */
uint16_t ver; /* Protocol version, currently set to 0. */
uint16_t notused0; /* 2 bytes not used. */
uint16_t type; /* Message type */
uint16_t count; /* Only used for some kind of messages. */
uint64_t currentEpoch; /* The epoch accordingly to the sending node. */
uint64_t configEpoch; /* The config epoch if it's a master, or the last
epoch advertised by its master if it is a
slave. */
uint64_t offset; /* Master replication offset if node is a master or
processed replication offset if node is a slave. */
char sender[REDIS_CLUSTER_NAMELEN]; /* Name of the sender node */
unsigned char myslots[REDIS_CLUSTER_SLOTS/8];
char slaveof[REDIS_CLUSTER_NAMELEN];
char notused1[32]; /* 32 bytes reserved for future usage. */
uint16_t port; /* Sender TCP base port */
uint16_t flags; /* Sender node flags */
unsigned char state; /* Cluster state from the POV of the sender */
unsigned char mflags[3]; /* Message flags: CLUSTERMSG_FLAG[012]_... */
union clusterMsgData data;
} clusterMsg;
#define CLUSTERMSG_MIN_LEN (sizeof(clusterMsg)-sizeof(union clusterMsgData))
/* Message flags better specify the packet content or are used to
* provide some information about the node state. */
#define CLUSTERMSG_FLAG0_PAUSED (1<<0) /* Master paused for manual failover. */
#define CLUSTERMSG_FLAG0_FORCEACK (1<<1) /* Give ACK to AUTH_REQUEST even if
master is up. */
/* ---------------------- API exported outside cluster.c -------------------- */
clusterNode *getNodeByQuery(redisClient *c, struct redisCommand *cmd, robj **argv, int argc, int *hashslot, int *ask);
#endif /* __REDIS_CLUSTER_H */
+48 -941
View File
File diff suppressed because it is too large Load Diff
+4 -33
View File
@@ -45,9 +45,7 @@
/* Test for proc filesystem */
#ifdef __linux__
#define HAVE_PROC_STAT 1
#define HAVE_PROC_MAPS 1
#define HAVE_PROC_SMAPS 1
#define HAVE_PROCFS 1
#endif
/* Test for task_info() */
@@ -56,7 +54,7 @@
#endif
/* Test for backtrace() */
#if defined(__APPLE__) || defined(__linux__)
#if defined(__APPLE__) || defined(__linux__) || defined(__sun)
#define HAVE_BACKTRACE 1
#endif
@@ -105,20 +103,6 @@
#define rdb_fsync_range(fd,off,size) fsync(fd)
#endif
/* Check if we can use setproctitle().
* BSD systems have support for it, we provide an implementation for
* Linux and osx. */
#if (defined __NetBSD__ || defined __FreeBSD__ || defined __OpenBSD__)
#define USE_SETPROCTITLE
#endif
#if (defined __linux || defined __APPLE__)
#define USE_SETPROCTITLE
#define INIT_SETPROCTITLE_REPLACEMENT
void spt_init(int argc, char *argv[]);
void setproctitle(const char *fmt, ...);
#endif
/* Byte ordering detection */
#include <sys/types.h> /* This will likely define BYTE_ORDER */
@@ -153,27 +137,13 @@ void setproctitle(const char *fmt, ...);
#endif /* BSD */
#endif /* BYTE_ORDER */
/* Sometimes after including an OS-specific header that defines the
* endianess we end with __BYTE_ORDER but not with BYTE_ORDER that is what
* the Redis code uses. In this case let's define everything without the
* underscores. */
#ifndef BYTE_ORDER
#ifdef __BYTE_ORDER
#if defined(__LITTLE_ENDIAN) && defined(__BIG_ENDIAN)
#ifndef LITTLE_ENDIAN
#define LITTLE_ENDIAN __LITTLE_ENDIAN
#endif
#ifndef BIG_ENDIAN
#define BIG_ENDIAN __BIG_ENDIAN
#endif
#if defined(__BYTE_ORDER) && !defined(BYTE_ORDER)
#if (__BYTE_ORDER == __LITTLE_ENDIAN)
#define BYTE_ORDER LITTLE_ENDIAN
#else
#define BYTE_ORDER BIG_ENDIAN
#endif
#endif
#endif
#endif
#if !defined(BYTE_ORDER) || \
(BYTE_ORDER != BIG_ENDIAN && BYTE_ORDER != LITTLE_ENDIAN)
@@ -192,4 +162,5 @@ void setproctitle(const char *fmt, ...);
#endif
#endif
#endif
-88
View File
@@ -1,88 +0,0 @@
#include "redis.h"
/*
* Copyright 2001-2010 Georges Menie (www.menie.org)
* Copyright 2010-2012 Salvatore Sanfilippo (adapted to Redis coding style)
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the University of California, Berkeley nor the
* names of its contributors may be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL THE REGENTS AND CONTRIBUTORS BE LIABLE FOR ANY
* DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
* ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* CRC16 implementation according to CCITT standards.
*
* Note by @antirez: this is actually the XMODEM CRC 16 algorithm, using the
* following parameters:
*
* Name : "XMODEM", also known as "ZMODEM", "CRC-16/ACORN"
* Width : 16 bit
* Poly : 1021 (That is actually x^16 + x^12 + x^5 + 1)
* Initialization : 0000
* Reflect Input byte : False
* Reflect Output CRC : False
* Xor constant to output CRC : 0000
* Output for "123456789" : 31C3
*/
static const uint16_t crc16tab[256]= {
0x0000,0x1021,0x2042,0x3063,0x4084,0x50a5,0x60c6,0x70e7,
0x8108,0x9129,0xa14a,0xb16b,0xc18c,0xd1ad,0xe1ce,0xf1ef,
0x1231,0x0210,0x3273,0x2252,0x52b5,0x4294,0x72f7,0x62d6,
0x9339,0x8318,0xb37b,0xa35a,0xd3bd,0xc39c,0xf3ff,0xe3de,
0x2462,0x3443,0x0420,0x1401,0x64e6,0x74c7,0x44a4,0x5485,
0xa56a,0xb54b,0x8528,0x9509,0xe5ee,0xf5cf,0xc5ac,0xd58d,
0x3653,0x2672,0x1611,0x0630,0x76d7,0x66f6,0x5695,0x46b4,
0xb75b,0xa77a,0x9719,0x8738,0xf7df,0xe7fe,0xd79d,0xc7bc,
0x48c4,0x58e5,0x6886,0x78a7,0x0840,0x1861,0x2802,0x3823,
0xc9cc,0xd9ed,0xe98e,0xf9af,0x8948,0x9969,0xa90a,0xb92b,
0x5af5,0x4ad4,0x7ab7,0x6a96,0x1a71,0x0a50,0x3a33,0x2a12,
0xdbfd,0xcbdc,0xfbbf,0xeb9e,0x9b79,0x8b58,0xbb3b,0xab1a,
0x6ca6,0x7c87,0x4ce4,0x5cc5,0x2c22,0x3c03,0x0c60,0x1c41,
0xedae,0xfd8f,0xcdec,0xddcd,0xad2a,0xbd0b,0x8d68,0x9d49,
0x7e97,0x6eb6,0x5ed5,0x4ef4,0x3e13,0x2e32,0x1e51,0x0e70,
0xff9f,0xefbe,0xdfdd,0xcffc,0xbf1b,0xaf3a,0x9f59,0x8f78,
0x9188,0x81a9,0xb1ca,0xa1eb,0xd10c,0xc12d,0xf14e,0xe16f,
0x1080,0x00a1,0x30c2,0x20e3,0x5004,0x4025,0x7046,0x6067,
0x83b9,0x9398,0xa3fb,0xb3da,0xc33d,0xd31c,0xe37f,0xf35e,
0x02b1,0x1290,0x22f3,0x32d2,0x4235,0x5214,0x6277,0x7256,
0xb5ea,0xa5cb,0x95a8,0x8589,0xf56e,0xe54f,0xd52c,0xc50d,
0x34e2,0x24c3,0x14a0,0x0481,0x7466,0x6447,0x5424,0x4405,
0xa7db,0xb7fa,0x8799,0x97b8,0xe75f,0xf77e,0xc71d,0xd73c,
0x26d3,0x36f2,0x0691,0x16b0,0x6657,0x7676,0x4615,0x5634,
0xd94c,0xc96d,0xf90e,0xe92f,0x99c8,0x89e9,0xb98a,0xa9ab,
0x5844,0x4865,0x7806,0x6827,0x18c0,0x08e1,0x3882,0x28a3,
0xcb7d,0xdb5c,0xeb3f,0xfb1e,0x8bf9,0x9bd8,0xabbb,0xbb9a,
0x4a75,0x5a54,0x6a37,0x7a16,0x0af1,0x1ad0,0x2ab3,0x3a92,
0xfd2e,0xed0f,0xdd6c,0xcd4d,0xbdaa,0xad8b,0x9de8,0x8dc9,
0x7c26,0x6c07,0x5c64,0x4c45,0x3ca2,0x2c83,0x1ce0,0x0cc1,
0xef1f,0xff3e,0xcf5d,0xdf7c,0xaf9b,0xbfba,0x8fd9,0x9ff8,
0x6e17,0x7e36,0x4e55,0x5e74,0x2e93,0x3eb2,0x0ed1,0x1ef0
};
uint16_t crc16(const char *buf, int len) {
int counter;
uint16_t crc = 0;
for (counter = 0; counter < len; counter++)
crc = (crc<<8) ^ crc16tab[((crc>>8) ^ *buf++)&0x00FF];
return crc;
}
-8
View File
@@ -1,8 +0,0 @@
#ifndef CRC64_H
#define CRC64_H
#include <stdint.h>
uint64_t crc64(uint64_t crc, const unsigned char *s, uint64_t l);
#endif
+46 -515
View File
@@ -28,14 +28,12 @@
*/
#include "redis.h"
#include "cluster.h"
#include <signal.h>
#include <ctype.h>
void slotToKeyAdd(robj *key);
void slotToKeyDel(robj *key);
void slotToKeyFlush(void);
void SlotToKeyAdd(robj *key);
void SlotToKeyDel(robj *key);
/*-----------------------------------------------------------------------------
* C-level DB API
@@ -46,11 +44,11 @@ robj *lookupKey(redisDb *db, robj *key) {
if (de) {
robj *val = dictGetVal(de);
/* Update the access time for the ageing algorithm.
/* Update the access time for the aging algorithm.
* Don't do it if we have a saving child, as this will trigger
* a copy on write madness. */
if (server.rdb_child_pid == -1 && server.aof_child_pid == -1)
val->lru = LRU_CLOCK();
val->lru = server.lruclock;
return val;
} else {
return NULL;
@@ -87,7 +85,7 @@ robj *lookupKeyWriteOrReply(redisClient *c, robj *key, robj *reply) {
}
/* Add the key to the DB. It's up to the caller to increment the reference
* counter of the value if needed.
* counte of the value if needed.
*
* The program is aborted if the key already exists. */
void dbAdd(redisDb *db, robj *key, robj *val) {
@@ -95,7 +93,6 @@ void dbAdd(redisDb *db, robj *key, robj *val) {
int retval = dictAdd(db->dict, copy, val);
redisAssertWithInfo(NULL,key,retval == REDIS_OK);
if (server.cluster_enabled) slotToKeyAdd(key);
}
/* Overwrite an existing key with a new value. Incrementing the reference
@@ -104,7 +101,7 @@ void dbAdd(redisDb *db, robj *key, robj *val) {
*
* The program is aborted if the key was not already present. */
void dbOverwrite(redisDb *db, robj *key, robj *val) {
dictEntry *de = dictFind(db->dict,key->ptr);
struct dictEntry *de = dictFind(db->dict,key->ptr);
redisAssertWithInfo(NULL,key,de != NULL);
dictReplace(db->dict, key->ptr, val);
@@ -136,7 +133,7 @@ int dbExists(redisDb *db, robj *key) {
*
* The function makes sure to return keys not already expired. */
robj *dbRandomKey(redisDb *db) {
dictEntry *de;
struct dictEntry *de;
while(1) {
sds key;
@@ -163,61 +160,21 @@ int dbDelete(redisDb *db, robj *key) {
* the key, because it is shared with the main dictionary. */
if (dictSize(db->expires) > 0) dictDelete(db->expires,key->ptr);
if (dictDelete(db->dict,key->ptr) == DICT_OK) {
if (server.cluster_enabled) slotToKeyDel(key);
return 1;
} else {
return 0;
}
}
/* Prepare the string object stored at 'key' to be modified destructively
* to implement commands like SETBIT or APPEND.
*
* An object is usually ready to be modified unless one of the two conditions
* are true:
*
* 1) The object 'o' is shared (refcount > 1), we don't want to affect
* other users.
* 2) The object encoding is not "RAW".
*
* If the object is found in one of the above conditions (or both) by the
* function, an unshared / not-encoded copy of the string object is stored
* at 'key' in the specified 'db'. Otherwise the object 'o' itself is
* returned.
*
* USAGE:
*
* The object 'o' is what the caller already obtained by looking up 'key'
* in 'db', the usage pattern looks like this:
*
* o = lookupKeyWrite(db,key);
* if (checkType(c,o,REDIS_STRING)) return;
* o = dbUnshareStringValue(db,key,o);
*
* At this point the caller is ready to modify the object, for example
* using an sdscat() call to append some data, or anything else.
*/
robj *dbUnshareStringValue(redisDb *db, robj *key, robj *o) {
redisAssert(o->type == REDIS_STRING);
if (o->refcount != 1 || o->encoding != REDIS_ENCODING_RAW) {
robj *decoded = getDecodedObject(o);
o = createRawStringObject(decoded->ptr, sdslen(decoded->ptr));
decrRefCount(decoded);
dbOverwrite(db,key,o);
}
return o;
}
long long emptyDb(void(callback)(void*)) {
long long emptyDb() {
int j;
long long removed = 0;
for (j = 0; j < server.dbnum; j++) {
removed += dictSize(server.db[j].dict);
dictEmpty(server.db[j].dict,callback);
dictEmpty(server.db[j].expires,callback);
dictEmpty(server.db[j].dict);
dictEmpty(server.db[j].expires);
}
if (server.cluster_enabled) slotToKeyFlush();
return removed;
}
@@ -252,18 +209,17 @@ void signalFlushedDb(int dbid) {
void flushdbCommand(redisClient *c) {
server.dirty += dictSize(c->db->dict);
signalFlushedDb(c->db->id);
dictEmpty(c->db->dict,NULL);
dictEmpty(c->db->expires,NULL);
if (server.cluster_enabled) slotToKeyFlush();
dictEmpty(c->db->dict);
dictEmpty(c->db->expires);
addReply(c,shared.ok);
}
void flushallCommand(redisClient *c) {
signalFlushedDb(-1);
server.dirty += emptyDb(NULL);
server.dirty += emptyDb();
addReply(c,shared.ok);
if (server.rdb_child_pid != -1) {
kill(server.rdb_child_pid,SIGUSR1);
kill(server.rdb_child_pid,SIGKILL);
rdbRemoveTempFile(server.rdb_child_pid);
}
if (server.saveparamslen > 0) {
@@ -282,8 +238,6 @@ void delCommand(redisClient *c) {
for (j = 1; j < c->argc; j++) {
if (dbDelete(c->db,c->argv[j])) {
signalModifiedKey(c->db,c->argv[j]);
notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,
"del",c->argv[j],c->db->id);
server.dirty++;
deleted++;
}
@@ -307,10 +261,6 @@ void selectCommand(redisClient *c) {
"invalid DB index") != REDIS_OK)
return;
if (server.cluster_enabled && id != 0) {
addReplyError(c,"SELECT is not allowed in cluster mode");
return;
}
if (selectDb(c,id) == REDIS_ERR) {
addReplyError(c,"invalid DB index");
} else {
@@ -357,248 +307,6 @@ void keysCommand(redisClient *c) {
setDeferredMultiBulkLength(c,replylen,numkeys);
}
/* This callback is used by scanGenericCommand in order to collect elements
* returned by the dictionary iterator into a list. */
void scanCallback(void *privdata, const dictEntry *de) {
void **pd = (void**) privdata;
list *keys = pd[0];
robj *o = pd[1];
robj *key, *val = NULL;
if (o == NULL) {
sds sdskey = dictGetKey(de);
key = createStringObject(sdskey, sdslen(sdskey));
} else if (o->type == REDIS_SET) {
key = dictGetKey(de);
incrRefCount(key);
} else if (o->type == REDIS_HASH) {
key = dictGetKey(de);
incrRefCount(key);
val = dictGetVal(de);
incrRefCount(val);
} else if (o->type == REDIS_ZSET) {
key = dictGetKey(de);
incrRefCount(key);
val = createStringObjectFromLongDouble(*(double*)dictGetVal(de));
} else {
redisPanic("Type not handled in SCAN callback.");
}
listAddNodeTail(keys, key);
if (val) listAddNodeTail(keys, val);
}
/* Try to parse a SCAN cursor stored at object 'o':
* if the cursor is valid, store it as unsigned integer into *cursor and
* returns REDIS_OK. Otherwise return REDIS_ERR and send an error to the
* client. */
int parseScanCursorOrReply(redisClient *c, robj *o, unsigned long *cursor) {
char *eptr;
/* Use strtoul() because we need an *unsigned* long, so
* getLongLongFromObject() does not cover the whole cursor space. */
errno = 0;
*cursor = strtoul(o->ptr, &eptr, 10);
if (isspace(((char*)o->ptr)[0]) || eptr[0] != '\0' || errno == ERANGE)
{
addReplyError(c, "invalid cursor");
return REDIS_ERR;
}
return REDIS_OK;
}
/* This command implements SCAN, HSCAN and SSCAN commands.
* If object 'o' is passed, then it must be a Hash or Set object, otherwise
* if 'o' is NULL the command will operate on the dictionary associated with
* the current database.
*
* When 'o' is not NULL the function assumes that the first argument in
* the client arguments vector is a key so it skips it before iterating
* in order to parse options.
*
* In the case of a Hash object the function returns both the field and value
* of every element on the Hash. */
void scanGenericCommand(redisClient *c, robj *o, unsigned long cursor) {
int rv;
int i, j;
char buf[REDIS_LONGSTR_SIZE];
list *keys = listCreate();
listNode *node, *nextnode;
long count = 10;
sds pat;
int patlen, use_pattern = 0;
dict *ht;
/* Object must be NULL (to iterate keys names), or the type of the object
* must be Set, Sorted Set, or Hash. */
redisAssert(o == NULL || o->type == REDIS_SET || o->type == REDIS_HASH ||
o->type == REDIS_ZSET);
/* Set i to the first option argument. The previous one is the cursor. */
i = (o == NULL) ? 2 : 3; /* Skip the key argument if needed. */
/* Step 1: Parse options. */
while (i < c->argc) {
j = c->argc - i;
if (!strcasecmp(c->argv[i]->ptr, "count") && j >= 2) {
if (getLongFromObjectOrReply(c, c->argv[i+1], &count, NULL)
!= REDIS_OK)
{
goto cleanup;
}
if (count < 1) {
addReply(c,shared.syntaxerr);
goto cleanup;
}
i += 2;
} else if (!strcasecmp(c->argv[i]->ptr, "match") && j >= 2) {
pat = c->argv[i+1]->ptr;
patlen = sdslen(pat);
/* The pattern always matches if it is exactly "*", so it is
* equivalent to disabling it. */
use_pattern = !(pat[0] == '*' && patlen == 1);
i += 2;
} else {
addReply(c,shared.syntaxerr);
goto cleanup;
}
}
/* Step 2: Iterate the collection.
*
* Note that if the object is encoded with a ziplist, intset, or any other
* representation that is not a hash table, we are sure that it is also
* composed of a small number of elements. So to avoid taking state we
* just return everything inside the object in a single call, setting the
* cursor to zero to signal the end of the iteration. */
/* Handle the case of a hash table. */
ht = NULL;
if (o == NULL) {
ht = c->db->dict;
} else if (o->type == REDIS_SET && o->encoding == REDIS_ENCODING_HT) {
ht = o->ptr;
} else if (o->type == REDIS_HASH && o->encoding == REDIS_ENCODING_HT) {
ht = o->ptr;
count *= 2; /* We return key / value for this type. */
} else if (o->type == REDIS_ZSET && o->encoding == REDIS_ENCODING_SKIPLIST) {
zset *zs = o->ptr;
ht = zs->dict;
count *= 2; /* We return key / value for this type. */
}
if (ht) {
void *privdata[2];
/* We pass two pointers to the callback: the list to which it will
* add new elements, and the object containing the dictionary so that
* it is possible to fetch more data in a type-dependent way. */
privdata[0] = keys;
privdata[1] = o;
do {
cursor = dictScan(ht, cursor, scanCallback, privdata);
} while (cursor && listLength(keys) < count);
} else if (o->type == REDIS_SET) {
int pos = 0;
int64_t ll;
while(intsetGet(o->ptr,pos++,&ll))
listAddNodeTail(keys,createStringObjectFromLongLong(ll));
cursor = 0;
} else if (o->type == REDIS_HASH || o->type == REDIS_ZSET) {
unsigned char *p = ziplistIndex(o->ptr,0);
unsigned char *vstr;
unsigned int vlen;
long long vll;
while(p) {
ziplistGet(p,&vstr,&vlen,&vll);
listAddNodeTail(keys,
(vstr != NULL) ? createStringObject((char*)vstr,vlen) :
createStringObjectFromLongLong(vll));
p = ziplistNext(o->ptr,p);
}
cursor = 0;
} else {
redisPanic("Not handled encoding in SCAN.");
}
/* Step 3: Filter elements. */
node = listFirst(keys);
while (node) {
robj *kobj = listNodeValue(node);
nextnode = listNextNode(node);
int filter = 0;
/* Filter element if it does not match the pattern. */
if (!filter && use_pattern) {
if (sdsEncodedObject(kobj)) {
if (!stringmatchlen(pat, patlen, kobj->ptr, sdslen(kobj->ptr), 0))
filter = 1;
} else {
char buf[REDIS_LONGSTR_SIZE];
int len;
redisAssert(kobj->encoding == REDIS_ENCODING_INT);
len = ll2string(buf,sizeof(buf),(long)kobj->ptr);
if (!stringmatchlen(pat, patlen, buf, len, 0)) filter = 1;
}
}
/* Filter element if it is an expired key. */
if (!filter && o == NULL && expireIfNeeded(c->db, kobj)) filter = 1;
/* Remove the element and its associted value if needed. */
if (filter) {
decrRefCount(kobj);
listDelNode(keys, node);
}
/* If this is a hash or a sorted set, we have a flat list of
* key-value elements, so if this element was filtered, remove the
* value, or skip it if it was not filtered: we only match keys. */
if (o && (o->type == REDIS_ZSET || o->type == REDIS_HASH)) {
node = nextnode;
nextnode = listNextNode(node);
if (filter) {
kobj = listNodeValue(node);
decrRefCount(kobj);
listDelNode(keys, node);
}
}
node = nextnode;
}
/* Step 4: Reply to the client. */
addReplyMultiBulkLen(c, 2);
rv = snprintf(buf, sizeof(buf), "%lu", cursor);
redisAssert(rv < sizeof(buf));
addReplyBulkCBuffer(c, buf, rv);
addReplyMultiBulkLen(c, listLength(keys));
while ((node = listFirst(keys)) != NULL) {
robj *kobj = listNodeValue(node);
addReplyBulk(c, kobj);
decrRefCount(kobj);
listDelNode(keys, node);
}
cleanup:
listSetFreeMethod(keys,decrRefCountVoid);
listRelease(keys);
}
/* The SCAN command completely relies on scanGenericCommand. */
void scanCommand(redisClient *c) {
unsigned long cursor;
if (parseScanCursorOrReply(c,c->argv[1],&cursor) == REDIS_ERR) return;
scanGenericCommand(c,NULL,cursor);
}
void dbsizeCommand(redisClient *c) {
addReplyLongLong(c,dictSize(c->db->dict));
}
@@ -643,14 +351,6 @@ void shutdownCommand(redisClient *c) {
return;
}
}
/* When SHUTDOWN is called while the server is loading a dataset in
* memory we need to make sure no attempt is performed to save
* the dataset on shutdown (otherwise it could overwrite the current DB
* with half-read data).
*
* Also when in Sentinel mode clear the SAVE flag and force NOSAVE. */
if (server.loading || server.sentinel_mode)
flags = (flags & ~REDIS_SHUTDOWN_SAVE) | REDIS_SHUTDOWN_NOSAVE;
if (prepareForShutdown(flags) == REDIS_OK) exit(0);
addReplyError(c,"Errors trying to SHUTDOWN. Check logs.");
}
@@ -676,8 +376,7 @@ void renameGenericCommand(redisClient *c, int nx) {
addReply(c,shared.czero);
return;
}
/* Overwrite: delete the old key before creating the new one
* with the same name. */
/* Overwrite: delete the old key before creating the new one with the same name. */
dbDelete(c->db,c->argv[2]);
}
dbAdd(c->db,c->argv[2],o);
@@ -685,10 +384,6 @@ void renameGenericCommand(redisClient *c, int nx) {
dbDelete(c->db,c->argv[1]);
signalModifiedKey(c->db,c->argv[1]);
signalModifiedKey(c->db,c->argv[2]);
notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,"rename_from",
c->argv[1],c->db->id);
notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,"rename_to",
c->argv[2],c->db->id);
server.dirty++;
addReply(c,nx ? shared.cone : shared.ok);
}
@@ -706,11 +401,6 @@ void moveCommand(redisClient *c) {
redisDb *src, *dst;
int srcid;
if (server.cluster_enabled) {
addReplyError(c,"MOVE is not allowed in cluster mode");
return;
}
/* Obtain source and target DB pointers */
src = c->db;
srcid = c->db->id;
@@ -803,28 +493,21 @@ void propagateExpire(redisDb *db, robj *key) {
if (server.aof_state != REDIS_AOF_OFF)
feedAppendOnlyFile(server.delCommand,db->id,argv,2);
replicationFeedSlaves(server.slaves,db->id,argv,2);
if (listLength(server.slaves))
replicationFeedSlaves(server.slaves,db->id,argv,2);
decrRefCount(argv[0]);
decrRefCount(argv[1]);
}
int expireIfNeeded(redisDb *db, robj *key) {
mstime_t when = getExpire(db,key);
mstime_t now;
long long when = getExpire(db,key);
if (when < 0) return 0; /* No expire for this key */
/* Don't expire anything while loading. It will be done later. */
if (server.loading) return 0;
/* If we are in the context of a Lua script, we claim that time is
* blocked to when the Lua script started. This way a key can expire
* only the first time it is accessed and not in the middle of the
* script execution, making propagation to slaves / AOF consistent.
* See issue #1525 on Github for more information. */
now = server.lua_caller ? server.lua_time_start : mstime();
/* If we are running in the context of a slave, return ASAP:
* the slave key expiration is controlled by the master that will
* send us synthesized DEL operations for expired keys.
@@ -832,16 +515,16 @@ int expireIfNeeded(redisDb *db, robj *key) {
* Still we try to return the right information to the caller,
* that is, 0 if we think the key should be still valid, 1 if
* we think the key is expired at this time. */
if (server.masterhost != NULL) return now > when;
if (server.masterhost != NULL) {
return mstime() > when;
}
/* Return when this key has not expired */
if (now <= when) return 0;
if (mstime() <= when) return 0;
/* Delete the key */
server.stat_expiredkeys++;
propagateExpire(db,key);
notifyKeyspaceEvent(REDIS_NOTIFY_EXPIRED,
"expired",key,db->id);
return dbDelete(db,key);
}
@@ -855,8 +538,9 @@ int expireIfNeeded(redisDb *db, robj *key) {
* for *AT variants of the command, or the current time for relative expires).
*
* unit is either UNIT_SECONDS or UNIT_MILLISECONDS, and is only used for
* the argv[2] parameter. The basetime is always specified in milliseconds. */
* the argv[2] parameter. The basetime is always specified in milliesconds. */
void expireGenericCommand(redisClient *c, long long basetime, int unit) {
dictEntry *de;
robj *key = c->argv[1], *param = c->argv[2];
long long when; /* unix time in milliseconds when the key will expire. */
@@ -866,12 +550,11 @@ void expireGenericCommand(redisClient *c, long long basetime, int unit) {
if (unit == UNIT_SECONDS) when *= 1000;
when += basetime;
/* No key, return zero. */
if (lookupKeyRead(c->db,key) == NULL) {
de = dictFind(c->db->dict,key->ptr);
if (de == NULL) {
addReply(c,shared.czero);
return;
}
/* EXPIRE with negative TTL, or EXPIREAT with a timestamp into the past
* should never be executed as a DEL when load the AOF or in the context
* of a slave instance.
@@ -889,14 +572,12 @@ void expireGenericCommand(redisClient *c, long long basetime, int unit) {
rewriteClientCommandVector(c,2,aux,key);
decrRefCount(aux);
signalModifiedKey(c->db,key);
notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,"del",key,c->db->id);
addReply(c, shared.cone);
return;
} else {
setExpire(c->db,key,when);
addReply(c,shared.cone);
signalModifiedKey(c->db,key);
notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,"expire",key,c->db->id);
server.dirty++;
return;
}
@@ -921,17 +602,10 @@ void pexpireatCommand(redisClient *c) {
void ttlGenericCommand(redisClient *c, int output_ms) {
long long expire, ttl = -1;
/* If the key does not exist at all, return -2 */
if (lookupKeyRead(c->db,c->argv[1]) == NULL) {
addReplyLongLong(c,-2);
return;
}
/* The key exists. Return -1 if it has no expire, or the actual
* TTL value otherwise. */
expire = getExpire(c->db,c->argv[1]);
if (expire != -1) {
ttl = expire-mstime();
if (ttl < 0) ttl = 0;
if (ttl < 0) ttl = -1;
}
if (ttl == -1) {
addReplyLongLong(c,-1);
@@ -968,8 +642,6 @@ void persistCommand(redisClient *c) {
* API to get key arguments from commands
* ---------------------------------------------------------------------------*/
/* The base case is to use the keys position as given in the command table
* (firstkey, lastkey, step). */
int *getKeysUsingCommandTable(struct redisCommand *cmd,robj **argv, int argc, int *numkeys) {
int j, i = 0, last, *keys;
REDIS_NOTUSED(argv);
@@ -989,65 +661,42 @@ int *getKeysUsingCommandTable(struct redisCommand *cmd,robj **argv, int argc, in
return keys;
}
/* Return all the arguments that are keys in the command passed via argc / argv.
*
* The command returns the positions of all the key arguments inside the array,
* so the actual return value is an heap allocated array of integers. The
* length of the array is returned by reference into *numkeys.
*
* 'cmd' must be point to the corresponding entry into the redisCommand
* table, according to the command name in argv[0].
*
* This function uses the command table if a command-specific helper function
* is not required, otherwise it calls the command-specific function. */
int *getKeysFromCommand(struct redisCommand *cmd, robj **argv, int argc, int *numkeys) {
int *getKeysFromCommand(struct redisCommand *cmd,robj **argv, int argc, int *numkeys, int flags) {
if (cmd->getkeys_proc) {
return cmd->getkeys_proc(cmd,argv,argc,numkeys);
return cmd->getkeys_proc(cmd,argv,argc,numkeys,flags);
} else {
return getKeysUsingCommandTable(cmd,argv,argc,numkeys);
}
}
/* Free the result of getKeysFromCommand. */
void getKeysFreeResult(int *result) {
zfree(result);
}
/* Helper function to extract keys from following commands:
* ZUNIONSTORE <destkey> <num-keys> <key> <key> ... <key> <options>
* ZINTERSTORE <destkey> <num-keys> <key> <key> ... <key> <options> */
int *zunionInterGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys) {
int i, num, *keys;
REDIS_NOTUSED(cmd);
num = atoi(argv[2]->ptr);
/* Sanity check. Don't return any key if the command is going to
* reply with syntax error. */
if (num > (argc-3)) {
int *noPreloadGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *numkeys, int flags) {
if (flags & REDIS_GETKEYS_PRELOAD) {
*numkeys = 0;
return NULL;
} else {
return getKeysUsingCommandTable(cmd,argv,argc,numkeys);
}
/* Keys in z{union,inter}store come from two places:
* argv[1] = storage key,
* argv[3...n] = keys to intersect */
keys = zmalloc(sizeof(int)*(num+1));
/* Add all key positions for argv[3...n] to keys[] */
for (i = 0; i < num; i++) keys[i] = 3+i;
/* Finally add the argv[1] key position (the storage key target). */
keys[num] = 1;
*numkeys = num+1; /* Total keys = {union,inter} keys + storage key */
return keys;
}
/* Helper function to extract keys from the following commands:
* EVAL <script> <num-keys> <key> <key> ... <key> [more stuff]
* EVALSHA <script> <num-keys> <key> <key> ... <key> [more stuff] */
int *evalGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys) {
int *renameGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *numkeys, int flags) {
if (flags & REDIS_GETKEYS_PRELOAD) {
int *keys = zmalloc(sizeof(int));
*numkeys = 1;
keys[0] = 1;
return keys;
} else {
return getKeysUsingCommandTable(cmd,argv,argc,numkeys);
}
}
int *zunionInterGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *numkeys, int flags) {
int i, num, *keys;
REDIS_NOTUSED(cmd);
REDIS_NOTUSED(flags);
num = atoi(argv[2]->ptr);
/* Sanity check. Don't return any key if the command is going to
@@ -1056,126 +705,8 @@ int *evalGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys)
*numkeys = 0;
return NULL;
}
keys = zmalloc(sizeof(int)*num);
*numkeys = num;
/* Add all key positions for argv[3...n] to keys[] */
for (i = 0; i < num; i++) keys[i] = 3+i;
return keys;
}
/* Helper function to extract keys from the SORT command.
*
* SORT <sort-key> ... STORE <store-key> ...
*
* The first argument of SORT is always a key, however a list of options
* follow in SQL-alike style. Here we parse just the minimum in order to
* correctly identify keys in the "STORE" option. */
int *sortGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys) {
int i, j, num, *keys;
REDIS_NOTUSED(cmd);
num = 0;
keys = zmalloc(sizeof(int)*2); /* Alloc 2 places for the worst case. */
keys[num++] = 1; /* <sort-key> is always present. */
/* Search for STORE option. By default we consider options to don't
* have arguments, so if we find an unknown option name we scan the
* next. However there are options with 1 or 2 arguments, so we
* provide a list here in order to skip the right number of args. */
struct {
char *name;
int skip;
} skiplist[] = {
{"limit", 2},
{"get", 1},
{"by", 1},
{NULL, 0} /* End of elements. */
};
for (i = 2; i < argc; i++) {
for (j = 0; skiplist[j].name != NULL; j++) {
if (!strcasecmp(argv[i]->ptr,skiplist[j].name)) {
i += skiplist[j].skip;
break;
} else if (!strcasecmp(argv[i]->ptr,"store") && i+1 < argc) {
/* Note: we don't increment "num" here and continue the loop
* to be sure to process the *last* "STORE" option if multiple
* ones are provided. This is same behavior as SORT. */
keys[num] = i+1; /* <store-key> */
break;
}
}
}
*numkeys = num;
return keys;
}
/* Slot to Key API. This is used by Redis Cluster in order to obtain in
* a fast way a key that belongs to a specified hash slot. This is useful
* while rehashing the cluster. */
void slotToKeyAdd(robj *key) {
unsigned int hashslot = keyHashSlot(key->ptr,sdslen(key->ptr));
zslInsert(server.cluster->slots_to_keys,hashslot,key);
incrRefCount(key);
}
void slotToKeyDel(robj *key) {
unsigned int hashslot = keyHashSlot(key->ptr,sdslen(key->ptr));
zslDelete(server.cluster->slots_to_keys,hashslot,key);
}
void slotToKeyFlush(void) {
zslFree(server.cluster->slots_to_keys);
server.cluster->slots_to_keys = zslCreate();
}
unsigned int getKeysInSlot(unsigned int hashslot, robj **keys, unsigned int count) {
zskiplistNode *n;
zrangespec range;
int j = 0;
range.min = range.max = hashslot;
range.minex = range.maxex = 0;
n = zslFirstInRange(server.cluster->slots_to_keys, range);
while(n && n->score == hashslot && count--) {
keys[j++] = n->obj;
n = n->level[0].forward;
}
return j;
}
unsigned int countKeysInSlot(unsigned int hashslot) {
zskiplist *zsl = server.cluster->slots_to_keys;
zskiplistNode *zn;
zrangespec range;
int rank, count = 0;
range.min = range.max = hashslot;
range.minex = range.maxex = 0;
/* Find first element in range */
zn = zslFirstInRange(zsl, range);
/* Use rank of first element, if any, to determine preliminary count */
if (zn != NULL) {
rank = zslGetRank(zsl, zn->score, zn->obj);
count = (zsl->length - (rank - 1));
/* Find last element in range */
zn = zslLastInRange(zsl, range);
/* Use rank of last element, if any, to determine the actual count */
if (zn != NULL) {
rank = zslGetRank(zsl, zn->score, zn->obj);
count -= (zsl->length - rank);
}
}
return count;
}
+96 -257
View File
@@ -29,7 +29,6 @@
#include "redis.h"
#include "sha1.h" /* SHA1 is used for DEBUG DIGEST */
#include "crc64.h"
#include <arpa/inet.h>
#include <signal.h>
@@ -38,13 +37,12 @@
#include <execinfo.h>
#include <ucontext.h>
#include <fcntl.h>
#include "bio.h"
#endif /* HAVE_BACKTRACE */
/* ================================= Debugging ============================== */
/* Compute the sha1 of string at 's' with 'len' bytes long.
* The SHA1 is then xored against the string pointed by digest.
* The SHA1 is then xored againt the string pointed by digest.
* Since xor is commutative, this operation is used in order to
* "add" digests relative to unordered elements.
*
@@ -69,7 +67,7 @@ void xorObjectDigest(unsigned char *digest, robj *o) {
}
/* This function instead of just computing the SHA1 and xoring it
* against digest, also perform the digest of "digest" itself and
* against diget, also perform the digest of "digest" itself and
* replace the old value with the new one.
*
* So the final digest will be:
@@ -261,7 +259,7 @@ void debugCommand(redisClient *c) {
addReply(c,shared.err);
return;
}
emptyDb(NULL);
emptyDb();
if (rdbLoad(server.rdb_filename) != REDIS_OK) {
addReplyError(c,"Error trying to load the RDB dump");
return;
@@ -269,7 +267,7 @@ void debugCommand(redisClient *c) {
redisLog(REDIS_WARNING,"DB reloaded by DEBUG RELOAD");
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"loadaof")) {
emptyDb(NULL);
emptyDb();
if (loadAppendOnlyFile(server.aof_filename) != REDIS_OK) {
addReply(c,shared.err);
return;
@@ -292,35 +290,10 @@ void debugCommand(redisClient *c) {
addReplyStatusFormat(c,
"Value at:%p refcount:%d "
"encoding:%s serializedlength:%lld "
"lru:%d lru_seconds_idle:%llu",
"lru:%d lru_seconds_idle:%lu",
(void*)val, val->refcount,
strenc, (long long) rdbSavedObjectLen(val),
val->lru, estimateObjectIdleTime(val));
} else if (!strcasecmp(c->argv[1]->ptr,"sdslen") && c->argc == 3) {
dictEntry *de;
robj *val;
sds key;
if ((de = dictFind(c->db->dict,c->argv[2]->ptr)) == NULL) {
addReply(c,shared.nokeyerr);
return;
}
val = dictGetVal(de);
key = dictGetKey(de);
if (val->type != REDIS_STRING ||
(!sdsEncodedObject(val) && val->encoding != REDIS_ENCODING_LZF))
{
addReplyError(c,"Not an sds/lzf encoded string.");
} else {
addReplyStatusFormat(c,
"key_sds_len:%lld, key_sds_avail:%lld, "
"val_sds_len:%lld, val_sds_avail:%lld",
(long long) sdslen(key),
(long long) sdsavail(key),
(long long) sdslen(val->ptr),
(long long) sdsavail(val->ptr));
}
} else if (!strcasecmp(c->argv[1]->ptr,"populate") && c->argc == 3) {
long keys, j;
robj *key, *val;
@@ -354,45 +327,12 @@ void debugCommand(redisClient *c) {
} else if (!strcasecmp(c->argv[1]->ptr,"sleep") && c->argc == 3) {
double dtime = strtod(c->argv[2]->ptr,NULL);
long long utime = dtime*1000000;
struct timespec tv;
tv.tv_sec = utime / 1000000;
tv.tv_nsec = (utime % 1000000) * 1000;
nanosleep(&tv, NULL);
usleep(utime);
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"set-active-expire") &&
c->argc == 3)
{
server.active_expire_enabled = atoi(c->argv[2]->ptr);
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"cmdkeys") && c->argc >= 3) {
struct redisCommand *cmd = lookupCommand(c->argv[2]->ptr);
int *keys, numkeys, j;
if (!cmd) {
addReplyError(c,"Invalid command specified");
return;
} else if ((cmd->arity > 0 && cmd->arity != c->argc-2) ||
((c->argc-2) < -cmd->arity))
{
addReplyError(c,"Invalid number of arguments specified for command");
return;
}
keys = getKeysFromCommand(cmd,c->argv+2,c->argc-2,&numkeys);
addReplyMultiBulkLen(c,numkeys);
for (j = 0; j < numkeys; j++) addReplyBulk(c,c->argv[keys[j]+2]);
getKeysFreeResult(keys);
} else if (!strcasecmp(c->argv[1]->ptr,"error") && c->argc == 3) {
sds errstr = sdsnewlen("-",1);
errstr = sdscatsds(errstr,c->argv[2]->ptr);
errstr = sdsmapchars(errstr,"\n\r"," ",2); /* no newlines in errors. */
errstr = sdscatlen(errstr,"\r\n",2);
addReplySds(c,errstr);
} else {
addReplyErrorFormat(c, "Unknown DEBUG subcommand or wrong number of arguments for '%s'",
(char*)c->argv[1]->ptr);
addReplyError(c,
"Syntax error, try DEBUG [SEGFAULT|OBJECT <key>|SWAPIN <key>|SWAPOUT <key>|RELOAD]");
}
}
@@ -423,7 +363,9 @@ void _redisAssertPrintClientInfo(redisClient *c) {
char buf[128];
char *arg;
if (c->argv[j]->type == REDIS_STRING && sdsEncodedObject(c->argv[j])) {
if (c->argv[j]->type == REDIS_STRING &&
c->argv[j]->encoding == REDIS_ENCODING_RAW)
{
arg = (char*) c->argv[j]->ptr;
} else {
snprintf(buf,sizeof(buf),"Object type: %d, encoding: %d",
@@ -439,13 +381,10 @@ void redisLogObjectDebugInfo(robj *o) {
redisLog(REDIS_WARNING,"Object type: %d", o->type);
redisLog(REDIS_WARNING,"Object encoding: %d", o->encoding);
redisLog(REDIS_WARNING,"Object refcount: %d", o->refcount);
if (o->type == REDIS_STRING && sdsEncodedObject(o)) {
redisLog(REDIS_WARNING,"Object raw string len: %zu", sdslen(o->ptr));
if (sdslen(o->ptr) < 4096) {
sds repr = sdscatrepr(sdsempty(),o->ptr,sdslen(o->ptr));
redisLog(REDIS_WARNING,"Object raw string content: %s", repr);
sdsfree(repr);
}
if (o->type == REDIS_STRING && o->encoding == REDIS_ENCODING_RAW) {
redisLog(REDIS_WARNING,"Object raw string len: %d", sdslen(o->ptr));
if (sdslen(o->ptr) < 4096)
redisLog(REDIS_WARNING,"Object raw string content: \"%s\"", (char*)o->ptr);
} else if (o->type == REDIS_LIST) {
redisLog(REDIS_WARNING,"List length: %d", (int) listTypeLength(o));
} else if (o->type == REDIS_SET) {
@@ -526,13 +465,10 @@ static void *getMcontextEip(ucontext_t *uc) {
void logStackContent(void **sp) {
int i;
for (i = 15; i >= 0; i--) {
unsigned long addr = (unsigned long) sp+i;
unsigned long val = (unsigned long) sp[i];
if (sizeof(long) == 4)
redisLog(REDIS_WARNING, "(%08lx) -> %08lx", addr, val);
redisLog(REDIS_WARNING, "(%08lx) -> %08lx", sp+i, sp[i]);
else
redisLog(REDIS_WARNING, "(%016lx) -> %016lx", addr, val);
redisLog(REDIS_WARNING, "(%016lx) -> %016lx", sp+i, sp[i]);
}
}
@@ -550,27 +486,27 @@ void logRegisters(ucontext_t *uc) {
"R8 :%016lx R9 :%016lx\nR10:%016lx R11:%016lx\n"
"R12:%016lx R13:%016lx\nR14:%016lx R15:%016lx\n"
"RIP:%016lx EFL:%016lx\nCS :%016lx FS:%016lx GS:%016lx",
(unsigned long) uc->uc_mcontext->__ss.__rax,
(unsigned long) uc->uc_mcontext->__ss.__rbx,
(unsigned long) uc->uc_mcontext->__ss.__rcx,
(unsigned long) uc->uc_mcontext->__ss.__rdx,
(unsigned long) uc->uc_mcontext->__ss.__rdi,
(unsigned long) uc->uc_mcontext->__ss.__rsi,
(unsigned long) uc->uc_mcontext->__ss.__rbp,
(unsigned long) uc->uc_mcontext->__ss.__rsp,
(unsigned long) uc->uc_mcontext->__ss.__r8,
(unsigned long) uc->uc_mcontext->__ss.__r9,
(unsigned long) uc->uc_mcontext->__ss.__r10,
(unsigned long) uc->uc_mcontext->__ss.__r11,
(unsigned long) uc->uc_mcontext->__ss.__r12,
(unsigned long) uc->uc_mcontext->__ss.__r13,
(unsigned long) uc->uc_mcontext->__ss.__r14,
(unsigned long) uc->uc_mcontext->__ss.__r15,
(unsigned long) uc->uc_mcontext->__ss.__rip,
(unsigned long) uc->uc_mcontext->__ss.__rflags,
(unsigned long) uc->uc_mcontext->__ss.__cs,
(unsigned long) uc->uc_mcontext->__ss.__fs,
(unsigned long) uc->uc_mcontext->__ss.__gs
uc->uc_mcontext->__ss.__rax,
uc->uc_mcontext->__ss.__rbx,
uc->uc_mcontext->__ss.__rcx,
uc->uc_mcontext->__ss.__rdx,
uc->uc_mcontext->__ss.__rdi,
uc->uc_mcontext->__ss.__rsi,
uc->uc_mcontext->__ss.__rbp,
uc->uc_mcontext->__ss.__rsp,
uc->uc_mcontext->__ss.__r8,
uc->uc_mcontext->__ss.__r9,
uc->uc_mcontext->__ss.__r10,
uc->uc_mcontext->__ss.__r11,
uc->uc_mcontext->__ss.__r12,
uc->uc_mcontext->__ss.__r13,
uc->uc_mcontext->__ss.__r14,
uc->uc_mcontext->__ss.__r15,
uc->uc_mcontext->__ss.__rip,
uc->uc_mcontext->__ss.__rflags,
uc->uc_mcontext->__ss.__cs,
uc->uc_mcontext->__ss.__fs,
uc->uc_mcontext->__ss.__gs
);
logStackContent((void**)uc->uc_mcontext->__ss.__rsp);
#else
@@ -581,22 +517,22 @@ void logRegisters(ucontext_t *uc) {
"EDI:%08lx ESI:%08lx EBP:%08lx ESP:%08lx\n"
"SS:%08lx EFL:%08lx EIP:%08lx CS :%08lx\n"
"DS:%08lx ES:%08lx FS :%08lx GS :%08lx",
(unsigned long) uc->uc_mcontext->__ss.__eax,
(unsigned long) uc->uc_mcontext->__ss.__ebx,
(unsigned long) uc->uc_mcontext->__ss.__ecx,
(unsigned long) uc->uc_mcontext->__ss.__edx,
(unsigned long) uc->uc_mcontext->__ss.__edi,
(unsigned long) uc->uc_mcontext->__ss.__esi,
(unsigned long) uc->uc_mcontext->__ss.__ebp,
(unsigned long) uc->uc_mcontext->__ss.__esp,
(unsigned long) uc->uc_mcontext->__ss.__ss,
(unsigned long) uc->uc_mcontext->__ss.__eflags,
(unsigned long) uc->uc_mcontext->__ss.__eip,
(unsigned long) uc->uc_mcontext->__ss.__cs,
(unsigned long) uc->uc_mcontext->__ss.__ds,
(unsigned long) uc->uc_mcontext->__ss.__es,
(unsigned long) uc->uc_mcontext->__ss.__fs,
(unsigned long) uc->uc_mcontext->__ss.__gs
uc->uc_mcontext->__ss.__eax,
uc->uc_mcontext->__ss.__ebx,
uc->uc_mcontext->__ss.__ecx,
uc->uc_mcontext->__ss.__edx,
uc->uc_mcontext->__ss.__edi,
uc->uc_mcontext->__ss.__esi,
uc->uc_mcontext->__ss.__ebp,
uc->uc_mcontext->__ss.__esp,
uc->uc_mcontext->__ss.__ss,
uc->uc_mcontext->__ss.__eflags,
uc->uc_mcontext->__ss.__eip,
uc->uc_mcontext->__ss.__cs,
uc->uc_mcontext->__ss.__ds,
uc->uc_mcontext->__ss.__es,
uc->uc_mcontext->__ss.__fs,
uc->uc_mcontext->__ss.__gs
);
logStackContent((void**)uc->uc_mcontext->__ss.__esp);
#endif
@@ -610,22 +546,22 @@ void logRegisters(ucontext_t *uc) {
"EDI:%08lx ESI:%08lx EBP:%08lx ESP:%08lx\n"
"SS :%08lx EFL:%08lx EIP:%08lx CS:%08lx\n"
"DS :%08lx ES :%08lx FS :%08lx GS:%08lx",
(unsigned long) uc->uc_mcontext.gregs[11],
(unsigned long) uc->uc_mcontext.gregs[8],
(unsigned long) uc->uc_mcontext.gregs[10],
(unsigned long) uc->uc_mcontext.gregs[9],
(unsigned long) uc->uc_mcontext.gregs[4],
(unsigned long) uc->uc_mcontext.gregs[5],
(unsigned long) uc->uc_mcontext.gregs[6],
(unsigned long) uc->uc_mcontext.gregs[7],
(unsigned long) uc->uc_mcontext.gregs[18],
(unsigned long) uc->uc_mcontext.gregs[17],
(unsigned long) uc->uc_mcontext.gregs[14],
(unsigned long) uc->uc_mcontext.gregs[15],
(unsigned long) uc->uc_mcontext.gregs[3],
(unsigned long) uc->uc_mcontext.gregs[2],
(unsigned long) uc->uc_mcontext.gregs[1],
(unsigned long) uc->uc_mcontext.gregs[0]
uc->uc_mcontext.gregs[11],
uc->uc_mcontext.gregs[8],
uc->uc_mcontext.gregs[10],
uc->uc_mcontext.gregs[9],
uc->uc_mcontext.gregs[4],
uc->uc_mcontext.gregs[5],
uc->uc_mcontext.gregs[6],
uc->uc_mcontext.gregs[7],
uc->uc_mcontext.gregs[18],
uc->uc_mcontext.gregs[17],
uc->uc_mcontext.gregs[14],
uc->uc_mcontext.gregs[15],
uc->uc_mcontext.gregs[3],
uc->uc_mcontext.gregs[2],
uc->uc_mcontext.gregs[1],
uc->uc_mcontext.gregs[0]
);
logStackContent((void**)uc->uc_mcontext.gregs[7]);
#elif defined(__X86_64__) || defined(__x86_64__)
@@ -637,25 +573,25 @@ void logRegisters(ucontext_t *uc) {
"R8 :%016lx R9 :%016lx\nR10:%016lx R11:%016lx\n"
"R12:%016lx R13:%016lx\nR14:%016lx R15:%016lx\n"
"RIP:%016lx EFL:%016lx\nCSGSFS:%016lx",
(unsigned long) uc->uc_mcontext.gregs[13],
(unsigned long) uc->uc_mcontext.gregs[11],
(unsigned long) uc->uc_mcontext.gregs[14],
(unsigned long) uc->uc_mcontext.gregs[12],
(unsigned long) uc->uc_mcontext.gregs[8],
(unsigned long) uc->uc_mcontext.gregs[9],
(unsigned long) uc->uc_mcontext.gregs[10],
(unsigned long) uc->uc_mcontext.gregs[15],
(unsigned long) uc->uc_mcontext.gregs[0],
(unsigned long) uc->uc_mcontext.gregs[1],
(unsigned long) uc->uc_mcontext.gregs[2],
(unsigned long) uc->uc_mcontext.gregs[3],
(unsigned long) uc->uc_mcontext.gregs[4],
(unsigned long) uc->uc_mcontext.gregs[5],
(unsigned long) uc->uc_mcontext.gregs[6],
(unsigned long) uc->uc_mcontext.gregs[7],
(unsigned long) uc->uc_mcontext.gregs[16],
(unsigned long) uc->uc_mcontext.gregs[17],
(unsigned long) uc->uc_mcontext.gregs[18]
uc->uc_mcontext.gregs[13],
uc->uc_mcontext.gregs[11],
uc->uc_mcontext.gregs[14],
uc->uc_mcontext.gregs[12],
uc->uc_mcontext.gregs[8],
uc->uc_mcontext.gregs[9],
uc->uc_mcontext.gregs[10],
uc->uc_mcontext.gregs[15],
uc->uc_mcontext.gregs[0],
uc->uc_mcontext.gregs[1],
uc->uc_mcontext.gregs[2],
uc->uc_mcontext.gregs[3],
uc->uc_mcontext.gregs[4],
uc->uc_mcontext.gregs[5],
uc->uc_mcontext.gregs[6],
uc->uc_mcontext.gregs[7],
uc->uc_mcontext.gregs[16],
uc->uc_mcontext.gregs[17],
uc->uc_mcontext.gregs[18]
);
logStackContent((void**)uc->uc_mcontext.gregs[15]);
#endif
@@ -670,12 +606,11 @@ void logRegisters(ucontext_t *uc) {
void logStackTrace(ucontext_t *uc) {
void *trace[100];
int trace_size = 0, fd;
int log_to_stdout = server.logfile[0] == '\0';
/* Open the log file in append mode. */
fd = log_to_stdout ?
STDOUT_FILENO :
open(server.logfile, O_APPEND|O_CREAT|O_WRONLY, 0644);
fd = server.logfile ?
open(server.logfile, O_APPEND|O_CREAT|O_WRONLY, 0644) :
STDOUT_FILENO;
if (fd == -1) return;
/* Generate the stack trace */
@@ -689,7 +624,7 @@ void logStackTrace(ucontext_t *uc) {
backtrace_symbols_fd(trace, trace_size, fd);
/* Cleanup */
if (!log_to_stdout) close(fd);
if (server.logfile) close(fd);
}
/* Log information about the "current" client, that is, the client that is
@@ -723,96 +658,13 @@ void logCurrentClient(void) {
de = dictFind(cc->db->dict, key->ptr);
if (de) {
val = dictGetVal(de);
redisLog(REDIS_WARNING,"key '%s' found in DB containing the following object:", (char*)key->ptr);
redisLog(REDIS_WARNING,"key '%s' found in DB containing the following object:", key->ptr);
redisLogObjectDebugInfo(val);
}
decrRefCount(key);
}
}
#if defined(HAVE_PROC_MAPS)
void memtest_non_destructive_invert(void *addr, size_t size);
void memtest_non_destructive_swap(void *addr, size_t size);
#define MEMTEST_MAX_REGIONS 128
int memtest_test_linux_anonymous_maps(void) {
FILE *fp = fopen("/proc/self/maps","r");
char line[1024];
size_t start_addr, end_addr, size;
size_t start_vect[MEMTEST_MAX_REGIONS];
size_t size_vect[MEMTEST_MAX_REGIONS];
int regions = 0, j;
uint64_t crc1 = 0, crc2 = 0, crc3 = 0;
while(fgets(line,sizeof(line),fp) != NULL) {
char *start, *end, *p = line;
start = p;
p = strchr(p,'-');
if (!p) continue;
*p++ = '\0';
end = p;
p = strchr(p,' ');
if (!p) continue;
*p++ = '\0';
if (strstr(p,"stack") ||
strstr(p,"vdso") ||
strstr(p,"vsyscall")) continue;
if (!strstr(p,"00:00")) continue;
if (!strstr(p,"rw")) continue;
start_addr = strtoul(start,NULL,16);
end_addr = strtoul(end,NULL,16);
size = end_addr-start_addr;
start_vect[regions] = start_addr;
size_vect[regions] = size;
printf("Testing %lx %lu\n", (unsigned long) start_vect[regions],
(unsigned long) size_vect[regions]);
regions++;
}
/* Test all the regions as an unique sequential region.
* 1) Take the CRC64 of the memory region. */
for (j = 0; j < regions; j++) {
crc1 = crc64(crc1,(void*)start_vect[j],size_vect[j]);
}
/* 2) Invert bits, swap adjacent words, swap again, invert bits.
* This is the error amplification step. */
for (j = 0; j < regions; j++)
memtest_non_destructive_invert((void*)start_vect[j],size_vect[j]);
for (j = 0; j < regions; j++)
memtest_non_destructive_swap((void*)start_vect[j],size_vect[j]);
for (j = 0; j < regions; j++)
memtest_non_destructive_swap((void*)start_vect[j],size_vect[j]);
for (j = 0; j < regions; j++)
memtest_non_destructive_invert((void*)start_vect[j],size_vect[j]);
/* 3) Take the CRC64 sum again. */
for (j = 0; j < regions; j++)
crc2 = crc64(crc2,(void*)start_vect[j],size_vect[j]);
/* 4) Swap + Swap again */
for (j = 0; j < regions; j++)
memtest_non_destructive_swap((void*)start_vect[j],size_vect[j]);
for (j = 0; j < regions; j++)
memtest_non_destructive_swap((void*)start_vect[j],size_vect[j]);
/* 5) Take the CRC64 sum again. */
for (j = 0; j < regions; j++)
crc3 = crc64(crc3,(void*)start_vect[j],size_vect[j]);
/* NOTE: It is very important to close the file descriptor only now
* because closing it before may result into unmapping of some memory
* region that we are testing. */
fclose(fp);
/* If the two CRC are not the same, we trapped a memory error. */
return crc1 != crc2 || crc2 != crc3;
}
#endif
void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
ucontext_t *uc = (ucontext_t*) secret;
sds infostring, clients;
@@ -848,19 +700,6 @@ void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
/* Log dump of processor registers */
logRegisters(uc);
#if defined(HAVE_PROC_MAPS)
/* Test memory */
redisLog(REDIS_WARNING, "--- FAST MEMORY TEST");
bioKillThreads();
if (memtest_test_linux_anonymous_maps()) {
redisLog(REDIS_WARNING,
"!!! MEMORY ERROR DETECTED! Check your memory ASAP !!!");
} else {
redisLog(REDIS_WARNING,
"Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.");
}
#endif
redisLog(REDIS_WARNING,
"\n=== REDIS BUG REPORT END. Make sure to include from START to END. ===\n\n"
" Please report the crash opening an issue on github:\n\n"
@@ -938,7 +777,7 @@ void watchdogScheduleSignal(int period) {
setitimer(ITIMER_REAL, &it, NULL);
}
/* Enable the software watchdog with the specified period in milliseconds. */
/* Enable the software watchdong with the specified period in milliseconds. */
void enableWatchdog(int period) {
int min_period;
@@ -955,7 +794,7 @@ void enableWatchdog(int period) {
/* If the configured period is smaller than twice the timer period, it is
* too short for the software watchdog to work reliably. Fix it now
* if needed. */
min_period = (1000/server.hz)*2;
min_period = (1000/REDIS_HZ)*2;
if (period < min_period) period = min_period;
watchdogScheduleSignal(period); /* Adjust the current timer. */
server.watchdog_period = period;
+19 -284
View File
@@ -39,13 +39,13 @@
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#include <assert.h>
#include <limits.h>
#include <sys/time.h>
#include <ctype.h>
#include "dict.h"
#include "zmalloc.h"
#include "redisassert.h"
/* Using dictEnableResize() / dictDisableResize() we make possible to
* enable/disable resizing of the hash table as needed. This is very important
@@ -53,7 +53,7 @@
* around when there is a child performing saving operations.
*
* Note that even when dict_can_resize is set to 0, not all resizes are
* prevented: a hash table is still allowed to grow if the ratio between
* prevented: an hash table is still allowed to grow if the ratio between
* the number of elements and the buckets > dict_force_resize_ratio. */
static int dict_can_resize = 1;
static unsigned int dict_force_resize_ratio = 5;
@@ -239,7 +239,7 @@ int dictExpand(dict *d, unsigned long size)
/* Performs N steps of incremental rehashing. Returns 1 if there are still
* keys to move from the old to the new hash table, otherwise 0 is returned.
* Note that a rehashing step consists in moving a bucket (that may have more
* than one key as we use chaining) from the old to the new hash table. */
* thank one key as we use chaining) from the old to the new hash table. */
int dictRehash(dict *d, int n) {
if (!dictIsRehashing(d)) return 0;
@@ -324,7 +324,7 @@ int dictAdd(dict *d, void *key, void *val)
* a value returns the dictEntry structure to the user, that will make
* sure to fill the value field as he wishes.
*
* This function is also directly exposed to the user API to be called
* This function is also directly exposed to user API to be called
* mainly in order to store non-pointers inside the hash value, example:
*
* entry = dictAddRaw(dict,mykey);
@@ -444,15 +444,14 @@ int dictDeleteNoFree(dict *ht, const void *key) {
}
/* Destroy an entire dictionary */
int _dictClear(dict *d, dictht *ht, void(callback)(void *)) {
int _dictClear(dict *d, dictht *ht)
{
unsigned long i;
/* Free all the elements */
for (i = 0; i < ht->size && ht->used > 0; i++) {
dictEntry *he, *nextHe;
if (callback && (i & 65535) == 0) callback(d->privdata);
if ((he = ht->table[i]) == NULL) continue;
while(he) {
nextHe = he->next;
@@ -473,8 +472,8 @@ int _dictClear(dict *d, dictht *ht, void(callback)(void *)) {
/* Clear & Release the hash table */
void dictRelease(dict *d)
{
_dictClear(d,&d->ht[0],NULL);
_dictClear(d,&d->ht[1],NULL);
_dictClear(d,&d->ht[0]);
_dictClear(d,&d->ht[1]);
zfree(d);
}
@@ -506,44 +505,6 @@ void *dictFetchValue(dict *d, const void *key) {
return he ? dictGetVal(he) : NULL;
}
/* A fingerprint is a 64 bit number that represents the state of the dictionary
* at a given time, it's just a few dict properties xored together.
* When an unsafe iterator is initialized, we get the dict fingerprint, and check
* the fingerprint again when the iterator is released.
* If the two fingerprints are different it means that the user of the iterator
* performed forbidden operations against the dictionary while iterating. */
long long dictFingerprint(dict *d) {
long long integers[6], hash = 0;
int j;
integers[0] = (long) d->ht[0].table;
integers[1] = d->ht[0].size;
integers[2] = d->ht[0].used;
integers[3] = (long) d->ht[1].table;
integers[4] = d->ht[1].size;
integers[5] = d->ht[1].used;
/* We hash N integers by summing every successive integer with the integer
* hashing of the previous sum. Basically:
*
* Result = hash(hash(hash(int1)+int2)+int3) ...
*
* This way the same set of integers in a different order will (likely) hash
* to a different number. */
for (j = 0; j < 6; j++) {
hash += integers[j];
/* For the hashing step we use Tomas Wang's 64 bit integer hash. */
hash = (~hash) + (hash << 21); // hash = (hash << 21) - hash - 1;
hash = hash ^ (hash >> 24);
hash = (hash + (hash << 3)) + (hash << 8); // hash * 265
hash = hash ^ (hash >> 14);
hash = (hash + (hash << 2)) + (hash << 4); // hash * 21
hash = hash ^ (hash >> 28);
hash = hash + (hash << 31);
}
return hash;
}
dictIterator *dictGetIterator(dict *d)
{
dictIterator *iter = zmalloc(sizeof(*iter));
@@ -569,12 +530,8 @@ dictEntry *dictNext(dictIterator *iter)
while (1) {
if (iter->entry == NULL) {
dictht *ht = &iter->d->ht[iter->table];
if (iter->index == -1 && iter->table == 0) {
if (iter->safe)
iter->d->iterators++;
else
iter->fingerprint = dictFingerprint(iter->d);
}
if (iter->safe && iter->index == -1 && iter->table == 0)
iter->d->iterators++;
iter->index++;
if (iter->index >= (signed) ht->size) {
if (dictIsRehashing(iter->d) && iter->table == 0) {
@@ -601,12 +558,8 @@ dictEntry *dictNext(dictIterator *iter)
void dictReleaseIterator(dictIterator *iter)
{
if (!(iter->index == -1 && iter->table == 0)) {
if (iter->safe)
iter->d->iterators--;
else
assert(iter->fingerprint == dictFingerprint(iter->d));
}
if (iter->safe && !(iter->index == -1 && iter->table == 0))
iter->d->iterators--;
zfree(iter);
}
@@ -649,225 +602,6 @@ dictEntry *dictGetRandomKey(dict *d)
return he;
}
/* This is a version of dictGetRandomKey() that is modified in order to
* return multiple entries by jumping at a random place of the hash table
* and scanning linearly for entries.
*
* Returned pointers to hash table entries are stored into 'des' that
* points to an array of dictEntry pointers. The array must have room for
* at least 'count' elements, that is the argument we pass to the function
* to tell how many random elements we need.
*
* The function returns the number of items stored into 'des', that may
* be less than 'count' if the hash table has less than 'count' elements
* inside.
*
* Note that this function is not suitable when you need a good distribution
* of the returned items, but only when you need to "sample" a given number
* of continuous elements to run some kind of algorithm or to produce
* statistics. However the function is much faster than dictGetRandomKey()
* at producing N elements, and the elements are guaranteed to be non
* repeating. */
int dictGetRandomKeys(dict *d, dictEntry **des, int count) {
int j; /* internal hash table id, 0 or 1. */
int stored = 0;
if (dictSize(d) < count) count = dictSize(d);
while(stored < count) {
for (j = 0; j < 2; j++) {
/* Pick a random point inside the hash table 0 or 1. */
unsigned int i = random() & d->ht[j].sizemask;
int size = d->ht[j].size;
/* Make sure to visit every bucket by iterating 'size' times. */
while(size--) {
dictEntry *he = d->ht[j].table[i];
while (he) {
/* Collect all the elements of the buckets found non
* empty while iterating. */
*des = he;
des++;
he = he->next;
stored++;
if (stored == count) return stored;
}
i = (i+1) & d->ht[j].sizemask;
}
/* If there is only one table and we iterated it all, we should
* already have 'count' elements. Assert this condition. */
assert(dictIsRehashing(d) != 0);
}
}
return stored; /* Never reached. */
}
/* Function to reverse bits. Algorithm from:
* http://graphics.stanford.edu/~seander/bithacks.html#ReverseParallel */
static unsigned long rev(unsigned long v) {
unsigned long s = 8 * sizeof(v); // bit size; must be power of 2
unsigned long mask = ~0;
while ((s >>= 1) > 0) {
mask ^= (mask << s);
v = ((v >> s) & mask) | ((v << s) & ~mask);
}
return v;
}
/* dictScan() is used to iterate over the elements of a dictionary.
*
* Iterating works in the following way:
*
* 1) Initially you call the function using a cursor (v) value of 0.
* 2) The function performs one step of the iteration, and returns the
* new cursor value that you must use in the next call.
* 3) When the returned cursor is 0, the iteration is complete.
*
* The function guarantees that all the elements that are present in the
* dictionary from the start to the end of the iteration are returned.
* However it is possible that some element is returned multiple time.
*
* For every element returned, the callback 'fn' passed as argument is
* called, with 'privdata' as first argument and the dictionar entry
* 'de' as second argument.
*
* HOW IT WORKS.
*
* The algorithm used in the iteration was designed by Pieter Noordhuis.
* The main idea is to increment a cursor starting from the higher order
* bits, that is, instead of incrementing the cursor normally, the bits
* of the cursor are reversed, then the cursor is incremented, and finally
* the bits are reversed again.
*
* This strategy is needed because the hash table may be resized from one
* call to the other call of the same iteration.
*
* dict.c hash tables are always power of two in size, and they
* use chaining, so the position of an element in a given table is given
* always by computing the bitwise AND between Hash(key) and SIZE-1
* (where SIZE-1 is always the mask that is equivalent to taking the rest
* of the division between the Hash of the key and SIZE).
*
* For example if the current hash table size is 16, the mask is
* (in binary) 1111. The position of a key in the hash table will be always
* the last four bits of the hash output, and so forth.
*
* WHAT HAPPENS IF THE TABLE CHANGES IN SIZE?
*
* If the hash table grows, elements can go anyway in one multiple of
* the old bucket: for example let's say that we already iterated with
* a 4 bit cursor 1100, since the mask is 1111 (hash table size = 16).
*
* If the hash table will be resized to 64 elements, and the new mask will
* be 111111, the new buckets that you obtain substituting in ??1100
* either 0 or 1, can be targeted only by keys that we already visited
* when scanning the bucket 1100 in the smaller hash table.
*
* By iterating the higher bits first, because of the inverted counter, the
* cursor does not need to restart if the table size gets bigger, and will
* just continue iterating with cursors that don't have '1100' at the end,
* nor any other combination of final 4 bits already explored.
*
* Similarly when the table size shrinks over time, for example going from
* 16 to 8, If a combination of the lower three bits (the mask for size 8
* is 111) was already completely explored, it will not be visited again
* as we are sure that, we tried for example, both 0111 and 1111 (all the
* variations of the higher bit) so we don't need to test it again.
*
* WAIT... YOU HAVE *TWO* TABLES DURING REHASHING!
*
* Yes, this is true, but we always iterate the smaller one of the tables,
* testing also all the expansions of the current cursor into the larger
* table. So for example if the current cursor is 101 and we also have a
* larger table of size 16, we also test (0)101 and (1)101 inside the larger
* table. This reduces the problem back to having only one table, where
* the larger one, if exists, is just an expansion of the smaller one.
*
* LIMITATIONS
*
* This iterator is completely stateless, and this is a huge advantage,
* including no additional memory used.
*
* The disadvantages resulting from this design are:
*
* 1) It is possible that we return duplicated elements. However this is usually
* easy to deal with in the application level.
* 2) The iterator must return multiple elements per call, as it needs to always
* return all the keys chained in a given bucket, and all the expansions, so
* we are sure we don't miss keys moving.
* 3) The reverse cursor is somewhat hard to understand at first, but this
* comment is supposed to help.
*/
unsigned long dictScan(dict *d,
unsigned long v,
dictScanFunction *fn,
void *privdata)
{
dictht *t0, *t1;
const dictEntry *de;
unsigned long m0, m1;
if (dictSize(d) == 0) return 0;
if (!dictIsRehashing(d)) {
t0 = &(d->ht[0]);
m0 = t0->sizemask;
/* Emit entries at cursor */
de = t0->table[v & m0];
while (de) {
fn(privdata, de);
de = de->next;
}
} else {
t0 = &d->ht[0];
t1 = &d->ht[1];
/* Make sure t0 is the smaller and t1 is the bigger table */
if (t0->size > t1->size) {
t0 = &d->ht[1];
t1 = &d->ht[0];
}
m0 = t0->sizemask;
m1 = t1->sizemask;
/* Emit entries at cursor */
de = t0->table[v & m0];
while (de) {
fn(privdata, de);
de = de->next;
}
/* Iterate over indices in larger table that are the expansion
* of the index pointed to by the cursor in the smaller table */
do {
/* Emit entries at cursor */
de = t1->table[v & m1];
while (de) {
fn(privdata, de);
de = de->next;
}
/* Increment bits not covered by the smaller mask */
v = (((v | m0) + 1) & ~m0) | (v & m0);
/* Continue while bits covered by mask difference is non-zero */
} while (v & (m0 ^ m1));
}
/* Set unmasked bits so incrementing the reversed cursor
* operates on the masked bits of the smaller table */
v |= ~m0;
/* Increment the reverse cursor */
v = rev(v);
v++;
v = rev(v);
return v;
}
/* ------------------------- private functions ------------------------------ */
/* Expand the hash table if needed */
@@ -876,7 +610,7 @@ static int _dictExpandIfNeeded(dict *d)
/* Incremental rehashing already in progress. Return. */
if (dictIsRehashing(d)) return DICT_OK;
/* If the hash table is empty expand it to the initial size. */
/* If the hash table is empty expand it to the intial size. */
if (d->ht[0].size == 0) return dictExpand(d, DICT_HT_INITIAL_SIZE);
/* If we reached the 1:1 ratio, and we are allowed to resize the hash
@@ -887,7 +621,8 @@ static int _dictExpandIfNeeded(dict *d)
(dict_can_resize ||
d->ht[0].used/d->ht[0].size > dict_force_resize_ratio))
{
return dictExpand(d, d->ht[0].used*2);
return dictExpand(d, ((d->ht[0].size > d->ht[0].used) ?
d->ht[0].size : d->ht[0].used)*2);
}
return DICT_OK;
}
@@ -906,7 +641,7 @@ static unsigned long _dictNextPower(unsigned long size)
}
/* Returns the index of a free slot that can be populated with
* a hash entry for the given 'key'.
* an hash entry for the given 'key'.
* If the key already exists, -1 is returned.
*
* Note that if we are in the process of rehashing the hash table, the
@@ -935,9 +670,9 @@ static int _dictKeyIndex(dict *d, const void *key)
return idx;
}
void dictEmpty(dict *d, void(callback)(void*)) {
_dictClear(d,&d->ht[0],callback);
_dictClear(d,&d->ht[1],callback);
void dictEmpty(dict *d) {
_dictClear(d,&d->ht[0]);
_dictClear(d,&d->ht[1]);
d->rehashidx = -1;
d->iterators = 0;
}
+1 -6
View File
@@ -88,11 +88,8 @@ typedef struct dictIterator {
dict *d;
int table, index, safe;
dictEntry *entry, *nextEntry;
long long fingerprint; /* unsafe iterator fingerprint for misuse detection */
} dictIterator;
typedef void (dictScanFunction)(void *privdata, const dictEntry *de);
/* This is the initial size of every hash table */
#define DICT_HT_INITIAL_SIZE 4
@@ -157,18 +154,16 @@ dictIterator *dictGetSafeIterator(dict *d);
dictEntry *dictNext(dictIterator *iter);
void dictReleaseIterator(dictIterator *iter);
dictEntry *dictGetRandomKey(dict *d);
int dictGetRandomKeys(dict *d, dictEntry **des, int count);
void dictPrintStats(dict *d);
unsigned int dictGenHashFunction(const void *key, int len);
unsigned int dictGenCaseHashFunction(const unsigned char *buf, int len);
void dictEmpty(dict *d, void(callback)(void*));
void dictEmpty(dict *d);
void dictEnableResize(void);
void dictDisableResize(void);
int dictRehash(dict *d, int n);
int dictRehashMilliseconds(dict *d, int ms);
void dictSetHashFunctionSeed(unsigned int initval);
unsigned int dictGetHashFunctionSeed(void);
unsigned long dictScan(dict *d, unsigned long v, dictScanFunction *fn, void *privdata);
/* Hash table types */
extern dictType dictTypeHeapStringCopyKey;
-11
View File
@@ -33,7 +33,6 @@
#ifndef __ENDIANCONV_H
#define __ENDIANCONV_H
#include "config.h"
#include <stdint.h>
void memrev16(void *p);
@@ -61,14 +60,4 @@ uint64_t intrev64(uint64_t v);
#define intrev64ifbe(v) intrev64(v)
#endif
/* The functions htonu64() and ntohu64() convert the specified value to
* network byte ordering and back. In big endian systems they are no-ops. */
#if (BYTE_ORDER == BIG_ENDIAN)
#define htonu64(v) (v)
#define ntohu64(v) (v)
#else
#define htonu64(v) intrev64(v)
#define ntohu64(v) intrev64(v)
#endif
#endif
+1 -5
View File
@@ -38,11 +38,7 @@
#if defined(__linux__) || defined(__OpenBSD__)
#define _XOPEN_SOURCE 700
/*
* On NetBSD, _XOPEN_SOURCE undefines _NETBSD_SOURCE and
* thus hides inet_aton etc.
*/
#elif !defined(__NetBSD__)
#else
#define _XOPEN_SOURCE
#endif
+3 -13
View File
@@ -69,11 +69,6 @@ struct commandHelp {
"Pop a value from a list, push it to another list and return it; or block until one is available",
2,
"2.2.0" },
{ "CLIENT GETNAME",
"-",
"Get the current connection name",
9,
"2.6.9" },
{ "CLIENT KILL",
"ip:port",
"Kill the connection of a client",
@@ -84,11 +79,6 @@ struct commandHelp {
"Get the list of client connections",
9,
"2.4.0" },
{ "CLIENT SETNAME",
"connection-name",
"Set the current connection name",
9,
"2.6.9" },
{ "CONFIG GET",
"parameter",
"Get the value of a configuration parameter",
@@ -290,7 +280,7 @@ struct commandHelp {
1,
"2.6.0" },
{ "INFO",
"[section]",
"-",
"Get information and statistics about the server",
9,
"1.0.0" },
@@ -535,7 +525,7 @@ struct commandHelp {
8,
"1.0.0" },
{ "SET",
"key value [EX seconds] [PX milliseconds] [NX|XX]",
"key value",
"Set the string value of a key",
1,
"1.0.0" },
@@ -675,7 +665,7 @@ struct commandHelp {
7,
"2.2.0" },
{ "ZADD",
"key score member [score member ...]",
"key score member [score] [member]",
"Add one or more members to a sorted set, or update its score if it already exists",
4,
"1.2.0" },
-714
View File
@@ -1,714 +0,0 @@
/* hyperloglog.c - Redis HyperLogLog probabilistic cardinality approximation.
* This file implements the algorithm and the exported Redis commands.
*
* Copyright (c) 2014, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "redis.h"
#include <stdint.h>
#include <math.h>
/* The Redis HyperLogLog implementation is based on the following ideas:
*
* * The use of a 64 bit hash function as proposed in [1], in order to don't
* limited to cardinalities up to 10^9, at the cost of just 1 additional
* bit per register.
* * The use of 16384 6-bit registers for a great level of accuracy, using
* a total of 12k per key.
* * The use of the Redis string data type. No new type is introduced.
* * No attempt is made to compress the data structure as in [1]. Also the
* algorithm used is the original HyperLogLog Algorithm as in [2], with
* the only difference that a 64 bit hash function is used, so no correction
* is performed for values near 2^32 as in [1].
*
* [1] Heule, Nunkesser, Hall: HyperLogLog in Practice: Algorithmic
* Engineering of a State of The Art Cardinality Estimation Algorithm.
*
* [2] P. Flajolet, Éric Fusy, O. Gandouet, and F. Meunier. Hyperloglog: The
* analysis of a near-optimal cardinality estimation algorithm.
*
* The representation used by Redis is the following:
*
* +--------+--------+--------+------// //--+----------+------+-----+
* |11000000|22221111|33333322|55444444 .... | uint64_t | HYLL | Ver |
* +--------+--------+--------+------// //--+----------+------+-----+
*
* The 6 bits counters are encoded one after the other starting from the
* LSB to the MSB, and using the next bytes as needed.
*
* At the end of the 16k counters, there is an additional 64 bit integer
* stored in little endian format with the latest cardinality computed that
* can be reused if the data structure was not modified since the last
* computation (this is useful because there are high probabilities that
* HLLADD operations don't modify the actual data structure and hence the
* approximated cardinality).
*
* After the cached cardinality there are 4 bytes of magic set to the
* string "HYLL", and a 4 bytes version field that is reserved for
* future uses and is currently set to 0.
*
* When the most significant bit in the most significant byte of the cached
* cardinality is set, it means that the data structure was modified and
* we can't reuse the cached value that must be recomputed. */
#define REDIS_HLL_P 14 /* The greater is P, the smaller the error. */
#define REDIS_HLL_REGISTERS (1<<REDIS_HLL_P) /* With P=14, 16384 registers. */
#define REDIS_HLL_P_MASK (REDIS_HLL_REGISTERS-1) /* Mask to index register. */
#define REDIS_HLL_BITS 6 /* Enough to count up to 63 leading zeroes. */
#define REDIS_HLL_REGISTER_MAX ((1<<REDIS_HLL_BITS)-1)
/* Note: REDIS_HLL_SIZE define has a final "+8" since we store a 64 bit
* integer at the end of the HyperLogLog structure to cache the cardinality. */
#define REDIS_HLL_SIZE ((REDIS_HLL_REGISTERS*REDIS_HLL_BITS+7)/8)+8+8
/* =========================== Low level bit macros ========================= */
/* We need to get and set 6 bit counters in an array of 8 bit bytes.
* We use macros to make sure the code is inlined since speed is critical
* especially in order to compute the approximated cardinality in
* HLLCOUNT where we need to access all the registers at once.
* For the same reason we also want to avoid conditionals in this code path.
*
* +--------+--------+--------+------//
* |11000000|22221111|33333322|55444444
* +--------+--------+--------+------//
*
* Note: in the above representation the most significant bit (MSB)
* of every byte is on the left. We start using bits from the LSB to MSB,
* and so forth passing to the next byte.
*
* Example, we want to access to counter at pos = 1 ("111111" in the
* illustration above).
*
* The index of the first byte b0 containing our data is:
*
* b0 = 6 * pos / 8 = 0
*
* +--------+
* |11000000| <- Our byte at b0
* +--------+
*
* The position of the first bit (counting from the LSB = 0) in the byte
* is given by:
*
* fb = 6 * pos % 8 -> 6
*
* Right shift b0 of 'fb' bits.
*
* +--------+
* |11000000| <- Initial value of b0
* |00000011| <- After right shift of 6 pos.
* +--------+
*
* Left shift b1 of bits 8-fb bits (2 bits)
*
* +--------+
* |22221111| <- Initial value of b1
* |22111100| <- After left shift of 2 bits.
* +--------+
*
* OR the two bits, and finally AND with 111111 (63 in decimal) to
* clean the higher order bits we are not interested in:
*
* +--------+
* |00000011| <- b0 right shifted
* |22111100| <- b1 left shifted
* |22111111| <- b0 OR b1
* | 111111| <- (b0 OR b1) AND 63, our value.
* +--------+
*
* We can try with a different example, like pos = 0. In this case
* the 6-bit counter is actually contained in a single byte.
*
* b0 = 6 * pos / 8 = 0
*
* +--------+
* |11000000| <- Our byte at b0
* +--------+
*
* fb = 6 * pos % 8 = 0
*
* So we right shift of 0 bits (no shift in practice) and
* left shift the next byte of 8 bits, even if we don't use it,
* but this has the effect of clearing the bits so the result
* will not be affacted after the OR.
*
* -------------------------------------------------------------------------
*
* Setting the register is a bit more complex, let's assume that 'val'
* is the value we want to set, already in the right range.
*
* We need two steps, in one we need to clear the bits, and in the other
* we need to bitwise-OR the new bits.
*
* Let's try with 'pos' = 1, so our first byte at 'b' is 0,
*
* "fb" is 6 in this case.
*
* +--------+
* |11000000| <- Our byte at b0
* +--------+
*
* To create a AND-mask to clear the bits about this position, we just
* initialize the mask with the value 63, left shift it of "fs" bits,
* and finally invert the result.
*
* +--------+
* |00111111| <- "mask" starts at 63
* |11000000| <- "mask" after left shift of "ls" bits.
* |00111111| <- "mask" after invert.
* +--------+
*
* Now we can bitwise-AND the byte at "b" with the mask, and bitwise-OR
* it with "val" left-shifted of "ls" bits to set the new bits.
*
* Now let's focus on the next byte b1:
*
* +--------+
* |22221111| <- Initial value of b1
* +--------+
*
* To build the AND mask we start again with the 63 value, right shift
* it by 8-fb bits, and invert it.
*
* +--------+
* |00111111| <- "mask" set at 2&6-1
* |00001111| <- "mask" after the right shift by 8-fb = 2 bits
* |11110000| <- "mask" after bitwise not.
* +--------+
*
* Now we can mask it with b+1 to clear the old bits, and bitwise-OR
* with "val" left-shifted by "rs" bits to set the new value.
*/
/* Note: if we access the last counter, we will also access the b+1 byte
* that is out of the array, but sds strings always have an implicit null
* term, so the byte exists, and we can skip the conditional (or the need
* to allocate 1 byte more explicitly). */
/* Store the value of the register at position 'regnum' into variable 'target'.
* 'p' is an array of unsigned bytes. */
#define HLL_GET_REGISTER(target,p,regnum) do { \
uint8_t *_p = (uint8_t*) p; \
unsigned long _byte = regnum*REDIS_HLL_BITS/8; \
unsigned long _fb = regnum*REDIS_HLL_BITS&7; \
unsigned long _fb8 = 8 - _fb; \
unsigned long b0 = _p[_byte]; \
unsigned long b1 = _p[_byte+1]; \
target = ((b0 >> _fb) | (b1 << _fb8)) & REDIS_HLL_REGISTER_MAX; \
} while(0)
/* Set the value of the register at position 'regnum' to 'val'.
* 'p' is an array of unsigned bytes. */
#define HLL_SET_REGISTER(p,regnum,val) do { \
uint8_t *_p = (uint8_t*) p; \
unsigned long _byte = regnum*REDIS_HLL_BITS/8; \
unsigned long _fb = regnum*REDIS_HLL_BITS&7; \
unsigned long _fb8 = 8 - _fb; \
unsigned long _v = val; \
_p[_byte] &= ~(REDIS_HLL_REGISTER_MAX << _fb); \
_p[_byte] |= _v << _fb; \
_p[_byte+1] &= ~(REDIS_HLL_REGISTER_MAX >> _fb8); \
_p[_byte+1] |= _v >> _fb8; \
} while(0)
/* ========================= HyperLogLog algorithm ========================= */
/* Our hash function is MurmurHash2, 64 bit version.
* It was modified for Redis in order to provide the same result in
* big and little endian archs (endian neutral). */
uint64_t MurmurHash64A (const void * key, int len, unsigned int seed) {
const uint64_t m = 0xc6a4a7935bd1e995;
const int r = 47;
uint64_t h = seed ^ (len * m);
const uint8_t *data = (const uint8_t *)key;
const uint8_t *end = data + (len-(len&7));
while(data != end) {
uint64_t k;
#if (BYTE_ORDER == LITTLE_ENDIAN)
k = *((uint64_t*)data);
#else
k = (uint64_t) data[0];
k |= (uint64_t) data[1] << 8;
k |= (uint64_t) data[2] << 16;
k |= (uint64_t) data[3] << 24;
k |= (uint64_t) data[4] << 32;
k |= (uint64_t) data[5] << 40;
k |= (uint64_t) data[6] << 48;
k |= (uint64_t) data[7] << 56;
#endif
k *= m;
k ^= k >> r;
k *= m;
h ^= k;
h *= m;
data += 8;
}
switch(len & 7) {
case 7: h ^= (uint64_t)data[6] << 48;
case 6: h ^= (uint64_t)data[5] << 40;
case 5: h ^= (uint64_t)data[4] << 32;
case 4: h ^= (uint64_t)data[3] << 24;
case 3: h ^= (uint64_t)data[2] << 16;
case 2: h ^= (uint64_t)data[1] << 8;
case 1: h ^= (uint64_t)data[0];
h *= m;
};
h ^= h >> r;
h *= m;
h ^= h >> r;
return h;
}
/* "Add" the element in the hyperloglog data structure.
* Actually nothing is added, but the max 0 pattern counter of the subset
* the element belongs to is incremented if needed.
*
* 'registers' is expected to have room for REDIS_HLL_REGISTERS plus an
* additional byte on the right. This requirement is met by sds strings
* automatically since they are implicitly null terminated.
*
* The function always succeed, however if as a result of the operation
* the approximated cardinality changed, 1 is returned. Otherwise 0
* is returned. */
int hllAdd(uint8_t *registers, unsigned char *ele, size_t elesize) {
uint64_t hash, bit, index;
uint8_t oldcount, count;
/* Count the number of zeroes starting from bit REDIS_HLL_REGISTERS
* (that is a power of two corresponding to the first bit we don't use
* as index). The max run can be 64-P+1 bits.
*
* Note that the final "1" ending the sequence of zeroes must be
* included in the count, so if we find "001" the count is 3, and
* the smallest count possible is no zeroes at all, just a 1 bit
* at the first position, that is a count of 1.
*
* This may sound like inefficient, but actually in the average case
* there are high probabilities to find a 1 after a few iterations. */
hash = MurmurHash64A(ele,elesize,0xadc83b19ULL);
hash |= ((uint64_t)1<<63); /* Make sure the loop terminates. */
bit = REDIS_HLL_REGISTERS; /* First bit not used to address the register. */
count = 1; /* Initialized to 1 since we count the "00000...1" pattern. */
while((hash & bit) == 0) {
count++;
bit <<= 1;
}
/* Update the register if this element produced a longer run of zeroes. */
index = hash & REDIS_HLL_P_MASK; /* Index a register inside registers. */
HLL_GET_REGISTER(oldcount,registers,index);
if (count > oldcount) {
HLL_SET_REGISTER(registers,index,count);
return 1;
} else {
return 0;
}
}
/* Return the approximated cardinality of the set based on the armonic
* mean of the registers values. */
uint64_t hllCount(uint8_t *registers) {
double m = REDIS_HLL_REGISTERS;
double alpha = 0.7213/(1+1.079/m);
double E = 0;
int ez = 0; /* Number of registers equal to 0. */
int j;
/* We precompute 2^(-reg[j]) in a small table in order to
* speedup the computation of SUM(2^-register[0..i]). */
static int initialized = 0;
static double PE[64];
if (!initialized) {
PE[0] = 1; /* 2^(-reg[j]) is 1 when m is 0. */
for (j = 1; j < 64; j++) {
/* 2^(-reg[j]) is the same as 1/2^reg[j]. */
PE[j] = 1.0/(1ULL << j);
}
initialized = 1;
}
/* Compute SUM(2^-register[0..i]).
* Redis default is to use 16384 registers 6 bits each. The code works
* with other values by modifying the defines, but for our target value
* we take a faster path with unrolled loops. */
if (REDIS_HLL_REGISTERS == 16384 && REDIS_HLL_BITS == 6) {
uint8_t *r = registers;
unsigned long r0, r1, r2, r3, r4, r5, r6, r7, r8, r9,
r10, r11, r12, r13, r14, r15;
for (j = 0; j < 1024; j++) {
/* Handle 16 registers per iteration. */
r0 = r[0] & 63; if (r0 == 0) ez++;
r1 = (r[0] >> 6 | r[1] << 2) & 63; if (r1 == 0) ez++;
r2 = (r[1] >> 4 | r[2] << 4) & 63; if (r2 == 0) ez++;
r3 = (r[2] >> 2) & 63; if (r3 == 0) ez++;
r4 = r[3] & 63; if (r4 == 0) ez++;
r5 = (r[3] >> 6 | r[4] << 2) & 63; if (r5 == 0) ez++;
r6 = (r[4] >> 4 | r[5] << 4) & 63; if (r6 == 0) ez++;
r7 = (r[5] >> 2) & 63; if (r7 == 0) ez++;
r8 = r[6] & 63; if (r8 == 0) ez++;
r9 = (r[6] >> 6 | r[7] << 2) & 63; if (r9 == 0) ez++;
r10 = (r[7] >> 4 | r[8] << 4) & 63; if (r10 == 0) ez++;
r11 = (r[8] >> 2) & 63; if (r11 == 0) ez++;
r12 = r[9] & 63; if (r12 == 0) ez++;
r13 = (r[9] >> 6 | r[10] << 2) & 63; if (r13 == 0) ez++;
r14 = (r[10] >> 4 | r[11] << 4) & 63; if (r14 == 0) ez++;
r15 = (r[11] >> 2) & 63; if (r15 == 0) ez++;
/* Additional parens will allow the compiler to optimize the
* code more with a loss of precision that is not very relevant
* here (floating point math is not commutative!). */
E += (PE[r0] + PE[r1]) + (PE[r2] + PE[r3]) + (PE[r4] + PE[r5]) +
(PE[r6] + PE[r7]) + (PE[r8] + PE[r9]) + (PE[r10] + PE[r11]) +
(PE[r12] + PE[r13]) + (PE[r14] + PE[r15]);
r += 12;
}
} else {
for (j = 0; j < REDIS_HLL_REGISTERS; j++) {
unsigned long reg;
HLL_GET_REGISTER(reg,registers,j);
if (reg == 0) {
ez++;
E += 1; /* 2^(-reg[j]) is 1 when m is 0. */
} else {
E += PE[reg]; /* Precomputed 2^(-reg[j]). */
}
}
}
/* Muliply the inverse of E for alpha_m * m^2 to have the raw estimate. */
E = (1/E)*alpha*m*m;
/* Use the LINEARCOUNTING algorithm for small cardinalities.
* For larger values but up to 72000 HyperLogLog raw approximation is
* used since linear counting error starts to increase. However HyperLogLog
* shows a strong bias in the range 2.5*16384 - 72000, so we try to
* compensate for it. */
if (E < m*2.5 && ez != 0) {
E = m*log(m/ez); /* LINEARCOUNTING() */
} else if (m == 16384 && E < 72000) {
/* We did polynomial regression of the bias for this range, this
* way we can compute the bias for a given cardinality and correct
* according to it. Only apply the correction for P=14 that's what
* we use and the value the correction was verified with. */
double bias = 5.9119*1.0e-18*(E*E*E*E)
-1.4253*1.0e-12*(E*E*E)+
1.2940*1.0e-7*(E*E)
-5.2921*1.0e-3*E+
83.3216;
E -= E*(bias/100);
}
/* We don't apply the correction for E > 1/30 of 2^32 since we use
* a 64 bit function and 6 bit counters. To apply the correction for
* 1/30 of 2^64 is not needed since it would require a huge set
* to approach such a value. */
return (uint64_t) E;
}
/* ========================== HyperLogLog commands ========================== */
/* An HyperLogLog object is a string with space for 16k 6-bit integers,
* a cached 64 bit cardinality value, and a 4 byte "magic" and additional
* 4 bytes for version reserved for future use. */
robj *createHLLObject(void) {
robj *o;
char *p;
/* Create a string of the right size filled with zero bytes.
* Note that the cached cardinality is set to 0 as a side effect
* that is exactly the cardinality of an empty HLL. */
o = createObject(REDIS_STRING,sdsnewlen(NULL,REDIS_HLL_SIZE));
p = o->ptr;
memcpy(p+REDIS_HLL_SIZE-8,"HYLL",4);
return o;
}
/* Check if the object is a String of REDIS_HLL_SIZE bytes.
* Return REDIS_OK if this is true, otherwise reply to the client
* with an error and return REDIS_ERR. */
int isHLLObjectOrReply(redisClient *c, robj *o) {
/* Key exists, check type */
if (checkType(c,o,REDIS_STRING))
return REDIS_ERR; /* Error already sent. */
/* If this is a string representing an HLL, the size should match
* exactly. */
if (stringObjectLen(o) != REDIS_HLL_SIZE) {
addReplySds(c,
sdsnew("-WRONGTYPE Key is not a valid "
"HyperLogLog string value.\r\n"));
return REDIS_ERR;
}
return REDIS_OK;
}
/* PFADD var ele ele ele ... ele => :0 or :1 */
void pfaddCommand(redisClient *c) {
robj *o = lookupKeyWrite(c->db,c->argv[1]);
uint8_t *registers;
int updated = 0, j;
if (o == NULL) {
/* Create the key with a string value of the exact length to
* hold our HLL data structure. sdsnewlen() when NULL is passed
* is guaranteed to return bytes initialized to zero. */
o = createHLLObject();
dbAdd(c->db,c->argv[1],o);
updated++;
} else {
if (isHLLObjectOrReply(c,o) != REDIS_OK) return;
o = dbUnshareStringValue(c->db,c->argv[1],o);
}
/* Perform the low level ADD operation for every element. */
registers = o->ptr;
for (j = 2; j < c->argc; j++) {
if (hllAdd(registers, (unsigned char*)c->argv[j]->ptr,
sdslen(c->argv[j]->ptr)))
{
updated++;
}
}
if (updated) {
signalModifiedKey(c->db,c->argv[1]);
notifyKeyspaceEvent(REDIS_NOTIFY_STRING,"pfadd",c->argv[1],c->db->id);
server.dirty++;
/* Invalidate the cached cardinality. */
registers[REDIS_HLL_SIZE-9] |= (1<<7);
}
addReply(c, updated ? shared.cone : shared.czero);
}
/* PFCOUNT var -> approximated cardinality of set. */
void pfcountCommand(redisClient *c) {
robj *o = lookupKeyRead(c->db,c->argv[1]);
uint8_t *registers;
uint64_t card;
if (o == NULL) {
/* No key? Cardinality is zero since no element was added, otherwise
* we would have a key as HLLADD creates it as a side effect. */
addReply(c,shared.czero);
} else {
if (isHLLObjectOrReply(c,o) != REDIS_OK) return;
o = dbUnshareStringValue(c->db,c->argv[1],o);
/* Check if the cached cardinality is valid. */
registers = o->ptr;
if ((registers[REDIS_HLL_SIZE-9] & (1<<7)) == 0) {
/* Just return the cached value. */
card = (uint64_t)registers[REDIS_HLL_SIZE-16];
card |= (uint64_t)registers[REDIS_HLL_SIZE-15] << 8;
card |= (uint64_t)registers[REDIS_HLL_SIZE-14] << 16;
card |= (uint64_t)registers[REDIS_HLL_SIZE-13] << 24;
card |= (uint64_t)registers[REDIS_HLL_SIZE-12] << 32;
card |= (uint64_t)registers[REDIS_HLL_SIZE-11] << 40;
card |= (uint64_t)registers[REDIS_HLL_SIZE-10] << 48;
card |= (uint64_t)registers[REDIS_HLL_SIZE-9] << 56;
} else {
/* Recompute it and update the cached value. */
card = hllCount(registers);
registers[REDIS_HLL_SIZE-16] = card & 0xff;
registers[REDIS_HLL_SIZE-15] = (card >> 8) & 0xff;
registers[REDIS_HLL_SIZE-14] = (card >> 16) & 0xff;
registers[REDIS_HLL_SIZE-13] = (card >> 24) & 0xff;
registers[REDIS_HLL_SIZE-12] = (card >> 32) & 0xff;
registers[REDIS_HLL_SIZE-11] = (card >> 40) & 0xff;
registers[REDIS_HLL_SIZE-10] = (card >> 48) & 0xff;
registers[REDIS_HLL_SIZE-9] = (card >> 56) & 0xff;
/* This is not considered a read-only command even if the
* data structure is not modified, since the cached value
* may be modified and given that the HLL is a Redis string
* we need to propagate the change. */
signalModifiedKey(c->db,c->argv[1]);
server.dirty++;
}
addReplyLongLong(c,card);
}
}
/* PFMERGE dest src1 src2 src3 ... srcN => OK */
void pfmergeCommand(redisClient *c) {
uint8_t max[REDIS_HLL_REGISTERS];
uint8_t *registers;
int j, i;
/* Compute an HLL with M[i] = MAX(M[i]_j).
* We we the maximum into the max array of registers. We'll write
* it to the target variable later. */
memset(max,0,sizeof(max));
for (j = 1; j < c->argc; j++) {
uint8_t val;
/* Check type and size. */
robj *o = lookupKeyRead(c->db,c->argv[j]);
if (o == NULL) continue; /* Assume empty HLL for non existing var. */
if (isHLLObjectOrReply(c,o) != REDIS_OK) return;
/* Merge with this HLL with our 'max' HHL by setting max[i]
* to MAX(max[i],hll[i]). */
registers = o->ptr;
for (i = 0; i < REDIS_HLL_REGISTERS; i++) {
HLL_GET_REGISTER(val,registers,i);
if (val > max[i]) max[i] = val;
}
}
/* Create / unshare the destination key's value if needed. */
robj *o = lookupKeyRead(c->db,c->argv[1]);
if (o == NULL) {
/* Create the key with a string value of the exact length to
* hold our HLL data structure. sdsnewlen() when NULL is passed
* is guaranteed to return bytes initialized to zero. */
o = createHLLObject();
dbAdd(c->db,c->argv[1],o);
} else {
/* If key exists we are sure it's of the right type/size
* since we checked when merging the different HLLs, so we
* don't check again. */
o = dbUnshareStringValue(c->db,c->argv[1],o);
}
/* Write the resulting HLL to the destination HLL registers and
* invalidate the cached value. */
registers = o->ptr;
for (j = 0; j < REDIS_HLL_REGISTERS; j++) {
HLL_SET_REGISTER(registers,j,max[j]);
}
registers[REDIS_HLL_SIZE-9] |= (1<<7);
signalModifiedKey(c->db,c->argv[1]);
/* We generate an HLLADD event for HLLMERGE for semantical simplicity
* since in theory this is a mass-add of elements. */
notifyKeyspaceEvent(REDIS_NOTIFY_STRING,"pfadd",c->argv[1],c->db->id);
server.dirty++;
addReply(c,shared.ok);
}
/* ========================== Testing / Debugging ========================== */
/* PFSELFTEST
* This command performs a self-test of the HLL registers implementation.
* Something that is not easy to test from within the outside. */
#define REDIS_HLL_TEST_CYCLES 1000
void pfselftestCommand(redisClient *c) {
int j, i;
sds bitcounters = sdsnewlen(NULL,REDIS_HLL_SIZE);
uint8_t bytecounters[REDIS_HLL_REGISTERS];
/* Test 1: access registers.
* The test is conceived to test that the different counters of our data
* structure are accessible and that setting their values both result in
* the correct value to be retained and not affect adjacent values. */
for (j = 0; j < REDIS_HLL_TEST_CYCLES; j++) {
/* Set the HLL counters and an array of unsigned byes of the
* same size to the same set of random values. */
for (i = 0; i < REDIS_HLL_REGISTERS; i++) {
unsigned int r = rand() & REDIS_HLL_REGISTER_MAX;
bytecounters[i] = r;
HLL_SET_REGISTER(bitcounters,i,r);
}
/* Check that we are able to retrieve the same values. */
for (i = 0; i < REDIS_HLL_REGISTERS; i++) {
unsigned int val;
HLL_GET_REGISTER(val,bitcounters,i);
if (val != bytecounters[i]) {
addReplyErrorFormat(c,
"TESTFAILED Register %d should be %d but is %d",
i, (int) bytecounters[i], (int) val);
goto cleanup;
}
}
}
/* Test 2: approximation error.
* The test is adds unique elements and check that the estimated value
* is always reasonable bounds.
*
* We check that the error is smaller than 4 times than the expected
* standard error, to make it very unlikely for the test to fail because
* of a "bad" run. */
memset(bitcounters,0,REDIS_HLL_SIZE);
double relerr = 1.04/sqrt(REDIS_HLL_REGISTERS);
int64_t checkpoint = 1000;
uint64_t seed = (uint64_t)rand() | (uint64_t)rand() << 32;
uint64_t ele;
for (j = 1; j <= 10000000; j++) {
ele = j ^ seed;
hllAdd((uint8_t*)bitcounters,(unsigned char*)&ele,sizeof(ele));
if (j == checkpoint) {
int64_t abserr = checkpoint-
(int64_t)hllCount((uint8_t*)bitcounters);
if (abserr < 0) abserr = -abserr;
if (abserr > (uint64_t)(relerr*4*checkpoint)) {
addReplyErrorFormat(c,
"TESTFAILED Too big error. card:%llu abserr:%llu",
(unsigned long long) checkpoint,
(unsigned long long) abserr);
goto cleanup;
}
checkpoint *= 10;
}
}
/* Success! */
addReply(c,shared.ok);
cleanup:
sdsfree(bitcounters);
}
/* PFGETREG
* Return the registers values of the specified HLL. */
void pfgetregCommand(redisClient *c) {
robj *o = lookupKeyRead(c->db,c->argv[1]);
uint8_t *registers;
int j;
if (o == NULL) {
addReplyError(c,"The specified key does not exist");
return;
} else {
if (isHLLObjectOrReply(c,o) != REDIS_OK) return;
registers = o->ptr;
addReplyMultiBulkLen(c,REDIS_HLL_REGISTERS);
for (j = 0; j < REDIS_HLL_REGISTERS; j++) {
uint8_t val;
HLL_GET_REGISTER(val,registers,j);
addReplyLongLong(c,val);
}
}
}
+4 -4
View File
@@ -93,7 +93,7 @@
/*
* Avoid assigning values to errno variable? for some embedding purposes
* (linux kernel for example), this is necessary. NOTE: this breaks
* (linux kernel for example), this is neccessary. NOTE: this breaks
* the documentation in lzf.h.
*/
#ifndef AVOID_ERRNO
@@ -101,7 +101,7 @@
#endif
/*
* Whether to pass the LZF_STATE variable as argument, or allocate it
* Wether to pass the LZF_STATE variable as argument, or allocate it
* on the stack. For small-stack environments, define this to 1.
* NOTE: this breaks the prototype in lzf.h.
*/
@@ -110,11 +110,11 @@
#endif
/*
* Whether to add extra checks for input validity in lzf_decompress
* Wether to add extra checks for input validity in lzf_decompress
* and return EINVAL if the input stream has been corrupted. This
* only shields against overflowing the input buffer and will not
* detect most corrupted streams.
* This check is not normally noticeable on modern hardware
* This check is not normally noticable on modern hardware
* (<1% slowdown), but might slow down older cpus considerably.
*/
#ifndef CHECK_INPUT
+4 -29
View File
@@ -35,7 +35,6 @@
#include <errno.h>
#include <termios.h>
#include <sys/ioctl.h>
#include "config.h"
#if (ULONG_MAX == 4294967295UL)
#define MEMTEST_32BIT
@@ -79,8 +78,10 @@ void memtest_progress_end(void) {
void memtest_progress_step(size_t curr, size_t size, char c) {
size_t chars = ((unsigned long long)curr*progress_full)/size, j;
for (j = 0; j < chars-progress_printed; j++) printf("%c",c);
progress_printed = chars;
for (j = 0; j < chars-progress_printed; j++) {
printf("%c",c);
progress_printed++;
}
fflush(stdout);
}
@@ -239,32 +240,6 @@ void memtest_test(size_t megabytes, int passes) {
}
}
void memtest_non_destructive_invert(void *addr, size_t size) {
volatile unsigned long *p = addr;
size_t words = size / sizeof(unsigned long);
size_t j;
/* Invert */
for (j = 0; j < words; j++)
p[j] = ~p[j];
}
void memtest_non_destructive_swap(void *addr, size_t size) {
volatile unsigned long *p = addr;
size_t words = size / sizeof(unsigned long);
size_t j;
/* Swap */
for (j = 0; j < words; j += 2) {
unsigned long a, b;
a = p[j];
b = p[j+1];
p[j] = b;
p[j+1] = a;
}
}
void memtest(size_t megabytes, int passes) {
if (ioctl(1, TIOCGWINSZ, &ws) == -1) {
ws.ws_col = 80;
+247
View File
@@ -0,0 +1,247 @@
#include "redis.h"
#include "endianconv.h"
/* -----------------------------------------------------------------------------
* DUMP, RESTORE and MIGRATE commands
* -------------------------------------------------------------------------- */
/* Generates a DUMP-format representation of the object 'o', adding it to the
* io stream pointed by 'rio'. This function can't fail. */
void createDumpPayload(rio *payload, robj *o) {
unsigned char buf[2];
uint64_t crc;
/* Serialize the object in a RDB-like format. It consist of an object type
* byte followed by the serialized object. This is understood by RESTORE. */
rioInitWithBuffer(payload,sdsempty());
redisAssert(rdbSaveObjectType(payload,o));
redisAssert(rdbSaveObject(payload,o));
/* Write the footer, this is how it looks like:
* ----------------+---------------------+---------------+
* ... RDB payload | 2 bytes RDB version | 8 bytes CRC64 |
* ----------------+---------------------+---------------+
* RDB version and CRC are both in little endian.
*/
/* RDB version */
buf[0] = REDIS_RDB_VERSION & 0xff;
buf[1] = (REDIS_RDB_VERSION >> 8) & 0xff;
payload->io.buffer.ptr = sdscatlen(payload->io.buffer.ptr,buf,2);
/* CRC64 */
crc = crc64(0,(unsigned char*)payload->io.buffer.ptr,
sdslen(payload->io.buffer.ptr));
memrev64ifbe(&crc);
payload->io.buffer.ptr = sdscatlen(payload->io.buffer.ptr,&crc,8);
}
/* Verify that the RDB version of the dump payload matches the one of this Redis
* instance and that the checksum is ok.
* If the DUMP payload looks valid REDIS_OK is returned, otherwise REDIS_ERR
* is returned. */
int verifyDumpPayload(unsigned char *p, size_t len) {
unsigned char *footer;
uint16_t rdbver;
uint64_t crc;
/* At least 2 bytes of RDB version and 8 of CRC64 should be present. */
if (len < 10) return REDIS_ERR;
footer = p+(len-10);
/* Verify RDB version */
rdbver = (footer[1] << 8) | footer[0];
if (rdbver != REDIS_RDB_VERSION) return REDIS_ERR;
/* Verify CRC64 */
crc = crc64(0,p,len-8);
memrev64ifbe(&crc);
return (memcmp(&crc,footer+2,8) == 0) ? REDIS_OK : REDIS_ERR;
}
/* DUMP keyname
* DUMP is actually not used by Redis Cluster but it is the obvious
* complement of RESTORE and can be useful for different applications. */
void dumpCommand(redisClient *c) {
robj *o, *dumpobj;
rio payload;
/* Check if the key is here. */
if ((o = lookupKeyRead(c->db,c->argv[1])) == NULL) {
addReply(c,shared.nullbulk);
return;
}
/* Create the DUMP encoded representation. */
createDumpPayload(&payload,o);
/* Transfer to the client */
dumpobj = createObject(REDIS_STRING,payload.io.buffer.ptr);
addReplyBulk(c,dumpobj);
decrRefCount(dumpobj);
return;
}
/* RESTORE key ttl serialized-value */
void restoreCommand(redisClient *c) {
long ttl;
rio payload;
int type;
robj *obj;
/* Make sure this key does not already exist here... */
if (lookupKeyWrite(c->db,c->argv[1]) != NULL) {
addReplyError(c,"Target key name is busy.");
return;
}
/* Check if the TTL value makes sense */
if (getLongFromObjectOrReply(c,c->argv[2],&ttl,NULL) != REDIS_OK) {
return;
} else if (ttl < 0) {
addReplyError(c,"Invalid TTL value, must be >= 0");
return;
}
/* Verify RDB version and data checksum. */
if (verifyDumpPayload(c->argv[3]->ptr,sdslen(c->argv[3]->ptr)) == REDIS_ERR) {
addReplyError(c,"DUMP payload version or checksum are wrong");
return;
}
rioInitWithBuffer(&payload,c->argv[3]->ptr);
if (((type = rdbLoadObjectType(&payload)) == -1) ||
((obj = rdbLoadObject(type,&payload)) == NULL))
{
addReplyError(c,"Bad data format");
return;
}
/* Create the key and set the TTL if any */
dbAdd(c->db,c->argv[1],obj);
if (ttl) setExpire(c->db,c->argv[1],mstime()+ttl);
signalModifiedKey(c->db,c->argv[1]);
addReply(c,shared.ok);
server.dirty++;
}
/* MIGRATE host port key dbid timeout */
void migrateCommand(redisClient *c) {
int fd;
long timeout;
long dbid;
long long ttl = 0, expireat;
robj *o;
rio cmd, payload;
/* Sanity check */
if (getLongFromObjectOrReply(c,c->argv[5],&timeout,NULL) != REDIS_OK)
return;
if (getLongFromObjectOrReply(c,c->argv[4],&dbid,NULL) != REDIS_OK)
return;
if (timeout <= 0) timeout = 1000;
/* Check if the key is here. If not we reply with success as there is
* nothing to migrate (for instance the key expired in the meantime), but
* we include such information in the reply string. */
if ((o = lookupKeyRead(c->db,c->argv[3])) == NULL) {
addReplySds(c,sdsnew("+NOKEY\r\n"));
return;
}
/* Connect */
fd = anetTcpNonBlockConnect(server.neterr,c->argv[1]->ptr,
atoi(c->argv[2]->ptr));
if (fd == -1) {
addReplyErrorFormat(c,"Can't connect to target node: %s",
server.neterr);
return;
}
if ((aeWait(fd,AE_WRITABLE,timeout) & AE_WRITABLE) == 0) {
close(fd);
addReplySds(c,sdsnew("-IOERR error or timeout connecting to the client\r\n"));
return;
}
/* Create RESTORE payload and generate the protocol to call the command. */
rioInitWithBuffer(&cmd,sdsempty());
redisAssertWithInfo(c,NULL,rioWriteBulkCount(&cmd,'*',2));
redisAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,"SELECT",6));
redisAssertWithInfo(c,NULL,rioWriteBulkLongLong(&cmd,dbid));
expireat = getExpire(c->db,c->argv[3]);
if (expireat != -1) {
ttl = expireat-mstime();
if (ttl < 1) ttl = 1;
}
redisAssertWithInfo(c,NULL,rioWriteBulkCount(&cmd,'*',4));
redisAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,"RESTORE",7));
redisAssertWithInfo(c,NULL,c->argv[3]->encoding == REDIS_ENCODING_RAW);
redisAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,c->argv[3]->ptr,sdslen(c->argv[3]->ptr)));
redisAssertWithInfo(c,NULL,rioWriteBulkLongLong(&cmd,ttl));
/* Finally the last argument that is the serailized object payload
* in the DUMP format. */
createDumpPayload(&payload,o);
redisAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,payload.io.buffer.ptr,
sdslen(payload.io.buffer.ptr)));
sdsfree(payload.io.buffer.ptr);
/* Tranfer the query to the other node in 64K chunks. */
{
sds buf = cmd.io.buffer.ptr;
size_t pos = 0, towrite;
int nwritten = 0;
while ((towrite = sdslen(buf)-pos) > 0) {
towrite = (towrite > (64*1024) ? (64*1024) : towrite);
nwritten = syncWrite(fd,buf+pos,towrite,timeout);
if (nwritten != (signed)towrite) goto socket_wr_err;
pos += nwritten;
}
}
/* Read back the reply. */
{
char buf1[1024];
char buf2[1024];
/* Read the two replies */
if (syncReadLine(fd, buf1, sizeof(buf1), timeout) <= 0)
goto socket_rd_err;
if (syncReadLine(fd, buf2, sizeof(buf2), timeout) <= 0)
goto socket_rd_err;
if (buf1[0] == '-' || buf2[0] == '-') {
addReplyErrorFormat(c,"Target instance replied with error: %s",
(buf1[0] == '-') ? buf1+1 : buf2+1);
} else {
robj *aux;
dbDelete(c->db,c->argv[3]);
signalModifiedKey(c->db,c->argv[3]);
addReply(c,shared.ok);
server.dirty++;
/* Translate MIGRATE as DEL for replication/AOF. */
aux = createStringObject("DEL",3);
rewriteClientCommandVector(c,2,aux,c->argv[3]);
decrRefCount(aux);
}
}
sdsfree(cmd.io.buffer.ptr);
close(fd);
return;
socket_wr_err:
addReplySds(c,sdsnew("-IOERR error or timeout writing to target instance\r\n"));
sdsfree(cmd.io.buffer.ptr);
close(fd);
return;
socket_rd_err:
addReplySds(c,sdsnew("-IOERR error or timeout reading from target node\r\n"));
sdsfree(cmd.io.buffer.ptr);
close(fd);
return;
}
+2 -4
View File
@@ -1,11 +1,9 @@
#!/bin/sh
GIT_SHA1=`(git show-ref --head --hash=8 2> /dev/null || echo 00000000) | head -n1`
GIT_DIRTY=`git diff --no-ext-diff 2> /dev/null | wc -l`
BUILD_ID=`uname -n`"-"`date +%s`
GIT_DIRTY=`git diff 2> /dev/null | wc -l`
test -f release.h || touch release.h
(cat release.h | grep SHA1 | grep $GIT_SHA1) && \
(cat release.h | grep DIRTY | grep $GIT_DIRTY) && exit 0 # Already up-to-date
(cat release.h | grep DIRTY | grep $GIT_DIRTY) && exit 0 # Already uptodate
echo "#define REDIS_GIT_SHA1 \"$GIT_SHA1\"" > release.h
echo "#define REDIS_GIT_DIRTY \"$GIT_DIRTY\"" >> release.h
echo "#define REDIS_BUILD_ID \"$BUILD_ID\"" >> release.h
touch release.c # Force recompile of release.c
+24 -21
View File
@@ -102,12 +102,14 @@ void discardCommand(redisClient *c) {
}
/* Send a MULTI command to all the slaves and AOF file. Check the execCommand
* implementation for more information. */
void execCommandPropagateMulti(redisClient *c) {
* implememntation for more information. */
void execCommandReplicateMulti(redisClient *c) {
robj *multistring = createStringObject("MULTI",5);
propagate(server.multiCommand,c->db->id,&multistring,1,
REDIS_PROPAGATE_AOF|REDIS_PROPAGATE_REPL);
if (server.aof_state != REDIS_AOF_OFF)
feedAppendOnlyFile(server.multiCommand,c->db->id,&multistring,1);
if (listLength(server.slaves))
replicationFeedSlaves(server.slaves,c->db->id,&multistring,1);
decrRefCount(multistring);
}
@@ -116,7 +118,6 @@ void execCommand(redisClient *c) {
robj **orig_argv;
int orig_argc;
struct redisCommand *orig_cmd;
int must_propagate = 0; /* Need to propagate MULTI/EXEC to AOF / slaves? */
if (!(c->flags & REDIS_MULTI)) {
addReplyError(c,"EXEC without MULTI");
@@ -132,10 +133,19 @@ void execCommand(redisClient *c) {
if (c->flags & (REDIS_DIRTY_CAS|REDIS_DIRTY_EXEC)) {
addReply(c, c->flags & REDIS_DIRTY_EXEC ? shared.execaborterr :
shared.nullmultibulk);
discardTransaction(c);
freeClientMultiState(c);
initClientMultiState(c);
c->flags &= ~(REDIS_MULTI|REDIS_DIRTY_CAS|REDIS_DIRTY_EXEC);
unwatchAllKeys(c);
goto handle_monitor;
}
/* Replicate a MULTI request now that we are sure the block is executed.
* This way we'll deliver the MULTI/..../EXEC block as a whole and
* both the AOF and the replication link will have the same consistency
* and atomicity guarantees. */
execCommandReplicateMulti(c);
/* Exec all the queued commands */
unwatchAllKeys(c); /* Unwatch ASAP otherwise we'll waste CPU cycles */
orig_argv = c->argv;
@@ -146,16 +156,6 @@ void execCommand(redisClient *c) {
c->argc = c->mstate.commands[j].argc;
c->argv = c->mstate.commands[j].argv;
c->cmd = c->mstate.commands[j].cmd;
/* Propagate a MULTI request once we encounter the first write op.
* This way we'll deliver the MULTI/..../EXEC block as a whole and
* both the AOF and the replication link will have the same consistency
* and atomicity guarantees. */
if (!must_propagate && !(c->cmd->flags & REDIS_CMD_READONLY)) {
execCommandPropagateMulti(c);
must_propagate = 1;
}
call(c,REDIS_CALL_FULL);
/* Commands may alter argc/argv, restore mstate. */
@@ -166,10 +166,13 @@ void execCommand(redisClient *c) {
c->argv = orig_argv;
c->argc = orig_argc;
c->cmd = orig_cmd;
discardTransaction(c);
/* Make sure the EXEC command will be propagated as well if MULTI
* was already propagated. */
if (must_propagate) server.dirty++;
freeClientMultiState(c);
initClientMultiState(c);
c->flags &= ~(REDIS_MULTI|REDIS_DIRTY_CAS|REDIS_DIRTY_EXEC);
/* Make sure the EXEC command is always replicated / AOF, since we
* always send the MULTI command (we can't know beforehand if the
* next operations will contain at least a modification to the DB). */
server.dirty++;
handle_monitor:
/* Send EXEC to clients waiting data from MONITOR. We do it here
@@ -220,7 +223,7 @@ void watchForKey(redisClient *c, robj *key) {
incrRefCount(key);
}
listAddNodeTail(clients,c);
/* Add the new key to the list of keys watched by this client */
/* Add the new key to the lits of keys watched by this client */
wk = zmalloc(sizeof(*wk));
wk->key = key;
wk->db = c->db;
+109 -358
View File
@@ -29,7 +29,6 @@
#include "redis.h"
#include <sys/uio.h>
#include <math.h>
static void setProtocolError(redisClient *c, int pos);
@@ -41,17 +40,6 @@ size_t zmalloc_size_sds(sds s) {
return zmalloc_size(s-sizeof(struct sdshdr));
}
/* Return the amount of memory used by the sds string at object->ptr
* for a string object. */
size_t getStringObjectSdsUsedMemory(robj *o) {
redisAssertWithInfo(NULL,o,o->type == REDIS_STRING);
switch(o->encoding) {
case REDIS_ENCODING_RAW: return zmalloc_size_sds(o->ptr);
case REDIS_ENCODING_EMBSTR: return sdslen(o->ptr);
default: return 0; /* Just integer encoding for now. */
}
}
void *dupClientReplyValue(void *o) {
incrRefCount((robj*)o);
return o;
@@ -70,9 +58,7 @@ redisClient *createClient(int fd) {
* contexts (for instance a Lua script) we need a non connected client. */
if (fd != -1) {
anetNonBlock(NULL,fd);
anetEnableTcpNoDelay(NULL,fd);
if (server.tcpkeepalive)
anetKeepAlive(NULL,fd,server.tcpkeepalive);
anetTcpNoDelay(NULL,fd);
if (aeCreateFileEvent(server.el,fd,AE_READABLE,
readQueryFromClient, c) == AE_ERR)
{
@@ -84,7 +70,6 @@ redisClient *createClient(int fd) {
selectDb(c,0);
c->fd = fd;
c->name = NULL;
c->bufpos = 0;
c->querybuf = sdsempty();
c->querybuf_peak = 0;
@@ -99,26 +84,22 @@ redisClient *createClient(int fd) {
c->ctime = c->lastinteraction = server.unixtime;
c->authenticated = 0;
c->replstate = REDIS_REPL_NONE;
c->reploff = 0;
c->repl_ack_off = 0;
c->repl_ack_time = 0;
c->slave_listening_port = 0;
c->reply = listCreate();
c->reply_bytes = 0;
c->obuf_soft_limit_reached_time = 0;
listSetFreeMethod(c->reply,decrRefCountVoid);
listSetFreeMethod(c->reply,decrRefCount);
listSetDupMethod(c->reply,dupClientReplyValue);
c->btype = REDIS_BLOCKED_NONE;
c->bpop.keys = NULL;
c->bpop.count = 0;
c->bpop.timeout = 0;
c->bpop.keys = dictCreate(&setDictType,NULL);
c->bpop.target = NULL;
c->bpop.numreplicas = 0;
c->bpop.reploffset = 0;
c->woff = 0;
c->io_keys = listCreate();
c->watched_keys = listCreate();
listSetFreeMethod(c->io_keys,decrRefCount);
c->pubsub_channels = dictCreate(&setDictType,NULL);
c->pubsub_patterns = listCreate();
listSetFreeMethod(c->pubsub_patterns,decrRefCountVoid);
listSetFreeMethod(c->pubsub_patterns,decrRefCount);
listSetMatchMethod(c->pubsub_patterns,listMatchObjects);
if (fd != -1) listAddNodeTail(server.clients,c);
initClientMultiState(c);
@@ -132,17 +113,15 @@ redisClient *createClient(int fd) {
* returns REDIS_OK, and make sure to install the write handler in our event
* loop so that when the socket is writable new data gets written.
*
* If the client should not receive new data, because it is a fake client,
* a master, a slave not yet online, or because the setup of the write handler
* failed, the function returns REDIS_ERR.
* If the client should not receive new data, because it is a fake client
* or a slave, or because the setup of the write handler failed, the function
* returns REDIS_ERR.
*
* Typically gets called every time a reply is built, before adding more
* data to the clients output buffers. If the function returns REDIS_ERR no
* data should be appended to the output buffers. */
int prepareClientToWrite(redisClient *c) {
if (c->flags & REDIS_LUA_CLIENT) return REDIS_OK;
if ((c->flags & REDIS_MASTER) &&
!(c->flags & REDIS_MASTER_FORCE_REPLY)) return REDIS_ERR;
if (c->fd <= 0) return REDIS_ERR; /* Fake client */
if (c->bufpos == 0 && listLength(c->reply) == 0 &&
(c->replstate == REDIS_REPL_NONE ||
@@ -197,13 +176,12 @@ void _addReplyObjectToList(redisClient *c, robj *o) {
if (listLength(c->reply) == 0) {
incrRefCount(o);
listAddNodeTail(c->reply,o);
c->reply_bytes += getStringObjectSdsUsedMemory(o);
c->reply_bytes += zmalloc_size_sds(o->ptr);
} else {
tail = listNodeValue(listLast(c->reply));
/* Append to this object when possible. */
if (tail->ptr != NULL &&
tail->encoding == REDIS_ENCODING_RAW &&
sdslen(tail->ptr)+sdslen(o->ptr) <= REDIS_REPLY_CHUNK_BYTES)
{
c->reply_bytes -= zmalloc_size_sds(tail->ptr);
@@ -213,7 +191,7 @@ void _addReplyObjectToList(redisClient *c, robj *o) {
} else {
incrRefCount(o);
listAddNodeTail(c->reply,o);
c->reply_bytes += getStringObjectSdsUsedMemory(o);
c->reply_bytes += zmalloc_size_sds(o->ptr);
}
}
asyncCloseClientOnOutputBufferLimitReached(c);
@@ -236,7 +214,7 @@ void _addReplySdsToList(redisClient *c, sds s) {
tail = listNodeValue(listLast(c->reply));
/* Append to this object when possible. */
if (tail->ptr != NULL && tail->encoding == REDIS_ENCODING_RAW &&
if (tail->ptr != NULL &&
sdslen(tail->ptr)+sdslen(s) <= REDIS_REPLY_CHUNK_BYTES)
{
c->reply_bytes -= zmalloc_size_sds(tail->ptr);
@@ -261,12 +239,12 @@ void _addReplyStringToList(redisClient *c, char *s, size_t len) {
robj *o = createStringObject(s,len);
listAddNodeTail(c->reply,o);
c->reply_bytes += getStringObjectSdsUsedMemory(o);
c->reply_bytes += zmalloc_size_sds(o->ptr);
} else {
tail = listNodeValue(listLast(c->reply));
/* Append to this object when possible. */
if (tail->ptr != NULL && tail->encoding == REDIS_ENCODING_RAW &&
if (tail->ptr != NULL &&
sdslen(tail->ptr)+len <= REDIS_REPLY_CHUNK_BYTES)
{
c->reply_bytes -= zmalloc_size_sds(tail->ptr);
@@ -277,7 +255,7 @@ void _addReplyStringToList(redisClient *c, char *s, size_t len) {
robj *o = createStringObject(s,len);
listAddNodeTail(c->reply,o);
c->reply_bytes += getStringObjectSdsUsedMemory(o);
c->reply_bytes += zmalloc_size_sds(o->ptr);
}
}
asyncCloseClientOnOutputBufferLimitReached(c);
@@ -298,7 +276,7 @@ void addReply(redisClient *c, robj *obj) {
* If the encoding is RAW and there is room in the static buffer
* we'll be able to send the object to the client without
* messing with its page. */
if (sdsEncodedObject(obj)) {
if (obj->encoding == REDIS_ENCODING_RAW) {
if (_addReplyToBuffer(c,obj->ptr,sdslen(obj->ptr)) != REDIS_OK)
_addReplyObjectToList(c,obj);
} else if (obj->encoding == REDIS_ENCODING_INT) {
@@ -319,11 +297,6 @@ void addReply(redisClient *c, robj *obj) {
if (_addReplyToBuffer(c,obj->ptr,sdslen(obj->ptr)) != REDIS_OK)
_addReplyObjectToList(c,obj);
decrRefCount(obj);
} else if (obj->encoding == REDIS_ENCODING_LZF) {
obj = getDecodedObject(obj);
if (_addReplyToBuffer(c,obj->ptr,sdslen(obj->ptr)) != REDIS_OK)
_addReplyObjectToList(c,obj);
decrRefCount(obj);
} else {
redisPanic("Wrong obj->encoding in addReply()");
}
@@ -405,7 +378,7 @@ void *addDeferredMultiBulkLength(redisClient *c) {
return listLast(c->reply);
}
/* Populate the length object and try gluing it to the next chunk. */
/* Populate the length object and try glueing it to the next chunk. */
void setDeferredMultiBulkLength(redisClient *c, void *node, long length) {
listNode *ln = (listNode*)node;
robj *len, *next;
@@ -415,7 +388,6 @@ void setDeferredMultiBulkLength(redisClient *c, void *node, long length) {
len = listNodeValue(ln);
len->ptr = sdscatprintf(sdsempty(),"*%ld\r\n",length);
len->encoding = REDIS_ENCODING_RAW; /* in case it was an EMBSTR. */
c->reply_bytes += zmalloc_size_sds(len->ptr);
if (ln->next != NULL) {
next = listNodeValue(ln->next);
@@ -423,7 +395,7 @@ void setDeferredMultiBulkLength(redisClient *c, void *node, long length) {
/* Only glue when the next node is non-NULL (an sds in this case) */
if (next->ptr != NULL) {
c->reply_bytes -= zmalloc_size_sds(len->ptr);
c->reply_bytes -= getStringObjectSdsUsedMemory(next);
c->reply_bytes -= zmalloc_size_sds(next->ptr);
len->ptr = sdscatlen(len->ptr,next->ptr,sdslen(next->ptr));
c->reply_bytes += zmalloc_size_sds(len->ptr);
listDelNode(c->reply,ln->next);
@@ -432,19 +404,13 @@ void setDeferredMultiBulkLength(redisClient *c, void *node, long length) {
asyncCloseClientOnOutputBufferLimitReached(c);
}
/* Add a double as a bulk reply */
/* Add a duble as a bulk reply */
void addReplyDouble(redisClient *c, double d) {
char dbuf[128], sbuf[128];
int dlen, slen;
if (isinf(d)) {
/* Libc in odd systems (Hi Solaris!) will format infinite in a
* different way, so better to handle it in an explicit way. */
addReplyBulkCString(c, d > 0 ? "inf" : "-inf");
} else {
dlen = snprintf(dbuf,sizeof(dbuf),"%.17g",d);
slen = snprintf(sbuf,sizeof(sbuf),"$%d\r\n%s\r\n",dlen,dbuf);
addReplyString(c,sbuf,slen);
}
dlen = snprintf(dbuf,sizeof(dbuf),"%.17g",d);
slen = snprintf(sbuf,sizeof(sbuf),"$%d\r\n%s\r\n",dlen,dbuf);
addReplyString(c,sbuf,slen);
}
/* Add a long long as integer reply or bulk len / multi bulk count.
@@ -481,19 +447,16 @@ void addReplyLongLong(redisClient *c, long long ll) {
}
void addReplyMultiBulkLen(redisClient *c, long length) {
if (length < REDIS_SHARED_BULKHDR_LEN)
addReply(c,shared.mbulkhdr[length]);
else
addReplyLongLongWithPrefix(c,length,'*');
addReplyLongLongWithPrefix(c,length,'*');
}
/* Create the length prefix of a bulk reply, example: $2234 */
void addReplyBulkLen(redisClient *c, robj *obj) {
size_t len;
if (sdsEncodedObject(obj)) {
if (obj->encoding == REDIS_ENCODING_RAW) {
len = sdslen(obj->ptr);
} else if (obj->encoding == REDIS_ENCODING_INT) {
} else {
long n = (long)obj->ptr;
/* Compute how many bytes will take this integer as a radix 10 string */
@@ -505,15 +468,8 @@ void addReplyBulkLen(redisClient *c, robj *obj) {
while((n = n/10) != 0) {
len++;
}
} else {
/* LZF and others not handled explicitly. */
len = stringObjectLen(obj);
}
if (len < REDIS_SHARED_BULKHDR_LEN)
addReply(c,shared.bulkhdr[len]);
else
addReplyLongLongWithPrefix(c,len,'$');
addReplyLongLongWithPrefix(c,len,'$');
}
/* Add a Redis Object as a bulk reply */
@@ -562,15 +518,13 @@ void copyClientOutputBuffer(redisClient *dst, redisClient *src) {
static void acceptCommonHandler(int fd, int flags) {
redisClient *c;
if ((c = createClient(fd)) == NULL) {
redisLog(REDIS_WARNING,
"Error registering fd event for the new client: %s (fd=%d)",
strerror(errno),fd);
redisLog(REDIS_WARNING,"Error allocating resources for the client");
close(fd); /* May be already closed, just ignore errors */
return;
}
/* If maxclient directive is set and this is one client more... close the
* connection. Note that we create the client instead to check before
* for this condition, since now the socket is already set in non-blocking
* for this condition, since now the socket is already set in nonblocking
* mode and we can send an error for free using the Kernel I/O */
if (listLength(server.clients) > server.maxclients) {
char *err = "-ERR max number of clients reached\r\n";
@@ -589,13 +543,13 @@ static void acceptCommonHandler(int fd, int flags) {
void acceptTcpHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
int cport, cfd;
char cip[REDIS_IP_STR_LEN];
char cip[128];
REDIS_NOTUSED(el);
REDIS_NOTUSED(mask);
REDIS_NOTUSED(privdata);
cfd = anetTcpAccept(server.neterr, fd, cip, sizeof(cip), &cport);
if (cfd == ANET_ERR) {
cfd = anetTcpAccept(server.neterr, fd, cip, &cport);
if (cfd == AE_ERR) {
redisLog(REDIS_WARNING,"Accepting client connection: %s", server.neterr);
return;
}
@@ -637,118 +591,71 @@ void disconnectSlaves(void) {
}
}
/* This function is called when the slave lose the connection with the
* master into an unexpected way. */
void replicationHandleMasterDisconnection(void) {
server.master = NULL;
server.repl_state = REDIS_REPL_CONNECT;
server.repl_down_since = server.unixtime;
/* We lost connection with our master, force our slaves to resync
* with us as well to load the new data set.
*
* If server.masterhost is NULL the user called SLAVEOF NO ONE so
* slave resync is not needed. */
if (server.masterhost != NULL) disconnectSlaves();
}
void freeClient(redisClient *c) {
listNode *ln;
/* If this is marked as current client unset it */
if (server.current_client == c) server.current_client = NULL;
/* If it is our master that's beging disconnected we should make sure
* to cache the state to try a partial resynchronization later.
*
* Note that before doing this we make sure that the client is not in
* some unexpected state, by checking its flags. */
if (server.master && c->flags & REDIS_MASTER) {
redisLog(REDIS_WARNING,"Connection with master lost.");
if (!(c->flags & (REDIS_CLOSE_AFTER_REPLY|
REDIS_CLOSE_ASAP|
REDIS_BLOCKED|
REDIS_UNBLOCKED)))
{
replicationCacheMaster(c);
return;
}
}
/* Log link disconnection with slave */
if ((c->flags & REDIS_SLAVE) && !(c->flags & REDIS_MONITOR)) {
char ip[REDIS_IP_STR_LEN];
if (anetPeerToString(c->fd,ip,sizeof(ip),NULL) != -1) {
redisLog(REDIS_WARNING,"Connection with slave %s:%d lost.",
ip, c->slave_listening_port);
}
}
/* Free the query buffer */
/* Note that if the client we are freeing is blocked into a blocking
* call, we have to set querybuf to NULL *before* to call
* unblockClientWaitingData() to avoid processInputBuffer() will get
* called. Also it is important to remove the file events after
* this, because this call adds the READABLE event. */
sdsfree(c->querybuf);
c->querybuf = NULL;
/* Deallocate structures used to block on blocking ops. */
if (c->flags & REDIS_BLOCKED) unblockClient(c);
dictRelease(c->bpop.keys);
if (c->flags & REDIS_BLOCKED)
unblockClientWaitingData(c);
/* UNWATCH all the keys */
unwatchAllKeys(c);
listRelease(c->watched_keys);
/* Unsubscribe from all the pubsub channels */
pubsubUnsubscribeAllChannels(c,0);
pubsubUnsubscribeAllPatterns(c,0);
dictRelease(c->pubsub_channels);
listRelease(c->pubsub_patterns);
/* Close socket, unregister events, and remove list of replies and
* accumulated arguments. */
if (c->fd != -1) {
aeDeleteFileEvent(server.el,c->fd,AE_READABLE);
aeDeleteFileEvent(server.el,c->fd,AE_WRITABLE);
close(c->fd);
}
/* Obvious cleanup */
aeDeleteFileEvent(server.el,c->fd,AE_READABLE);
aeDeleteFileEvent(server.el,c->fd,AE_WRITABLE);
listRelease(c->reply);
freeClientArgv(c);
close(c->fd);
/* Remove from the list of clients */
if (c->fd != -1) {
ln = listSearchKey(server.clients,c);
redisAssert(ln != NULL);
listDelNode(server.clients,ln);
}
ln = listSearchKey(server.clients,c);
redisAssert(ln != NULL);
listDelNode(server.clients,ln);
/* When client was just unblocked because of a blocking operation,
* remove it from the list of unblocked clients. */
* remove it from the list with unblocked clients. */
if (c->flags & REDIS_UNBLOCKED) {
ln = listSearchKey(server.unblocked_clients,c);
redisAssert(ln != NULL);
listDelNode(server.unblocked_clients,ln);
}
/* Master/slave cleanup Case 1:
* we lost the connection with a slave. */
listRelease(c->io_keys);
/* Master/slave cleanup.
* Case 1: we lost the connection with a slave. */
if (c->flags & REDIS_SLAVE) {
if (c->replstate == REDIS_REPL_SEND_BULK) {
if (c->repldbfd != -1) close(c->repldbfd);
if (c->replpreamble) sdsfree(c->replpreamble);
}
if (c->replstate == REDIS_REPL_SEND_BULK && c->repldbfd != -1)
close(c->repldbfd);
list *l = (c->flags & REDIS_MONITOR) ? server.monitors : server.slaves;
ln = listSearchKey(l,c);
redisAssert(ln != NULL);
listDelNode(l,ln);
/* We need to remember the time when we started to have zero
* attached slaves, as after some time we'll free the replication
* backlog. */
if (c->flags & REDIS_SLAVE && listLength(server.slaves) == 0)
server.repl_no_slaves_since = server.unixtime;
refreshGoodSlavesCount();
}
/* Master/slave cleanup Case 2:
* we lost the connection with the master. */
if (c->flags & REDIS_MASTER) replicationHandleMasterDisconnection();
/* Case 2: we lost the connection with the master. */
if (c->flags & REDIS_MASTER) {
server.master = NULL;
server.repl_state = REDIS_REPL_CONNECT;
server.repl_down_since = server.unixtime;
/* We lost connection with our master, force our slaves to resync
* with us as well to load the new data set.
*
* If server.masterhost is NULL the user called SLAVEOF NO ONE so
* slave resync is not needed. */
if (server.masterhost != NULL) disconnectSlaves();
}
/* If this client was scheduled for async freeing we need to remove it
* from the queue. */
@@ -758,9 +665,7 @@ void freeClient(redisClient *c) {
listDelNode(server.clients_to_close,ln);
}
/* Release other dynamically allocated client structure fields,
* and finally release the client structure itself. */
if (c->name) decrRefCount(c->name);
/* Release memory */
zfree(c->argv);
freeClientMultiState(c);
zfree(c);
@@ -797,8 +702,13 @@ void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask) {
while(c->bufpos > 0 || listLength(c->reply)) {
if (c->bufpos > 0) {
nwritten = write(fd,c->buf+c->sentlen,c->bufpos-c->sentlen);
if (nwritten <= 0) break;
if (c->flags & REDIS_MASTER) {
/* Don't reply to a master */
nwritten = c->bufpos - c->sentlen;
} else {
nwritten = write(fd,c->buf+c->sentlen,c->bufpos-c->sentlen);
if (nwritten <= 0) break;
}
c->sentlen += nwritten;
totwritten += nwritten;
@@ -811,16 +721,20 @@ void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask) {
} else {
o = listNodeValue(listFirst(c->reply));
objlen = sdslen(o->ptr);
objmem = getStringObjectSdsUsedMemory(o);
objmem = zmalloc_size_sds(o->ptr);
if (objlen == 0) {
listDelNode(c->reply,listFirst(c->reply));
c->reply_bytes -= objmem;
continue;
}
nwritten = write(fd, ((char*)o->ptr)+c->sentlen,objlen-c->sentlen);
if (nwritten <= 0) break;
if (c->flags & REDIS_MASTER) {
/* Don't reply to a master */
nwritten = objlen - c->sentlen;
} else {
nwritten = write(fd, ((char*)o->ptr)+c->sentlen,objlen-c->sentlen);
if (nwritten <= 0) break;
}
c->sentlen += nwritten;
totwritten += nwritten;
@@ -853,13 +767,7 @@ void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask) {
return;
}
}
if (totwritten > 0) {
/* For clients representing masters we don't count sending data
* as an interaction, since we always send REPLCONF ACK commands
* that take some time to just fill the socket output buffer.
* We just rely on data / pings received for timeout detection. */
if (!(c->flags & REDIS_MASTER)) c->lastinteraction = server.unixtime;
}
if (totwritten > 0) c->lastinteraction = server.unixtime;
if (c->bufpos == 0 && listLength(c->reply) == 0) {
c->sentlen = 0;
aeDeleteFileEvent(server.el,c->fd,AE_WRITABLE);
@@ -871,27 +779,20 @@ void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask) {
/* resetClient prepare the client to process the next command */
void resetClient(redisClient *c) {
redisCommandProc *prevcmd = c->cmd ? c->cmd->proc : NULL;
freeClientArgv(c);
c->reqtype = 0;
c->multibulklen = 0;
c->bulklen = -1;
/* We clear the ASKING flag as well if we are not inside a MULTI, and
* if what we just executed is not the ASKING command itself. */
if (!(c->flags & REDIS_MULTI) && prevcmd != askingCommand)
c->flags &= (~REDIS_ASKING);
/* We clear the ASKING flag as well if we are not inside a MULTI. */
if (!(c->flags & REDIS_MULTI)) c->flags &= (~REDIS_ASKING);
}
int processInlineBuffer(redisClient *c) {
char *newline;
char *newline = strstr(c->querybuf,"\r\n");
int argc, j;
sds *argv, aux;
sds *argv;
size_t querylen;
/* Search for end of line */
newline = strchr(c->querybuf,'\n');
/* Nothing to do without a \r\n */
if (newline == NULL) {
if (sdslen(c->querybuf) > REDIS_INLINE_MAX_SIZE) {
@@ -901,29 +802,12 @@ int processInlineBuffer(redisClient *c) {
return REDIS_ERR;
}
/* Handle the \r\n case. */
if (newline && newline != c->querybuf && *(newline-1) == '\r')
newline--;
/* Split the input buffer up to the \r\n */
querylen = newline-(c->querybuf);
aux = sdsnewlen(c->querybuf,querylen);
argv = sdssplitargs(aux,&argc);
sdsfree(aux);
if (argv == NULL) {
addReplyError(c,"Protocol error: unbalanced quotes in request");
setProtocolError(c,0);
return REDIS_ERR;
}
/* Newline from slaves can be used to refresh the last ACK time.
* This is useful for a slave to ping back while loading a big
* RDB file. */
if (querylen == 0 && c->flags & REDIS_SLAVE)
c->repl_ack_time = server.unixtime;
argv = sdssplitlen(c->querybuf,querylen," ",1,&argc);
/* Leave data after the first line of the query in the buffer */
sdsrange(c->querybuf,querylen+2,-1);
c->querybuf = sdsrange(c->querybuf,querylen+2,-1);
/* Setup argv array on client structure */
if (c->argv) zfree(c->argv);
@@ -952,7 +836,7 @@ static void setProtocolError(redisClient *c, int pos) {
sdsfree(client);
}
c->flags |= REDIS_CLOSE_AFTER_REPLY;
sdsrange(c->querybuf,pos,-1);
c->querybuf = sdsrange(c->querybuf,pos,-1);
}
int processMultibulkBuffer(redisClient *c) {
@@ -990,7 +874,7 @@ int processMultibulkBuffer(redisClient *c) {
pos = (newline-c->querybuf)+2;
if (ll <= 0) {
sdsrange(c->querybuf,pos,-1);
c->querybuf = sdsrange(c->querybuf,pos,-1);
return REDIS_OK;
}
@@ -1035,19 +919,15 @@ int processMultibulkBuffer(redisClient *c) {
pos += newline-(c->querybuf+pos)+2;
if (ll >= REDIS_MBULK_BIG_ARG) {
size_t qblen;
/* If we are going to read a large object from network
* try to make it likely that it will start at c->querybuf
* boundary so that we can optimize object creation
* boundary so that we can optimized object creation
* avoiding a large copy of data. */
sdsrange(c->querybuf,pos,-1);
c->querybuf = sdsrange(c->querybuf,pos,-1);
pos = 0;
qblen = sdslen(c->querybuf);
/* Hint the sds library about the amount of bytes this string is
* going to contain. */
if (qblen < ll+2)
c->querybuf = sdsMakeRoomFor(c->querybuf,ll+2-qblen);
c->querybuf = sdsMakeRoomFor(c->querybuf,ll+2);
}
c->bulklen = ll;
}
@@ -1057,7 +937,7 @@ int processMultibulkBuffer(redisClient *c) {
/* Not enough data (+2 == trailing \r\n) */
break;
} else {
/* Optimization: if the buffer contains JUST our bulk element
/* Optimization: if the buffer contanins JUST our bulk element
* instead of creating a new object by *copying* the sds we
* just use the current sds string. */
if (pos == 0 &&
@@ -1082,7 +962,7 @@ int processMultibulkBuffer(redisClient *c) {
}
/* Trim to pos */
if (pos) sdsrange(c->querybuf,pos,-1);
if (pos) c->querybuf = sdsrange(c->querybuf,pos,-1);
/* We're done when c->multibulk == 0 */
if (c->multibulklen == 0) return REDIS_OK;
@@ -1094,9 +974,6 @@ int processMultibulkBuffer(redisClient *c) {
void processInputBuffer(redisClient *c) {
/* Keep processing while there is something in the input buffer */
while(sdslen(c->querybuf)) {
/* Return if clients are paused. */
if (!(c->flags & REDIS_SLAVE) && clientsArePaused()) return;
/* Immediately abort if the client is in the middle of something. */
if (c->flags & REDIS_BLOCKED) return;
@@ -1176,7 +1053,6 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
if (nread) {
sdsIncrLen(c->querybuf,nread);
c->lastinteraction = server.unixtime;
if (c->flags & REDIS_MASTER) c->reploff += nread;
} else {
server.current_client = NULL;
return;
@@ -1213,52 +1089,14 @@ void getClientsMaxBuffers(unsigned long *longest_output_list,
*biggest_input_buffer = bib;
}
/* This is a helper function for getClientPeerId().
* It writes the specified ip/port to "peerid" as a null termiated string
* in the form ip:port if ip does not contain ":" itself, otherwise
* [ip]:port format is used (for IPv6 addresses basically). */
void formatPeerId(char *peerid, size_t peerid_len, char *ip, int port) {
if (strchr(ip,':'))
snprintf(peerid,peerid_len,"[%s]:%d",ip,port);
else
snprintf(peerid,peerid_len,"%s:%d",ip,port);
}
/* A Redis "Peer ID" is a colon separated ip:port pair.
* For IPv4 it's in the form x.y.z.k:pork, example: "127.0.0.1:1234".
* For IPv6 addresses we use [] around the IP part, like in "[::1]:1234".
* For Unix socekts we use path:0, like in "/tmp/redis:0".
*
* A Peer ID always fits inside a buffer of REDIS_PEER_ID_LEN bytes, including
* the null term.
*
* The function returns REDIS_OK on succcess, and REDIS_ERR on failure.
*
* On failure the function still populates 'peerid' with the "?:0" string
* in case you want to relax error checking or need to display something
* anyway (see anetPeerToString implementation for more info). */
int getClientPeerId(redisClient *client, char *peerid, size_t peerid_len) {
char ip[REDIS_IP_STR_LEN];
int port;
if (client->flags & REDIS_UNIX_SOCKET) {
/* Unix socket client. */
snprintf(peerid,peerid_len,"%s:0",server.unixsocket);
return REDIS_OK;
} else {
/* TCP client. */
int retval = anetPeerToString(client->fd,ip,sizeof(ip),&port);
formatPeerId(peerid,peerid_len,ip,port);
return (retval == -1) ? REDIS_ERR : REDIS_OK;
}
}
/* Turn a Redis client into an sds string representing its state. */
sds getClientInfoString(redisClient *client) {
char peerid[REDIS_PEER_ID_LEN], flags[16], events[3], *p;
char ip[32], flags[16], events[3], *p;
int port = 0; /* initialized to zero for the unix socket case. */
int emask;
getClientPeerId(client,peerid,sizeof(peerid));
if (!(client->flags & REDIS_UNIX_SOCKET))
anetPeerToString(client->fd,ip,&port);
p = flags;
if (client->flags & REDIS_SLAVE) {
if (client->flags & REDIS_MONITOR)
@@ -1274,7 +1112,6 @@ sds getClientInfoString(redisClient *client) {
if (client->flags & REDIS_UNBLOCKED) *p++ = 'u';
if (client->flags & REDIS_CLOSE_ASAP) *p++ = 'A';
if (client->flags & REDIS_UNIX_SOCKET) *p++ = 'U';
if (client->flags & REDIS_READONLY) *p++ = 'r';
if (p == flags) *p++ = 'N';
*p++ = '\0';
@@ -1284,10 +1121,9 @@ sds getClientInfoString(redisClient *client) {
if (emask & AE_WRITABLE) *p++ = 'w';
*p = '\0';
return sdscatprintf(sdsempty(),
"addr=%s fd=%d name=%s age=%ld idle=%ld flags=%s db=%d sub=%d psub=%d multi=%d qbuf=%lu qbuf-free=%lu obl=%lu oll=%lu omem=%lu events=%s cmd=%s",
peerid,
client->fd,
client->name ? (char*)client->name->ptr : "",
"addr=%s:%d fd=%d age=%ld idle=%ld flags=%s db=%d sub=%d psub=%d multi=%d qbuf=%lu qbuf-free=%lu obl=%lu oll=%lu omem=%lu events=%s cmd=%s",
(client->flags & REDIS_UNIX_SOCKET) ? server.unixsocket : ip,
port,client->fd,
(long)(server.unixtime - client->ctime),
(long)(server.unixtime - client->lastinteraction),
flags,
@@ -1335,12 +1171,13 @@ void clientCommand(redisClient *c) {
} else if (!strcasecmp(c->argv[1]->ptr,"kill") && c->argc == 3) {
listRewind(server.clients,&li);
while ((ln = listNext(&li)) != NULL) {
char peerid[REDIS_PEER_ID_LEN];
char ip[32], addr[64];
int port;
client = listNodeValue(ln);
if (getClientPeerId(client,peerid,sizeof(peerid)) == REDIS_ERR)
continue;
if (strcmp(peerid,c->argv[2]->ptr) == 0) {
if (anetPeerToString(client->fd,ip,&port) == -1) continue;
snprintf(addr,sizeof(addr),"%s:%d",ip,port);
if (strcmp(addr,c->argv[2]->ptr) == 0) {
addReply(c,shared.ok);
if (c == client) {
client->flags |= REDIS_CLOSE_AFTER_REPLY;
@@ -1351,48 +1188,8 @@ void clientCommand(redisClient *c) {
}
}
addReplyError(c,"No such client");
} else if (!strcasecmp(c->argv[1]->ptr,"setname") && c->argc == 3) {
int j, len = sdslen(c->argv[2]->ptr);
char *p = c->argv[2]->ptr;
/* Setting the client name to an empty string actually removes
* the current name. */
if (len == 0) {
if (c->name) decrRefCount(c->name);
c->name = NULL;
addReply(c,shared.ok);
return;
}
/* Otherwise check if the charset is ok. We need to do this otherwise
* CLIENT LIST format will break. You should always be able to
* split by space to get the different fields. */
for (j = 0; j < len; j++) {
if (p[j] < '!' || p[j] > '~') { /* ASCII is assumed. */
addReplyError(c,
"Client names cannot contain spaces, "
"newlines or special characters.");
return;
}
}
if (c->name) decrRefCount(c->name);
c->name = c->argv[2];
incrRefCount(c->name);
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"getname") && c->argc == 2) {
if (c->name)
addReplyBulk(c,c->name);
else
addReply(c,shared.nullbulk);
} else if (!strcasecmp(c->argv[1]->ptr,"pause") && c->argc == 3) {
long long duration;
if (getTimeoutFromObjectOrReply(c,c->argv[2],&duration,UNIT_MILLISECONDS)
!= REDIS_OK) return;
pauseClients(duration);
addReply(c,shared.ok);
} else {
addReplyError(c, "Syntax error, try CLIENT (LIST | KILL ip:port | GETNAME | SETNAME connection-name)");
addReplyError(c, "Syntax error, try CLIENT (LIST | KILL ip:port)");
}
}
@@ -1421,7 +1218,7 @@ void rewriteClientCommandVector(redisClient *c, int argc, ...) {
/* Replace argv and argc with our new versions. */
c->argv = argv;
c->argc = argc;
c->cmd = lookupCommandOrOriginal(c->argv[0]->ptr);
c->cmd = lookupCommand(c->argv[0]->ptr);
redisAssertWithInfo(c,NULL,c->cmd != NULL);
va_end(ap);
}
@@ -1439,7 +1236,7 @@ void rewriteClientCommandArgument(redisClient *c, int i, robj *newval) {
/* If this is the command name make sure to fix c->cmd. */
if (i == 0) {
c->cmd = lookupCommandOrOriginal(c->argv[0]->ptr);
c->cmd = lookupCommand(c->argv[0]->ptr);
redisAssertWithInfo(c,NULL,c->cmd != NULL);
}
}
@@ -1573,49 +1370,3 @@ void flushSlavesOutputBuffers(void) {
}
}
}
/* Pause clients up to the specified unixtime (in ms). While clients
* are paused no command is processed from clients, so the data set can't
* change during that time.
*
* However while this function pauses normal and Pub/Sub clients, slaves are
* still served, so this function can be used on server upgrades where it is
* required that slaves process the latest bytes from the replication stream
* before being turned to masters.
*
* This function is also internally used by Redis Cluster for the manual
* failover procedure implemented by CLUSTER FAILOVER.
*
* The function always succeed, even if there is already a pause in progress.
* In such a case, the pause is extended if the duration is more than the
* time left for the previous duration. However if the duration is smaller
* than the time left for the previous pause, no change is made to the
* left duration. */
void pauseClients(mstime_t end) {
if (!server.clients_paused || end > server.clients_pause_end_time)
server.clients_pause_end_time = end;
server.clients_paused = 1;
}
/* Return non-zero if clients are currently paused. As a side effect the
* function checks if the pause time was reached and clear it. */
int clientsArePaused(void) {
if (server.clients_paused && server.clients_pause_end_time < server.mstime) {
listNode *ln;
listIter li;
redisClient *c;
server.clients_paused = 0;
/* Put all the clients in the unblocked clients queue in order to
* force the re-processing of the input buffer if any. */
listRewind(server.clients,&li);
while ((ln = listNext(&li)) != NULL) {
c = listNodeValue(ln);
if (c->flags & REDIS_SLAVE) continue;
listAddNodeTail(server.unblocked_clients,c);
}
}
return server.clients_paused;
}
-130
View File
@@ -1,130 +0,0 @@
/*
* Copyright (c) 2013, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "redis.h"
/* This file implements keyspace events notification via Pub/Sub ad
* described at http://redis.io/topics/keyspace-events. */
/* Turn a string representing notification classes into an integer
* representing notification classes flags xored.
*
* The function returns -1 if the input contains characters not mapping to
* any class. */
int keyspaceEventsStringToFlags(char *classes) {
char *p = classes;
int c, flags = 0;
while((c = *p++) != '\0') {
switch(c) {
case 'A': flags |= REDIS_NOTIFY_ALL; break;
case 'g': flags |= REDIS_NOTIFY_GENERIC; break;
case '$': flags |= REDIS_NOTIFY_STRING; break;
case 'l': flags |= REDIS_NOTIFY_LIST; break;
case 's': flags |= REDIS_NOTIFY_SET; break;
case 'h': flags |= REDIS_NOTIFY_HASH; break;
case 'z': flags |= REDIS_NOTIFY_ZSET; break;
case 'x': flags |= REDIS_NOTIFY_EXPIRED; break;
case 'e': flags |= REDIS_NOTIFY_EVICTED; break;
case 'K': flags |= REDIS_NOTIFY_KEYSPACE; break;
case 'E': flags |= REDIS_NOTIFY_KEYEVENT; break;
default: return -1;
}
}
return flags;
}
/* This function does exactly the revese of the function above: it gets
* as input an integer with the xored flags and returns a string representing
* the selected classes. The string returned is an sds string that needs to
* be released with sdsfree(). */
sds keyspaceEventsFlagsToString(int flags) {
sds res;
res = sdsempty();
if ((flags & REDIS_NOTIFY_ALL) == REDIS_NOTIFY_ALL) {
res = sdscatlen(res,"A",1);
} else {
if (flags & REDIS_NOTIFY_GENERIC) res = sdscatlen(res,"g",1);
if (flags & REDIS_NOTIFY_STRING) res = sdscatlen(res,"$",1);
if (flags & REDIS_NOTIFY_LIST) res = sdscatlen(res,"l",1);
if (flags & REDIS_NOTIFY_SET) res = sdscatlen(res,"s",1);
if (flags & REDIS_NOTIFY_HASH) res = sdscatlen(res,"h",1);
if (flags & REDIS_NOTIFY_ZSET) res = sdscatlen(res,"z",1);
if (flags & REDIS_NOTIFY_EXPIRED) res = sdscatlen(res,"x",1);
if (flags & REDIS_NOTIFY_EVICTED) res = sdscatlen(res,"e",1);
}
if (flags & REDIS_NOTIFY_KEYSPACE) res = sdscatlen(res,"K",1);
if (flags & REDIS_NOTIFY_KEYEVENT) res = sdscatlen(res,"E",1);
return res;
}
/* The API provided to the rest of the Redis core is a simple function:
*
* notifyKeyspaceEvent(char *event, robj *key, int dbid);
*
* 'event' is a C string representing the event name.
* 'key' is a Redis object representing the key name.
* 'dbid' is the database ID where the key lives. */
void notifyKeyspaceEvent(int type, char *event, robj *key, int dbid) {
sds chan;
robj *chanobj, *eventobj;
int len = -1;
char buf[24];
/* If notifications for this class of events are off, return ASAP. */
if (!(server.notify_keyspace_events & type)) return;
eventobj = createStringObject(event,strlen(event));
/* __keyspace@<db>__:<key> <event> notifications. */
if (server.notify_keyspace_events & REDIS_NOTIFY_KEYSPACE) {
chan = sdsnewlen("__keyspace@",11);
len = ll2string(buf,sizeof(buf),dbid);
chan = sdscatlen(chan, buf, len);
chan = sdscatlen(chan, "__:", 3);
chan = sdscatsds(chan, key->ptr);
chanobj = createObject(REDIS_STRING, chan);
pubsubPublishMessage(chanobj, eventobj);
decrRefCount(chanobj);
}
/* __keyevente@<db>__:<event> <key> notifications. */
if (server.notify_keyspace_events & REDIS_NOTIFY_KEYEVENT) {
chan = sdsnewlen("__keyevent@",11);
if (len == -1) len = ll2string(buf,sizeof(buf),dbid);
chan = sdscatlen(chan, buf, len);
chan = sdscatlen(chan, "__:", 3);
chan = sdscatsds(chan, eventobj->ptr);
chanobj = createObject(REDIS_STRING, chan);
pubsubPublishMessage(chanobj, key);
decrRefCount(chanobj);
}
decrRefCount(eventobj);
}
+72 -308
View File
@@ -29,7 +29,6 @@
*/
#include "redis.h"
#include "lzf.h" /* LZF compression library */
#include <math.h>
#include <ctype.h>
@@ -41,49 +40,12 @@ robj *createObject(int type, void *ptr) {
o->refcount = 1;
/* Set the LRU to the current lruclock (minutes resolution). */
o->lru = LRU_CLOCK();
o->lru = server.lruclock;
return o;
}
/* Create a string object with encoding REDIS_ENCODING_RAW, that is a plain
* string object where o->ptr points to a proper sds string. */
robj *createRawStringObject(char *ptr, size_t len) {
return createObject(REDIS_STRING,sdsnewlen(ptr,len));
}
/* Create a string object with encoding REDIS_ENCODING_EMBSTR, that is
* an object where the sds string is actually an unmodifiable string
* allocated in the same chunk as the object itself. */
robj *createEmbeddedStringObject(char *ptr, size_t len) {
robj *o = zmalloc(sizeof(robj)+sizeof(struct sdshdr)+len+1);
struct sdshdr *sh = (void*)(o+1);
o->type = REDIS_STRING;
o->encoding = REDIS_ENCODING_EMBSTR;
o->ptr = sh+1;
o->refcount = 1;
o->lru = LRU_CLOCK();
sh->len = len;
sh->free = 0;
if (ptr) {
memcpy(sh->buf,ptr,len);
sh->buf[len] = '\0';
} else {
memset(sh->buf,0,len+1);
}
return o;
}
/* Create a string object with EMBSTR encoding if it is smaller than
* REIDS_ENCODING_EMBSTR_SIZE_LIMIT, otherwise the RAW encoding is
* used. */
#define REDIS_ENCODING_EMBSTR_SIZE_LIMIT 32
robj *createStringObject(char *ptr, size_t len) {
if (len <= REDIS_ENCODING_EMBSTR_SIZE_LIMIT)
return createEmbeddedStringObject(ptr,len);
else
return createRawStringObject(ptr,len);
return createObject(REDIS_STRING,sdsnewlen(ptr,len));
}
robj *createStringObjectFromLongLong(long long value) {
@@ -110,7 +72,7 @@ robj *createStringObjectFromLongDouble(long double value) {
int len;
/* We use 17 digits precision since with 128 bit floats that precision
* after rounding is able to represent most small decimal numbers in a way
* after rouding is able to represent most small decimal numbers in a way
* that is "non surprising" for the user (that is, most small decimal
* numbers will be represented in a way that when converted back into
* a string are exactly the same as what the user typed.) */
@@ -127,43 +89,15 @@ robj *createStringObjectFromLongDouble(long double value) {
return createStringObject(buf,len);
}
/* Duplicate a string object, with the guarantee that the returned object
* has the same encoding as the original one.
*
* This function also guarantees that duplicating a small integere object
* (or a string object that contains a representation of a small integer)
* will always result in a fresh object that is unshared (refcount == 1).
*
* The resulting object always has refcount set to 1. */
robj *dupStringObject(robj *o) {
robj *d;
redisAssert(o->type == REDIS_STRING);
switch(o->encoding) {
case REDIS_ENCODING_RAW:
return createRawStringObject(o->ptr,sdslen(o->ptr));
case REDIS_ENCODING_EMBSTR:
return createEmbeddedStringObject(o->ptr,sdslen(o->ptr));
case REDIS_ENCODING_INT:
d = createObject(REDIS_STRING, NULL);
d->encoding = REDIS_ENCODING_INT;
d->ptr = o->ptr;
return d;
case REDIS_ENCODING_LZF:
d = createObject(REDIS_STRING,sdsdup(o->ptr));
d->encoding = REDIS_ENCODING_LZF;
return d;
default:
redisPanic("Wrong encoding.");
break;
}
redisAssertWithInfo(NULL,o,o->encoding == REDIS_ENCODING_RAW);
return createStringObject(o->ptr,sdslen(o->ptr));
}
robj *createListObject(void) {
list *l = listCreate();
robj *o = createObject(REDIS_LIST,l);
listSetFreeMethod(l,decrRefCountVoid);
listSetFreeMethod(l,decrRefCount);
o->encoding = REDIS_ENCODING_LINKEDLIST;
return o;
}
@@ -215,8 +149,7 @@ robj *createZsetZiplistObject(void) {
}
void freeStringObject(robj *o) {
if (o->encoding == REDIS_ENCODING_RAW ||
o->encoding == REDIS_ENCODING_LZF) {
if (o->encoding == REDIS_ENCODING_RAW) {
sdsfree(o->ptr);
}
}
@@ -282,7 +215,9 @@ void incrRefCount(robj *o) {
o->refcount++;
}
void decrRefCount(robj *o) {
void decrRefCount(void *obj) {
robj *o = obj;
if (o->refcount <= 0) redisPanic("decrRefCount against refcount <= 0");
if (o->refcount == 1) {
switch(o->type) {
@@ -299,13 +234,6 @@ void decrRefCount(robj *o) {
}
}
/* This variant of decrRefCount() gets its argument as void, and is useful
* as free method in data structures that expect a 'void free_object(void*)'
* prototype for the free method. */
void decrRefCountVoid(void *o) {
decrRefCount(o);
}
/* This function set the ref count to zero without freeing the object.
* It is useful in order to pass a new object to functions incrementing
* the ref count of the received object. Example:
@@ -333,138 +261,50 @@ int checkType(redisClient *c, robj *o, int type) {
int isObjectRepresentableAsLongLong(robj *o, long long *llval) {
redisAssertWithInfo(NULL,o,o->type == REDIS_STRING);
if (intEncodedObject(o)) {
if (o->encoding == REDIS_ENCODING_INT) {
if (llval) *llval = (long) o->ptr;
return REDIS_OK;
} else if (sdsEncodedObject(o)) {
return string2ll(o->ptr,sdslen(o->ptr),llval) ? REDIS_OK : REDIS_ERR;
} else if (lzfEncodedObject(o)) {
int retval;
robj *decoded = getDecodedObject(o);
retval = string2ll(o->ptr,sdslen(o->ptr),llval) ? REDIS_OK : REDIS_ERR;
decrRefCount(decoded);
return retval;
} else {
redisPanic("Unknown encoding.");
return string2ll(o->ptr,sdslen(o->ptr),llval) ? REDIS_OK : REDIS_ERR;
}
}
/* Try to encode a string object in order to save space. */
#define LZF_COMPR_STATIC_BUF (1024*32)
/* Try to encode a string object in order to save space */
robj *tryObjectEncoding(robj *o) {
long value;
sds s = o->ptr;
size_t len;
/* Make sure this is a string object, the only type we encode
* in this function. Other types use encoded memory efficient
* representations but are handled by the commands implementing
* the type. */
redisAssertWithInfo(NULL,o,o->type == REDIS_STRING);
/* We try some specialized encoding only for objects that are
* RAW or EMBSTR encoded, in other words objects that are still
* in represented by an actually array of chars. */
if (!sdsEncodedObject(o)) return o;
if (o->encoding != REDIS_ENCODING_RAW)
return o; /* Already encoded */
/* It's not safe to encode shared objects: shared objects can be shared
* everywhere in the "object space" of Redis and may end in places where
* they are not handled. We handle them only as values in the keyspace. */
* everywhere in the "object space" of Redis. Encoded objects can only
* appear as "values" (and not, for instance, as keys) */
if (o->refcount > 1) return o;
/* Check if we can represent this string as a long integer.
* Note that we are sure that a string larger than 21 chars is not
* representable as a 32 nor 64 bit integer. */
len = sdslen(s);
if (len <= 21 && string2l(s,len,&value)) {
/* This object is encodable as a long. Try to use a shared object.
* Note that we avoid using shared integers when maxmemory is used
* because every object needs to have a private LRU field for the LRU
* algorithm to work well. */
if (server.maxmemory == 0 &&
value >= 0 &&
value < REDIS_SHARED_INTEGERS)
{
decrRefCount(o);
incrRefCount(shared.integers[value]);
return shared.integers[value];
} else {
if (o->encoding == REDIS_ENCODING_RAW) sdsfree(o->ptr);
o->encoding = REDIS_ENCODING_INT;
o->ptr = (void*) value;
return o;
}
}
/* Currently we try to encode only strings */
redisAssertWithInfo(NULL,o,o->type == REDIS_STRING);
/* If the string is small and is still RAW encoded,
* try the EMBSTR encoding which is more efficient.
* In this representation the object and the SDS string are allocated
* in the same chunk of memory to save space and cache misses. */
if (len <= REDIS_ENCODING_EMBSTR_SIZE_LIMIT) {
robj *emb;
/* Check if we can represent this string as a long integer */
if (!string2l(s,sdslen(s),&value)) return o;
if (o->encoding == REDIS_ENCODING_EMBSTR) return o;
emb = createEmbeddedStringObject(s,sdslen(s));
/* Ok, this object can be encoded...
*
* Can I use a shared object? Only if the object is inside a given range
*
* Note that we also avoid using shared integers when maxmemory is used
* because every object needs to have a private LRU field for the LRU
* algorithm to work well. */
if (server.maxmemory == 0 && value >= 0 && value < REDIS_SHARED_INTEGERS) {
decrRefCount(o);
return emb;
incrRefCount(shared.integers[value]);
return shared.integers[value];
} else {
o->encoding = REDIS_ENCODING_INT;
sdsfree(o->ptr);
o->ptr = (void*) value;
return o;
}
/* Try LZF compression for objects up to server.mem_compression_max_size
* and greater than REDIS_ENCODING_EMBSTR_SIZE_LIMIT. */
if (server.mem_compression && len <= server.mem_compression_max_size) {
/* Allocate four more bytes in our buffer since we need to store
* the size of the compressed string as header. */
unsigned char compr_static[LZF_COMPR_STATIC_BUF];
unsigned char *compr = compr_static;
size_t comprlen, outlen;
/* Save want to save at least 25% of memory for this to make sense. */
outlen = len-4-(len/4);
/* Use an heap allocated buffer if the output is too big for our
* static buffer. We use the trick to directly allocating an empty
* SDS string so we can use it directly to create the object later. */
if (outlen+4 > LZF_COMPR_STATIC_BUF)
compr = (unsigned char*) sdsnewlen(NULL,outlen+4);
comprlen = lzf_compress(s,len,compr+4,outlen);
if (comprlen != 0) {
/* Object successfully compressed within the required space. */
compr[0] = len & 0xff;
compr[1] = (len >> 8) & 0xff;
compr[2] = (len >> 16) & 0xff;
compr[3] = (len >> 24) & 0xff;
if (o->encoding == REDIS_ENCODING_RAW) sdsfree(o->ptr);
o->encoding = REDIS_ENCODING_LZF;
if (compr == compr_static) {
/* When compressing to the static buffer we have to
* generate the SDS string here. */
o->ptr = sdsnewlen(compr,comprlen+4);
} else {
/* Already an SDS, use it. */
o->ptr = compr;
}
return o;
}
if (compr != compr_static) sdsfree((char*)compr);
}
/* We can't encode the object...
*
* Do the last try, and at least optimize the SDS string inside
* the string object to require little space, in case there
* is more than 10% of free space at the end of the SDS string.
*
* We do that only for relatively large strings as this branch
* is only entered if the length of the string is greater than
* REDIS_ENCODING_EMBSTR_SIZE_LIMIT. */
if (o->encoding == REDIS_ENCODING_RAW &&
sdsavail(s) > len/10)
{
o->ptr = sdsRemoveFreeSpace(o->ptr);
}
/* Return the original object. */
return o;
}
/* Get a decoded version of an encoded object (returned as a new object).
@@ -472,91 +312,50 @@ robj *tryObjectEncoding(robj *o) {
robj *getDecodedObject(robj *o) {
robj *dec;
if (sdsEncodedObject(o)) {
if (o->encoding == REDIS_ENCODING_RAW) {
incrRefCount(o);
return o;
}
if (o->type == REDIS_STRING && intEncodedObject(o)) {
if (o->type == REDIS_STRING && o->encoding == REDIS_ENCODING_INT) {
char buf[32];
ll2string(buf,32,(long)o->ptr);
dec = createStringObject(buf,strlen(buf));
return dec;
} else if (o->type == REDIS_STRING && lzfEncodedObject(o)) {
int origlen = stringObjectLen(o);
sds orig = sdsnewlen(NULL,origlen);
unsigned char *p = o->ptr;
if (lzf_decompress(p+4,sdslen(o->ptr)-4,orig,origlen) == 0)
redisPanic("LZF error during object decoding.");
return createObject(REDIS_STRING,orig);
} else {
redisPanic("Unknown encoding type");
}
}
/* Compare two string objects via strcmp() or strcoll() depending on flags.
/* Compare two string objects via strcmp() or alike.
* Note that the objects may be integer-encoded. In such a case we
* use ll2string() to get a string representation of the numbers on the stack
* and compare the strings, it's much faster than calling getDecodedObject().
*
* Important note: when REDIS_COMPARE_BINARY is used a binary-safe comparison
* is used. */
#define REDIS_COMPARE_BINARY (1<<0)
#define REDIS_COMPARE_COLL (1<<1)
int compareStringObjectsWithFlags(robj *a, robj *b, int flags) {
* Important note: if objects are not integer encoded, but binary-safe strings,
* sdscmp() from sds.c will apply memcmp() so this function ca be considered
* binary safe. */
int compareStringObjects(robj *a, robj *b) {
redisAssertWithInfo(NULL,a,a->type == REDIS_STRING && b->type == REDIS_STRING);
char bufa[128], bufb[128], *astr, *bstr;
size_t alen, blen, minlen;
int decr = 0;
int bothsds = 1;
if (a == b) return 0;
if (lzfEncodedObject(a) || lzfEncodedObject(b)) {
a = getDecodedObject(a);
b = getDecodedObject(b);
decr = 1;
}
if (sdsEncodedObject(a)) {
astr = a->ptr;
alen = sdslen(astr);
} else {
alen = ll2string(bufa,sizeof(bufa),(long) a->ptr);
if (a->encoding != REDIS_ENCODING_RAW) {
ll2string(bufa,sizeof(bufa),(long) a->ptr);
astr = bufa;
}
if (sdsEncodedObject(b)) {
bstr = b->ptr;
blen = sdslen(bstr);
bothsds = 0;
} else {
blen = ll2string(bufb,sizeof(bufb),(long) b->ptr);
astr = a->ptr;
}
if (b->encoding != REDIS_ENCODING_RAW) {
ll2string(bufb,sizeof(bufb),(long) b->ptr);
bstr = bufb;
}
if (flags & REDIS_COMPARE_COLL) {
return strcoll(astr,bstr);
bothsds = 0;
} else {
int cmp;
minlen = (alen < blen) ? alen : blen;
cmp = memcmp(astr,bstr,minlen);
if (cmp == 0) return alen-blen;
return cmp;
bstr = b->ptr;
}
if (decr) {
decrRefCount(a);
decrRefCount(b);
}
}
/* Wrapper for compareStringObjectsWithFlags() using binary comparison. */
int compareStringObjects(robj *a, robj *b) {
return compareStringObjectsWithFlags(a,b,REDIS_COMPARE_BINARY);
}
/* Wrapper for compareStringObjectsWithFlags() using collation. */
int collateStringObjects(robj *a, robj *b) {
return compareStringObjectsWithFlags(a,b,REDIS_COMPARE_COLL);
return bothsds ? sdscmp(astr,bstr) : strcmp(astr,bstr);
}
/* Equal string objects return 1 if the two objects are the same from the
@@ -564,30 +363,20 @@ int collateStringObjects(robj *a, robj *b) {
* this function is faster then checking for (compareStringObject(a,b) == 0)
* because it can perform some more optimization. */
int equalStringObjects(robj *a, robj *b) {
if (intEncodedObject(a) && intEncodedObject(b)) {
/* If both strings are integer encoded just check if the stored
* long is the same. */
if (a->encoding != REDIS_ENCODING_RAW && b->encoding != REDIS_ENCODING_RAW){
return a->ptr == b->ptr;
} else {
return compareStringObjects(a,b) == 0;
}
}
/* Returns the original (uncompressed) size of an LZF encoded object.
* Only called by stringObjectLen() that should be the main interface. */
size_t stringObjectUncompressedLen(robj *o) {
unsigned char *p = o->ptr;
return p[0] | (p[1]<<8) | (p[2]<<16) | (p[3]<<24);
}
size_t stringObjectLen(robj *o) {
redisAssertWithInfo(NULL,o,o->type == REDIS_STRING);
if (sdsEncodedObject(o)) {
if (o->encoding == REDIS_ENCODING_RAW) {
return sdslen(o->ptr);
} else if (o->encoding == REDIS_ENCODING_LZF) {
return stringObjectUncompressedLen(o);
} else {
char buf[32];
return ll2string(buf,32,(long)o->ptr);
}
}
@@ -600,24 +389,14 @@ int getDoubleFromObject(robj *o, double *target) {
value = 0;
} else {
redisAssertWithInfo(NULL,o,o->type == REDIS_STRING);
if (sdsEncodedObject(o)) {
if (o->encoding == REDIS_ENCODING_RAW) {
errno = 0;
value = strtod(o->ptr, &eptr);
if (isspace(((char*)o->ptr)[0]) ||
eptr[0] != '\0' ||
(errno == ERANGE &&
(value == HUGE_VAL || value == -HUGE_VAL || value == 0)) ||
errno == EINVAL ||
isnan(value))
if (isspace(((char*)o->ptr)[0]) || eptr[0] != '\0' ||
errno == ERANGE || isnan(value))
return REDIS_ERR;
} else if (intEncodedObject(o)) {
} else if (o->encoding == REDIS_ENCODING_INT) {
value = (long)o->ptr;
} else if (lzfEncodedObject(o)) {
int retval;
o = getDecodedObject(o);
retval = getDoubleFromObject(o,target);
decrRefCount(o);
return retval;
} else {
redisPanic("Unknown string encoding");
}
@@ -648,20 +427,14 @@ int getLongDoubleFromObject(robj *o, long double *target) {
value = 0;
} else {
redisAssertWithInfo(NULL,o,o->type == REDIS_STRING);
if (sdsEncodedObject(o)) {
if (o->encoding == REDIS_ENCODING_RAW) {
errno = 0;
value = strtold(o->ptr, &eptr);
if (isspace(((char*)o->ptr)[0]) || eptr[0] != '\0' ||
errno == ERANGE || isnan(value))
return REDIS_ERR;
} else if (intEncodedObject(o)) {
} else if (o->encoding == REDIS_ENCODING_INT) {
value = (long)o->ptr;
} else if (lzfEncodedObject(o)) {
int retval;
o = getDecodedObject(o);
retval = getLongDoubleFromObject(o,target);
decrRefCount(o);
return retval;
} else {
redisPanic("Unknown string encoding");
}
@@ -692,20 +465,14 @@ int getLongLongFromObject(robj *o, long long *target) {
value = 0;
} else {
redisAssertWithInfo(NULL,o,o->type == REDIS_STRING);
if (sdsEncodedObject(o)) {
if (o->encoding == REDIS_ENCODING_RAW) {
errno = 0;
value = strtoll(o->ptr, &eptr, 10);
if (isspace(((char*)o->ptr)[0]) || eptr[0] != '\0' ||
errno == ERANGE)
return REDIS_ERR;
} else if (intEncodedObject(o)) {
} else if (o->encoding == REDIS_ENCODING_INT) {
value = (long)o->ptr;
} else if (lzfEncodedObject(o)) {
int retval;
o = getDecodedObject(o);
retval = getLongLongFromObject(o,target);
decrRefCount(o);
return retval;
} else {
redisPanic("Unknown string encoding");
}
@@ -753,25 +520,22 @@ char *strEncoding(int encoding) {
case REDIS_ENCODING_ZIPLIST: return "ziplist";
case REDIS_ENCODING_INTSET: return "intset";
case REDIS_ENCODING_SKIPLIST: return "skiplist";
case REDIS_ENCODING_EMBSTR: return "embstr";
case REDIS_ENCODING_LZF: return "lzf";
default: return "unknown";
}
}
/* Given an object returns the min number of milliseconds the object was never
/* Given an object returns the min number of seconds the object was never
* requested, using an approximated LRU algorithm. */
unsigned long long estimateObjectIdleTime(robj *o) {
unsigned long long lruclock = LRU_CLOCK();
if (lruclock >= o->lru) {
return (lruclock - o->lru) * REDIS_LRU_CLOCK_RESOLUTION;
unsigned long estimateObjectIdleTime(robj *o) {
if (server.lruclock >= o->lru) {
return (server.lruclock - o->lru) * REDIS_LRU_CLOCK_RESOLUTION;
} else {
return (lruclock + (REDIS_LRU_CLOCK_MAX - o->lru)) *
return ((REDIS_LRU_CLOCK_MAX - o->lru) + server.lruclock) *
REDIS_LRU_CLOCK_RESOLUTION;
}
}
/* This is a helper function for the OBJECT command. We need to lookup keys
/* This is an helper function for the DEBUG command. We need to lookup keys
* without any modification of LRU or other parameters. */
robj *objectCommandLookup(redisClient *c, robj *key) {
dictEntry *de;
@@ -803,7 +567,7 @@ void objectCommand(redisClient *c) {
} else if (!strcasecmp(c->argv[1]->ptr,"idletime") && c->argc == 3) {
if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.nullbulk))
== NULL) return;
addReplyLongLong(c,estimateObjectIdleTime(o)/1000);
addReplyLongLong(c,estimateObjectIdleTime(o));
} else {
addReplyError(c,"Syntax error. Try OBJECT (refcount|encoding|idletime)");
}
+13 -1
View File
@@ -39,6 +39,7 @@
#include <sys/types.h>
#include <assert.h>
#include <errno.h>
#include <stdlib.h>
@@ -101,9 +102,10 @@ _pqsort(void *a, size_t n, size_t es,
{
char *pa, *pb, *pc, *pd, *pl, *pm, *pn;
size_t d, r;
int swaptype, cmp_result;
int swaptype, swap_cnt, cmp_result;
loop: SWAPINIT(a, es);
swap_cnt = 0;
if (n < 7) {
for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)
for (pl = pm; pl > (char *) a && cmp(pl - es, pl) > 0;
@@ -130,6 +132,7 @@ loop: SWAPINIT(a, es);
for (;;) {
while (pb <= pc && (cmp_result = cmp(pb, a)) <= 0) {
if (cmp_result == 0) {
swap_cnt = 1;
swap(pa, pb);
pa += es;
}
@@ -137,6 +140,7 @@ loop: SWAPINIT(a, es);
}
while (pb <= pc && (cmp_result = cmp(pc, a)) >= 0) {
if (cmp_result == 0) {
swap_cnt = 1;
swap(pc, pd);
pd -= es;
}
@@ -145,9 +149,17 @@ loop: SWAPINIT(a, es);
if (pb > pc)
break;
swap(pb, pc);
swap_cnt = 1;
pb += es;
pc -= es;
}
if (swap_cnt == 0) { /* Switch to insertion sort */
for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)
for (pl = pm; pl > (char *) a && cmp(pl - es, pl) > 0;
pl -= es)
swap(pl, pl - es);
return;
}
pn = (char *) a + n * es;
r = min(pa - (char *) a, pb - pa);
+6 -71
View File
@@ -50,7 +50,7 @@ int listMatchPubsubPattern(void *a, void *b) {
/* Subscribe a client to a channel. Returns 1 if the operation succeeded, or
* 0 if the client was already subscribed to that channel. */
int pubsubSubscribeChannel(redisClient *c, robj *channel) {
dictEntry *de;
struct dictEntry *de;
list *clients = NULL;
int retval = 0;
@@ -80,7 +80,7 @@ int pubsubSubscribeChannel(redisClient *c, robj *channel) {
/* Unsubscribe a client from a channel. Returns 1 if the operation succeeded, or
* 0 if the client was not subscribed to the specified channel. */
int pubsubUnsubscribeChannel(redisClient *c, robj *channel, int notify) {
dictEntry *de;
struct dictEntry *de;
list *clients;
listNode *ln;
int retval = 0;
@@ -117,7 +117,7 @@ int pubsubUnsubscribeChannel(redisClient *c, robj *channel, int notify) {
return retval;
}
/* Subscribe a client to a pattern. Returns 1 if the operation succeeded, or 0 if the client was already subscribed to that pattern. */
/* Subscribe a client to a pattern. Returns 1 if the operation succeeded, or 0 if the clinet was already subscribed to that pattern. */
int pubsubSubscribePattern(redisClient *c, robj *pattern) {
int retval = 0;
@@ -179,14 +179,6 @@ int pubsubUnsubscribeAllChannels(redisClient *c, int notify) {
count += pubsubUnsubscribeChannel(c,channel,notify);
}
/* We were subscribed to nothing? Still reply to the client. */
if (notify && count == 0) {
addReply(c,shared.mbulkhdr[3]);
addReply(c,shared.unsubscribebulk);
addReply(c,shared.nullbulk);
addReplyLongLong(c,dictSize(c->pubsub_channels)+
listLength(c->pubsub_patterns));
}
dictReleaseIterator(di);
return count;
}
@@ -204,21 +196,13 @@ int pubsubUnsubscribeAllPatterns(redisClient *c, int notify) {
count += pubsubUnsubscribePattern(c,pattern,notify);
}
if (notify && count == 0) {
/* We were subscribed to nothing? Still reply to the client. */
addReply(c,shared.mbulkhdr[3]);
addReply(c,shared.punsubscribebulk);
addReply(c,shared.nullbulk);
addReplyLongLong(c,dictSize(c->pubsub_channels)+
listLength(c->pubsub_patterns));
}
return count;
}
/* Publish a message */
int pubsubPublishMessage(robj *channel, robj *message) {
int receivers = 0;
dictEntry *de;
struct dictEntry *de;
listNode *ln;
listIter li;
@@ -278,6 +262,7 @@ void subscribeCommand(redisClient *c) {
void unsubscribeCommand(redisClient *c) {
if (c->argc == 1) {
pubsubUnsubscribeAllChannels(c,1);
return;
} else {
int j;
@@ -296,6 +281,7 @@ void psubscribeCommand(redisClient *c) {
void punsubscribeCommand(redisClient *c) {
if (c->argc == 1) {
pubsubUnsubscribeAllPatterns(c,1);
return;
} else {
int j;
@@ -306,56 +292,5 @@ void punsubscribeCommand(redisClient *c) {
void publishCommand(redisClient *c) {
int receivers = pubsubPublishMessage(c->argv[1],c->argv[2]);
if (server.cluster_enabled)
clusterPropagatePublish(c->argv[1],c->argv[2]);
else
forceCommandPropagation(c,REDIS_PROPAGATE_REPL);
addReplyLongLong(c,receivers);
}
/* PUBSUB command for Pub/Sub introspection. */
void pubsubCommand(redisClient *c) {
if (!strcasecmp(c->argv[1]->ptr,"channels") &&
(c->argc == 2 || c->argc ==3))
{
/* PUBSUB CHANNELS [<pattern>] */
sds pat = (c->argc == 2) ? NULL : c->argv[2]->ptr;
dictIterator *di = dictGetIterator(server.pubsub_channels);
dictEntry *de;
long mblen = 0;
void *replylen;
replylen = addDeferredMultiBulkLength(c);
while((de = dictNext(di)) != NULL) {
robj *cobj = dictGetKey(de);
sds channel = cobj->ptr;
if (!pat || stringmatchlen(pat, sdslen(pat),
channel, sdslen(channel),0))
{
addReplyBulk(c,cobj);
mblen++;
}
}
dictReleaseIterator(di);
setDeferredMultiBulkLength(c,replylen,mblen);
} else if (!strcasecmp(c->argv[1]->ptr,"numsub") && c->argc >= 2) {
/* PUBSUB NUMSUB [Channel_1 ... Channel_N] */
int j;
addReplyMultiBulkLen(c,(c->argc-2)*2);
for (j = 2; j < c->argc; j++) {
list *l = dictFetchValue(server.pubsub_channels,c->argv[j]);
addReplyBulk(c,c->argv[j]);
addReplyBulkLongLong(c,l ? listLength(l) : 0);
}
} else if (!strcasecmp(c->argv[1]->ptr,"numpat") && c->argc == 2) {
/* PUBSUB NUMPAT */
addReplyLongLong(c,listLength(server.pubsub_patterns));
} else {
addReplyErrorFormat(c,
"Unknown PUBSUB subcommand or wrong number of arguments for '%s'",
(char*)c->argv[1]->ptr);
}
}
+59 -89
View File
@@ -99,7 +99,7 @@ int rdbSaveLen(rio *rdb, uint32_t len) {
buf[0] = (REDIS_RDB_32BITLEN<<6);
if (rdbWriteRaw(rdb,buf,1) == -1) return -1;
len = htonl(len);
if (rdbWriteRaw(rdb,&len,4) == -1) return -1;
if (rdbWriteRaw(rdb,&len,4) == -4) return -1;
nwritten = 1+4;
}
return nwritten;
@@ -209,41 +209,11 @@ int rdbTryIntegerEncoding(char *s, size_t len, unsigned char *enc) {
return rdbEncodeInteger(value,enc);
}
/* Save an already compressed object in LZF encoding.
*
* On success the length of the strored object is returned, otherwise
* 0 is returned. */
int rdbSaveLzfStringObject(rio *rdb, unsigned char *out, size_t len, size_t comprlen) {
int rdbSaveLzfStringObject(rio *rdb, unsigned char *s, size_t len) {
size_t comprlen, outlen;
unsigned char byte;
int n, nwritten = 0;
/* Data compressed! Let's save it on disk */
byte = (REDIS_RDB_ENCVAL<<6)|REDIS_RDB_ENC_LZF;
if ((n = rdbWriteRaw(rdb,&byte,1)) == -1) goto writeerr;
nwritten += n;
if ((n = rdbSaveLen(rdb,comprlen)) == -1) goto writeerr;
nwritten += n;
if ((n = rdbSaveLen(rdb,len)) == -1) goto writeerr;
nwritten += n;
if ((n = rdbWriteRaw(rdb,out,comprlen)) == -1) goto writeerr;
nwritten += n;
return nwritten;
writeerr:
zfree(out);
return -1;
}
/* Try to compress the string at 's' for 'len' bytes using LZF.
* If successful save the object with LZF encoding, otherwise
* returns 0 if the string can't be compressed, or -1 if the
* compressed string can't be saved.
*
* On success the number of bytes used is returned. */
int rdbTrySaveLzfStringObject(rio *rdb, unsigned char *s, size_t len) {
size_t comprlen, outlen;
void *out;
int retval;
/* We require at least four bytes compression for this to be worth it */
if (len <= 4) return 0;
@@ -254,9 +224,26 @@ int rdbTrySaveLzfStringObject(rio *rdb, unsigned char *s, size_t len) {
zfree(out);
return 0;
}
retval = rdbSaveLzfStringObject(rdb,out,len,comprlen);
/* Data compressed! Let's save it on disk */
byte = (REDIS_RDB_ENCVAL<<6)|REDIS_RDB_ENC_LZF;
if ((n = rdbWriteRaw(rdb,&byte,1)) == -1) goto writeerr;
nwritten += n;
if ((n = rdbSaveLen(rdb,comprlen)) == -1) goto writeerr;
nwritten += n;
if ((n = rdbSaveLen(rdb,len)) == -1) goto writeerr;
nwritten += n;
if ((n = rdbWriteRaw(rdb,out,comprlen)) == -1) goto writeerr;
nwritten += n;
zfree(out);
return retval;
return nwritten;
writeerr:
zfree(out);
return -1;
}
robj *rdbLoadLzfStringObject(rio *rdb) {
@@ -278,7 +265,7 @@ err:
return NULL;
}
/* Save a string object as [len][data] on disk. If the object is a string
/* Save a string objet as [len][data] on disk. If the object is a string
* representation of an integer value we try to save it in a special form */
int rdbSaveRawString(rio *rdb, unsigned char *s, size_t len) {
int enclen;
@@ -296,7 +283,7 @@ int rdbSaveRawString(rio *rdb, unsigned char *s, size_t len) {
/* Try LZF compression - under 20 bytes it's unable to compress even
* aaaaaaaaaaaaaaaaaa so skip it */
if (server.rdb_compression && len > 20) {
n = rdbTrySaveLzfStringObject(rdb,s,len);
n = rdbSaveLzfStringObject(rdb,s,len);
if (n == -1) return -1;
if (n > 0) return n;
/* Return value of 0 means data can't be compressed, save the old way */
@@ -334,16 +321,11 @@ int rdbSaveLongLongAsStringObject(rio *rdb, long long value) {
/* Like rdbSaveStringObjectRaw() but handle encoded objects */
int rdbSaveStringObject(rio *rdb, robj *obj) {
/* Avoid to decode the object, then encode it again, if the
* object is already integer encoded. */
* object is alrady integer encoded. */
if (obj->encoding == REDIS_ENCODING_INT) {
return rdbSaveLongLongAsStringObject(rdb,(long)obj->ptr);
} else if (obj->encoding == REDIS_ENCODING_LZF) {
/* Data is already compressed, save it with LZF encoding. */
int len = stringObjectLen(obj);
unsigned char *p = obj->ptr;
return rdbSaveLzfStringObject(rdb,p+4,len,sdslen(obj->ptr)-4);
} else {
redisAssertWithInfo(NULL,obj,sdsEncodedObject(obj));
redisAssertWithInfo(NULL,obj,obj->encoding == REDIS_ENCODING_RAW);
return rdbSaveRawString(rdb,obj->ptr,sdslen(obj->ptr));
}
}
@@ -385,7 +367,7 @@ robj *rdbLoadEncodedStringObject(rio *rdb) {
}
/* Save a double value. Doubles are saved as strings prefixed by an unsigned
* 8 bit integer specifying the length of the representation.
* 8 bit integer specifing the length of the representation.
* This 8 bit integer has special values in order to specify the following
* conditions:
* 253: not a number
@@ -624,7 +606,7 @@ off_t rdbSavedObjectLen(robj *o) {
/* Save a key-value pair, with expire time, type, key, value.
* On error -1 is returned.
* On success if the key was actually saved 1 is returned, otherwise 0
* On success if the key was actaully saved 1 is returned, otherwise 0
* is returned (the key was already expired). */
int rdbSaveKeyValuePair(rio *rdb, robj *key, robj *val,
long long expiretime, long long now)
@@ -708,9 +690,9 @@ int rdbSave(char *filename) {
rioWrite(&rdb,&cksum,8);
/* Make sure data will not remain on the OS's output buffers */
if (fflush(fp) == EOF) goto werr;
if (fsync(fileno(fp)) == -1) goto werr;
if (fclose(fp) == EOF) goto werr;
fflush(fp);
fsync(fileno(fp));
fclose(fp);
/* Use RENAME to make sure the DB file is changed atomically only
* if the generate DB file is ok. */
@@ -740,22 +722,21 @@ int rdbSaveBackground(char *filename) {
if (server.rdb_child_pid != -1) return REDIS_ERR;
server.dirty_before_bgsave = server.dirty;
server.lastbgsave_try = time(NULL);
start = ustime();
if ((childpid = fork()) == 0) {
int retval;
/* Child */
closeListeningSockets(0);
redisSetProcTitle("redis-rdb-bgsave");
if (server.ipfd > 0) close(server.ipfd);
if (server.sofd > 0) close(server.sofd);
retval = rdbSave(filename);
if (retval == REDIS_OK) {
size_t private_dirty = zmalloc_get_private_dirty();
if (private_dirty) {
redisLog(REDIS_NOTICE,
"RDB: %zu MB of memory used by copy-on-write",
"RDB: %lu MB of memory used by copy-on-write",
private_dirty/(1024*1024));
}
}
@@ -764,7 +745,6 @@ int rdbSaveBackground(char *filename) {
/* Parent */
server.stat_fork_time = ustime()-start;
if (childpid == -1) {
server.lastbgsave_status = REDIS_ERR;
redisLog(REDIS_WARNING,"Can't save in background: fork: %s",
strerror(errno));
return REDIS_ERR;
@@ -792,6 +772,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
size_t len;
unsigned int i;
redisLog(REDIS_DEBUG,"LOADING OBJECT %d (at %d)\n",rdbtype,rioTell(rdb));
if (rdbtype == REDIS_RDB_TYPE_STRING) {
/* Read string value */
if ((o = rdbLoadEncodedStringObject(rdb)) == NULL) return NULL;
@@ -814,7 +795,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
/* If we are using a ziplist and the value is too big, convert
* the object to a real list. */
if (o->encoding == REDIS_ENCODING_ZIPLIST &&
sdsEncodedObject(ele) &&
ele->encoding == REDIS_ENCODING_RAW &&
sdslen(ele->ptr) > server.list_max_ziplist_value)
listTypeConvert(o,REDIS_ENCODING_LINKEDLIST);
@@ -860,7 +841,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
}
/* This will also be called when the set was just converted
* to a regular hash table encoded set */
* to regular hash table encoded set */
if (o->encoding == REDIS_ENCODING_HT) {
dictAdd((dict*)o->ptr,ele,NULL);
} else {
@@ -888,8 +869,9 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
if (rdbLoadDoubleValue(rdb,&score) == -1) return NULL;
/* Don't care about integer-encoded strings. */
if (sdsEncodedObject(ele) && sdslen(ele->ptr) > maxelelen)
maxelelen = sdslen(ele->ptr);
if (ele->encoding == REDIS_ENCODING_RAW &&
sdslen(ele->ptr) > maxelelen)
maxelelen = sdslen(ele->ptr);
znode = zslInsert(zs->zsl,score,ele);
dictAdd(zs->dict,ele,&znode->score);
@@ -909,7 +891,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
o = createHashObject();
/* Too many entries? Use a hash table. */
/* Too many entries? Use an hash table. */
if (len > server.hash_max_ziplist_entries)
hashTypeConvert(o, REDIS_ENCODING_HT);
@@ -921,10 +903,10 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
/* Load raw strings */
field = rdbLoadStringObject(rdb);
if (field == NULL) return NULL;
redisAssert(sdsEncodedObject(field));
redisAssert(field->encoding == REDIS_ENCODING_RAW);
value = rdbLoadStringObject(rdb);
if (value == NULL) return NULL;
redisAssert(sdsEncodedObject(value));
redisAssert(field->encoding == REDIS_ENCODING_RAW);
/* Add pair to ziplist */
o->ptr = ziplistPush(o->ptr, field->ptr, sdslen(field->ptr), ZIPLIST_TAIL);
@@ -1075,39 +1057,24 @@ void stopLoading(void) {
server.loading = 0;
}
/* Track loading progress in order to serve client's from time to time
and if needed calculate rdb checksum */
void rdbLoadProgressCallback(rio *r, const void *buf, size_t len) {
if (server.rdb_checksum)
rioGenericUpdateChecksum(r, buf, len);
if (server.loading_process_events_interval_bytes &&
(r->processed_bytes + len)/server.loading_process_events_interval_bytes > r->processed_bytes/server.loading_process_events_interval_bytes)
{
/* The DB can take some non trivial amount of time to load. Update
* our cached time since it is used to create and update the last
* interaction time with clients and for other important things. */
updateCachedTime();
if (server.masterhost && server.repl_state == REDIS_REPL_TRANSFER)
replicationSendNewlineToMaster();
loadingProgress(r->processed_bytes);
aeProcessEvents(server.el, AE_FILE_EVENTS|AE_DONT_WAIT);
}
}
int rdbLoad(char *filename) {
uint32_t dbid;
int type, rdbver;
redisDb *db = server.db+0;
char buf[1024];
long long expiretime, now = mstime();
long loops = 0;
FILE *fp;
rio rdb;
if ((fp = fopen(filename,"r")) == NULL) return REDIS_ERR;
fp = fopen(filename,"r");
if (!fp) {
errno = ENOENT;
return REDIS_ERR;
}
rioInitWithFile(&rdb,fp);
rdb.update_cksum = rdbLoadProgressCallback;
rdb.max_processing_chunk = server.loading_process_events_interval_bytes;
if (server.rdb_checksum)
rdb.update_cksum = rioGenericUpdateChecksum;
if (rioRead(&rdb,buf,9) == 0) goto eoferr;
buf[9] = '\0';
if (memcmp(buf,"REDIS",5) != 0) {
@@ -1129,6 +1096,12 @@ int rdbLoad(char *filename) {
robj *key, *val;
expiretime = -1;
/* Serve the clients from time to time */
if (!(loops++ % 1000)) {
loadingProgress(rioTell(&rdb));
aeProcessEvents(server.el, AE_FILE_EVENTS|AE_DONT_WAIT);
}
/* Read type. */
if ((type = rdbLoadType(&rdb)) == -1) goto eoferr;
if (type == REDIS_RDB_OPCODE_EXPIRETIME) {
@@ -1136,7 +1109,7 @@ int rdbLoad(char *filename) {
/* We read the time so we need to read the object type again. */
if ((type = rdbLoadType(&rdb)) == -1) goto eoferr;
/* the EXPIRETIME opcode specifies time in seconds, so convert
* into milliseconds. */
* into milliesconds. */
expiretime *= 1000;
} else if (type == REDIS_RDB_OPCODE_EXPIRETIME_MS) {
/* Milliseconds precision expire times introduced with RDB
@@ -1221,10 +1194,7 @@ void backgroundSaveDoneHandler(int exitcode, int bysignal) {
redisLog(REDIS_WARNING,
"Background saving terminated by signal %d", bysignal);
rdbRemoveTempFile(server.rdb_child_pid);
/* SIGUSR1 is whitelisted, so we have a way to kill a child without
* tirggering an error conditon. */
if (bysignal != SIGUSR1)
server.lastbgsave_status = REDIS_ERR;
server.lastbgsave_status = REDIS_ERR;
}
server.rdb_child_pid = -1;
server.rdb_save_time_last = time(NULL)-server.rdb_save_time_start;
+1 -1
View File
@@ -51,7 +51,7 @@
* number specify the kind of object that follows.
* See the REDIS_RDB_ENC_* defines.
*
* Lengths up to 63 are stored using a single byte, most DB keys, and may
* Lenghts up to 63 are stored using a single byte, most DB keys, and may
* values, will fit inside. */
#define REDIS_RDB_6BITLEN 0
#define REDIS_RDB_14BITLEN 1
+39 -152
View File
@@ -35,7 +35,6 @@
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <time.h>
#include <sys/time.h>
#include <signal.h>
#include <assert.h>
@@ -47,7 +46,6 @@
#include "zmalloc.h"
#define REDIS_NOTUSED(V) ((void) V)
#define RANDPTR_INITIAL_SIZE 8
static struct config {
aeEventLoop *el;
@@ -74,24 +72,18 @@ static struct config {
int csv;
int loop;
int idlemode;
int dbnum;
sds dbnumstr;
char *tests;
} config;
typedef struct _client {
redisContext *context;
sds obuf;
char **randptr; /* Pointers to :rand: strings inside the command buf */
size_t randlen; /* Number of pointers in client->randptr */
size_t randfree; /* Number of unused pointers in client->randptr */
unsigned int written; /* Bytes of 'obuf' already written */
long long start; /* Start time of a request */
long long latency; /* Request latency */
int pending; /* Number of pending requests (replies to consume) */
int selectlen; /* If non-zero, a SELECT of 'selectlen' bytes is currently
used as a prefix of the pipline of commands. This gets
discarded the first time it's sent. */
char *randptr[32]; /* needed for MSET against 10 keys */
size_t randlen;
unsigned int written; /* bytes of 'obuf' already written */
long long start; /* start time of a request */
long long latency; /* request latency */
int pending; /* Number of pending requests (sent but no reply received) */
} *client;
/* Prototypes */
@@ -114,7 +106,7 @@ static long long mstime(void) {
long long mst;
gettimeofday(&tv, NULL);
mst = ((long long)tv.tv_sec)*1000;
mst = ((long)tv.tv_sec)*1000;
mst += tv.tv_usec/1000;
return mst;
}
@@ -125,7 +117,6 @@ static void freeClient(client c) {
aeDeleteFileEvent(config.el,c->context->fd,AE_READABLE);
redisFree(c->context);
sdsfree(c->obuf);
zfree(c->randptr);
zfree(c);
config.liveclients--;
ln = listSearchKey(config.clients,c);
@@ -152,18 +143,13 @@ static void resetClient(client c) {
}
static void randomizeClientKey(client c) {
size_t i;
char buf[32];
size_t i, r;
for (i = 0; i < c->randlen; i++) {
char *p = c->randptr[i]+11;
size_t r = random() % config.randomkeys_keyspacelen;
size_t j;
for (j = 0; j < 12; j++) {
*p = '0'+r%10;
r/=10;
p--;
}
r = random() % config.randomkeys_keyspacelen;
snprintf(buf,sizeof(buf),"%012zu",r);
memcpy(c->randptr[i],buf,12);
}
}
@@ -212,21 +198,6 @@ static void readHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
freeReplyObject(reply);
if (c->selectlen) {
int j;
/* This is the OK from SELECT. Just discard the SELECT
* from the buffer. */
c->pending--;
sdsrange(c->obuf,c->selectlen,-1);
/* We also need to fix the pointers to the strings
* we need to randomize. */
for (j = 0; j < c->randlen; j++)
c->randptr[j] -= c->selectlen;
c->selectlen = 0;
continue;
}
if (config.requests_finished < config.requests)
config.latency[config.requests_finished++] = c->latency;
c->pending--;
@@ -278,29 +249,7 @@ static void writeHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
}
}
/* Create a benchmark client, configured to send the command passed as 'cmd' of
* 'len' bytes.
*
* The command is copied N times in the client output buffer (that is reused
* again and again to send the request to the server) accordingly to the configured
* pipeline size.
*
* Also an initial SELECT command is prepended in order to make sure the right
* database is selected, if needed. The initial SELECT will be discarded as soon
* as the first reply is received.
*
* To create a client from scratch, the 'from' pointer is set to NULL. If instead
* we want to create a client using another client as reference, the 'from' pointer
* points to the client to use as reference. In such a case the following
* information is take from the 'from' client:
*
* 1) The command line to use.
* 2) The offsets of the __rand_int__ elements inside the command line, used
* for arguments randomization.
*
* Even when cloning another client, the SELECT command is automatically prefixed
* if needed. */
static client createClient(char *cmd, size_t len, client from) {
static client createClient(char *cmd, size_t len) {
int j;
client c = zmalloc(sizeof(struct _client));
@@ -319,68 +268,25 @@ static client createClient(char *cmd, size_t len, client from) {
}
/* Suppress hiredis cleanup of unused buffers for max speed. */
c->context->reader->maxbuf = 0;
/* Build the request buffer:
* Queue N requests accordingly to the pipeline size, or simply clone
* the example client buffer. */
/* Queue N requests accordingly to the pipeline size. */
c->obuf = sdsempty();
/* If a DB number different than zero is selected, prefix our request
* buffer with the SELECT command, that will be discarded the first
* time the replies are received, so if the client is reused the
* SELECT command will not be used again. */
if (config.dbnum != 0) {
c->obuf = sdscatprintf(c->obuf,"*2\r\n$6\r\nSELECT\r\n$%d\r\n%s\r\n",
(int)sdslen(config.dbnumstr),config.dbnumstr);
c->selectlen = sdslen(c->obuf);
} else {
c->selectlen = 0;
}
/* Append the request itself. */
if (from) {
c->obuf = sdscatlen(c->obuf,
from->obuf+from->selectlen,
sdslen(from->obuf)-from->selectlen);
} else {
for (j = 0; j < config.pipeline; j++)
c->obuf = sdscatlen(c->obuf,cmd,len);
}
for (j = 0; j < config.pipeline; j++)
c->obuf = sdscatlen(c->obuf,cmd,len);
c->randlen = 0;
c->written = 0;
c->pending = config.pipeline;
c->randptr = NULL;
c->randlen = 0;
if (c->selectlen) c->pending++;
/* Find substrings in the output buffer that need to be randomized. */
if (config.randomkeys) {
if (from) {
c->randlen = from->randlen;
c->randfree = 0;
c->randptr = zmalloc(sizeof(char*)*c->randlen);
/* copy the offsets. */
for (j = 0; j < c->randlen; j++) {
c->randptr[j] = c->obuf + (from->randptr[j]-from->obuf);
/* Adjust for the different select prefix length. */
c->randptr[j] += c->selectlen - from->selectlen;
}
} else {
char *p = c->obuf;
c->randlen = 0;
c->randfree = RANDPTR_INITIAL_SIZE;
c->randptr = zmalloc(sizeof(char*)*c->randfree);
while ((p = strstr(p,"__rand_int__")) != NULL) {
if (c->randfree == 0) {
c->randptr = zrealloc(c->randptr,sizeof(char*)*c->randlen*2);
c->randfree += c->randlen;
}
c->randptr[c->randlen++] = p;
c->randfree--;
p += 12; /* 12 is strlen("__rand_int__). */
}
char *p = c->obuf;
while ((p = strstr(p,":rand:")) != NULL) {
assert(c->randlen < (signed)(sizeof(c->randptr)/sizeof(char*)));
c->randptr[c->randlen++] = p+6;
p += 6;
}
}
/* redisSetReplyObjectFunctions(c->context,NULL); */
aeCreateFileEvent(config.el,c->context->fd,AE_WRITABLE,writeHandler,c);
listAddNodeTail(config.clients,c);
config.liveclients++;
@@ -389,18 +295,9 @@ static client createClient(char *cmd, size_t len, client from) {
static void createMissingClients(client c) {
int n = 0;
char *buf = c->obuf;
size_t buflen = sdslen(c->obuf);
/* If we are cloning from a client with a SELECT prefix, skip it since the
* client will be created with the prefixed SELECT if needed. */
if (c->selectlen) {
buf += c->selectlen;
buflen -= c->selectlen;
}
while(config.liveclients < config.numclients) {
createClient(NULL,0,c);
createClient(c->obuf,sdslen(c->obuf)/config.pipeline);
/* Listen backlog is quite limited on most systems */
if (++n > 64) {
@@ -451,7 +348,7 @@ static void benchmark(char *title, char *cmd, int len) {
config.requests_issued = 0;
config.requests_finished = 0;
c = createClient(cmd,len,NULL);
c = createClient(cmd,len);
createMissingClients(c);
config.start = mstime();
@@ -523,10 +420,6 @@ int parseOptions(int argc, const char **argv) {
config.tests = sdscat(config.tests,(char*)argv[++i]);
config.tests = sdscat(config.tests,",");
sdstolower(config.tests);
} else if (!strcmp(argv[i],"--dbnum")) {
if (lastarg) goto invalid;
config.dbnum = atoi(argv[++i]);
config.dbnumstr = sdsfromlonglong(config.dbnum);
} else if (!strcmp(argv[i],"--help")) {
exit_status = 0;
goto usage;
@@ -553,14 +446,14 @@ usage:
" -c <clients> Number of parallel connections (default 50)\n"
" -n <requests> Total number of requests (default 10000)\n"
" -d <size> Data size of SET/GET value in bytes (default 2)\n"
" -dbnum <db> SELECT the specified db number (default 0)\n"
" -k <boolean> 1=keep alive 0=reconnect (default 1)\n"
" -r <keyspacelen> Use random keys for SET/GET/INCR, random values for SADD\n"
" Using this option the benchmark will expand the string __rand_int__\n"
" inside an argument with a 12 digits number in the specified range\n"
" from 0 to keyspacelen-1. The substitution changes every time a command\n"
" is executed. Default tests use this to hit random keys in the\n"
" specified range.\n"
" Using this option the benchmark will get/set keys\n"
" in the form mykey_rand:000000012456 instead of constant\n"
" keys, the <keyspacelen> argument determines the max\n"
" number of values for the random number. For instance\n"
" if set to 10 only rand:000000000000 - rand:000000000009\n"
" range will be allowed.\n"
" -P <numreq> Pipeline <numreq> requests. Default 1 (no pipeline).\n"
" -q Quiet. Just show query/sec values\n"
" --csv Output in CSV format\n"
@@ -577,12 +470,8 @@ usage:
" $ redis-benchmark -t set -n 1000000 -r 100000000\n\n"
" Benchmark 127.0.0.1:6379 for a few commands producing CSV output:\n"
" $ redis-benchmark -t ping,set,get -n 100000 --csv\n\n"
" Benchmark a specific command line:\n"
" $ redis-benchmark -r 10000 -n 10000 eval 'return redis.call(\"ping\")' 0\n\n"
" Fill a list with 10000 random elements:\n"
" $ redis-benchmark -r 10000 -n 10000 lpush mylist __rand_int__\n\n"
" On user specified command lines __rand_int__ is replaced with a random integer\n"
" with a range of values selected by the -r option.\n"
" $ redis-benchmark -r 10000 -n 10000 lpush mylist ele:rand:000000000000\n\n"
);
exit(exit_status);
}
@@ -621,7 +510,6 @@ int main(int argc, const char **argv) {
client c;
srandom(time(NULL));
signal(SIGHUP, SIG_IGN);
signal(SIGPIPE, SIG_IGN);
@@ -645,7 +533,6 @@ int main(int argc, const char **argv) {
config.hostport = 6379;
config.hostsocket = NULL;
config.tests = NULL;
config.dbnum = 0;
i = parseOptions(argc,argv);
argc -= i;
@@ -659,7 +546,7 @@ int main(int argc, const char **argv) {
if (config.idlemode) {
printf("Creating %d idle connections and waiting forever (Ctrl+C when done)\n", config.numclients);
c = createClient("",0,NULL); /* will never receive a reply */
c = createClient("",0); /* will never receive a reply */
createMissingClients(c);
aeMain(config.el);
/* and will wait for every */
@@ -698,19 +585,19 @@ int main(int argc, const char **argv) {
}
if (test_is_selected("set")) {
len = redisFormatCommand(&cmd,"SET key:__rand_int__ %s",data);
len = redisFormatCommand(&cmd,"SET foo:rand:000000000000 %s",data);
benchmark("SET",cmd,len);
free(cmd);
}
if (test_is_selected("get")) {
len = redisFormatCommand(&cmd,"GET key:__rand_int__");
len = redisFormatCommand(&cmd,"GET foo:rand:000000000000");
benchmark("GET",cmd,len);
free(cmd);
}
if (test_is_selected("incr")) {
len = redisFormatCommand(&cmd,"INCR counter:__rand_int__");
len = redisFormatCommand(&cmd,"INCR counter:rand:000000000000");
benchmark("INCR",cmd,len);
free(cmd);
}
@@ -729,7 +616,7 @@ int main(int argc, const char **argv) {
if (test_is_selected("sadd")) {
len = redisFormatCommand(&cmd,
"SADD myset element:__rand_int__");
"SADD myset counter:rand:000000000000");
benchmark("SADD",cmd,len);
free(cmd);
}
@@ -779,7 +666,7 @@ int main(int argc, const char **argv) {
const char *argv[21];
argv[0] = "MSET";
for (i = 1; i < 21; i += 2) {
argv[i] = "key:__rand_int__";
argv[i] = "foo:rand:000000000000";
argv[i+1] = data;
}
len = redisFormatCommandArgv(&cmd,21,argv,NULL);
+6 -4
View File
@@ -40,7 +40,6 @@
#include <stdint.h>
#include <limits.h>
#include "lzf.h"
#include "crc64.h"
/* Object types */
#define REDIS_STRING 0
@@ -60,7 +59,7 @@
#define REDIS_ENCODING_RAW 0 /* Raw representation */
#define REDIS_ENCODING_INT 1 /* Encoded as integer */
#define REDIS_ENCODING_ZIPMAP 2 /* Encoded as zipmap */
#define REDIS_ENCODING_HT 3 /* Encoded as a hash table */
#define REDIS_ENCODING_HT 3 /* Encoded as an hash table */
/* Object types only used for dumping to disk */
#define REDIS_EXPIRETIME_MS 252
@@ -79,7 +78,7 @@
* number specify the kind of object that follows.
* See the REDIS_RDB_ENC_* defines.
*
* Lengths up to 63 are stored using a single byte, most DB keys, and may
* Lenghts up to 63 are stored using a single byte, most DB keys, and may
* values, will fit inside. */
#define REDIS_RDB_6BITLEN 0
#define REDIS_RDB_14BITLEN 1
@@ -133,7 +132,7 @@ typedef struct {
char success;
} entry;
/* Global vars that are actually used as constants. The following double
/* Global vars that are actally used as constants. The following double
* values are used for double on-disk serialization, and are initialized
* at runtime to avoid strange compiler optimizations. */
static double R_Zero, R_PosInf, R_NegInf, R_Nan;
@@ -141,6 +140,9 @@ static double R_Zero, R_PosInf, R_NegInf, R_Nan;
/* store string types for output */
static char types[256][16];
/* Prototypes */
uint64_t crc64(uint64_t crc, const unsigned char *s, uint64_t l);
/* Return true if 't' is a valid object type. */
int checkType(unsigned char t) {
/* In case a new object type is added, update the following
+110 -796
View File
File diff suppressed because it is too large Load Diff
-1075
View File
File diff suppressed because it is too large Load Diff
+336 -1125
View File
File diff suppressed because it is too large Load Diff
+70 -348
View File
@@ -67,22 +67,20 @@
#define REDIS_ERR -1
/* Static server configuration */
#define REDIS_DEFAULT_HZ 10 /* Time interrupt calls/sec. */
#define REDIS_MIN_HZ 1
#define REDIS_MAX_HZ 500
#define REDIS_HZ 100 /* Time interrupt calls/sec. */
#define REDIS_SERVERPORT 6379 /* TCP port */
#define REDIS_TCP_BACKLOG 511 /* TCP listen backlog */
#define REDIS_MAXIDLETIME 0 /* default client timeout: infinite */
#define REDIS_DEFAULT_DBNUM 16
#define REDIS_CONFIGLINE_MAX 1024
#define REDIS_DBCRON_DBS_PER_CALL 16
#define REDIS_EXPIRELOOKUPS_PER_CRON 10 /* lookup 10 expires per loop */
#define REDIS_EXPIRELOOKUPS_TIME_PERC 25 /* CPU max % for keys collection */
#define REDIS_MAX_WRITE_PER_EVENT (1024*64)
#define REDIS_SHARED_SELECT_CMDS 10
#define REDIS_SHARED_INTEGERS 10000
#define REDIS_SHARED_BULKHDR_LEN 32
#define REDIS_MAX_LOGMSG_LEN 1024 /* Default maximum length of syslog messages */
#define REDIS_AOF_REWRITE_PERC 100
#define REDIS_AOF_REWRITE_MIN_SIZE (64*1024*1024)
#define REDIS_AOF_REWRITE_MIN_SIZE (1024*1024)
#define REDIS_AOF_REWRITE_ITEMS_PER_CMD 64
#define REDIS_SLOWLOG_LOG_SLOWER_THAN 10000
#define REDIS_SLOWLOG_MAX_LEN 128
@@ -93,45 +91,6 @@
#define REDIS_REPL_PING_SLAVE_PERIOD 10
#define REDIS_RUN_ID_SIZE 40
#define REDIS_OPS_SEC_SAMPLES 16
#define REDIS_DEFAULT_REPL_BACKLOG_SIZE (1024*1024) /* 1mb */
#define REDIS_DEFAULT_REPL_BACKLOG_TIME_LIMIT (60*60) /* 1 hour */
#define REDIS_REPL_BACKLOG_MIN_SIZE (1024*16) /* 16k */
#define REDIS_BGSAVE_RETRY_DELAY 5 /* Wait a few secs before trying again. */
#define REDIS_DEFAULT_PID_FILE "/var/run/redis.pid"
#define REDIS_DEFAULT_SYSLOG_IDENT "redis"
#define REDIS_DEFAULT_CLUSTER_CONFIG_FILE "nodes.conf"
#define REDIS_DEFAULT_DAEMONIZE 0
#define REDIS_DEFAULT_UNIX_SOCKET_PERM 0
#define REDIS_DEFAULT_TCP_KEEPALIVE 0
#define REDIS_DEFAULT_LOGFILE ""
#define REDIS_DEFAULT_SYSLOG_ENABLED 0
#define REDIS_DEFAULT_STOP_WRITES_ON_BGSAVE_ERROR 1
#define REDIS_DEFAULT_RDB_COMPRESSION 1
#define REDIS_DEFAULT_MEM_COMPRESSION 0
#define REDIS_DEFAULT_MEM_COMPRESSION_MAX_SIZE (1024*64)
#define REDIS_DEFAULT_RDB_CHECKSUM 1
#define REDIS_DEFAULT_RDB_FILENAME "dump.rdb"
#define REDIS_DEFAULT_SLAVE_SERVE_STALE_DATA 1
#define REDIS_DEFAULT_SLAVE_READ_ONLY 1
#define REDIS_DEFAULT_REPL_DISABLE_TCP_NODELAY 0
#define REDIS_DEFAULT_MAXMEMORY 0
#define REDIS_DEFAULT_MAXMEMORY_SAMPLES 5
#define REDIS_DEFAULT_AOF_FILENAME "appendonly.aof"
#define REDIS_DEFAULT_AOF_NO_FSYNC_ON_REWRITE 0
#define REDIS_DEFAULT_ACTIVE_REHASHING 1
#define REDIS_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC 1
#define REDIS_DEFAULT_MIN_SLAVES_TO_WRITE 0
#define REDIS_DEFAULT_MIN_SLAVES_MAX_LAG 10
#define REDIS_IP_STR_LEN INET6_ADDRSTRLEN
#define REDIS_PEER_ID_LEN (REDIS_IP_STR_LEN+32) /* Must be enough for ip:port */
#define REDIS_BINDADDR_MAX 16
#define REDIS_MIN_RESERVED_FDS 32
#define ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP 20 /* Loopkups per loop. */
#define ACTIVE_EXPIRE_CYCLE_FAST_DURATION 1000 /* Microseconds */
#define ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC 25 /* CPU max % for keys collection */
#define ACTIVE_EXPIRE_CYCLE_SLOW 0
#define ACTIVE_EXPIRE_CYCLE_FAST 1
/* Protocol and I/O related defines */
#define REDIS_MAX_QUERYBUF_LEN (1024*1024*1024) /* 1GB max query buffer. */
@@ -139,12 +98,6 @@
#define REDIS_REPLY_CHUNK_BYTES (16*1024) /* 16k output buffer */
#define REDIS_INLINE_MAX_SIZE (1024*64) /* Max size of inline reads */
#define REDIS_MBULK_BIG_ARG (1024*32)
#define REDIS_LONGSTR_SIZE 21 /* Bytes needed for long -> str */
#define REDIS_AOF_AUTOSYNC_BYTES (1024*1024*32) /* fdatasync every 32MB */
/* When configuring the Redis eventloop, we setup it so that the total number
* of file descriptors we can handle are server.maxclients + RESERVED_FDS + FDSET_INCR
* that is our safety margin. */
#define REDIS_EVENTLOOP_FDSET_INCR (REDIS_MIN_RESERVED_FDS+96)
/* Hash table parameters */
#define REDIS_HT_MINFILL 10 /* Minimal hash table fill 10% */
@@ -154,7 +107,7 @@
#define REDIS_CMD_WRITE 1 /* "w" flag */
#define REDIS_CMD_READONLY 2 /* "r" flag */
#define REDIS_CMD_DENYOOM 4 /* "m" flag */
#define REDIS_CMD_NOT_USED_1 8 /* no longer used flag */
#define REDIS_CMD_FORCE_REPLICATION 8 /* "f" flag */
#define REDIS_CMD_ADMIN 16 /* "a" flag */
#define REDIS_CMD_PUBSUB 32 /* "p" flag */
#define REDIS_CMD_NOSCRIPT 64 /* "s" flag */
@@ -163,7 +116,6 @@
#define REDIS_CMD_LOADING 512 /* "l" flag */
#define REDIS_CMD_STALE 1024 /* "t" flag */
#define REDIS_CMD_SKIP_MONITOR 2048 /* "M" flag */
#define REDIS_CMD_ASKING 4096 /* "k" flag */
/* Object types */
#define REDIS_STRING 0
@@ -174,7 +126,7 @@
/* Objects encoding. Some kind of objects like Strings and Hashes can be
* internally represented in multiple ways. The 'encoding' field of the object
* is set to one of this values. */
* is set to one of this fields for this object. */
#define REDIS_ENCODING_RAW 0 /* Raw representation */
#define REDIS_ENCODING_INT 1 /* Encoded as integer */
#define REDIS_ENCODING_HT 2 /* Encoded as hash table */
@@ -183,8 +135,6 @@
#define REDIS_ENCODING_ZIPLIST 5 /* Encoded as ziplist */
#define REDIS_ENCODING_INTSET 6 /* Encoded as intset */
#define REDIS_ENCODING_SKIPLIST 7 /* Encoded as skiplist */
#define REDIS_ENCODING_EMBSTR 8 /* Embedded sds string encoding */
#define REDIS_ENCODING_LZF 9 /* LZF compressed string. */
/* Defines related to the dump file format. To store 32 bits lengths for short
* keys requires a lot of space, so we check the most significant 2 bits of
@@ -192,12 +142,12 @@
*
* 00|000000 => if the two MSB are 00 the len is the 6 bits of this byte
* 01|000000 00000000 => 01, the len is 14 byes, 6 bits + 8 bits of next byte
* 10|000000 [32 bit integer] => if it's 10, a full 32 bit len will follow
* 10|000000 [32 bit integer] => if it's 01, a full 32 bit len will follow
* 11|000000 this means: specially encoded object will follow. The six bits
* number specify the kind of object that follows.
* See the REDIS_RDB_ENC_* defines.
*
* Lengths up to 63 are stored using a single byte, most DB keys, and may
* Lenghts up to 63 are stored using a single byte, most DB keys, and may
* values, will fit inside. */
#define REDIS_RDB_6BITLEN 0
#define REDIS_RDB_14BITLEN 1
@@ -233,17 +183,6 @@
#define REDIS_CLOSE_ASAP (1<<10)/* Close this client ASAP */
#define REDIS_UNIX_SOCKET (1<<11) /* Client connected via Unix domain socket */
#define REDIS_DIRTY_EXEC (1<<12) /* EXEC will fail for errors while queueing */
#define REDIS_MASTER_FORCE_REPLY (1<<13) /* Queue replies even if is master */
#define REDIS_FORCE_AOF (1<<14) /* Force AOF propagation of current cmd. */
#define REDIS_FORCE_REPL (1<<15) /* Force replication of current cmd. */
#define REDIS_PRE_PSYNC (1<<16) /* Instance don't understand PSYNC. */
#define REDIS_READONLY (1<<17) /* Cluster client is in read-only state. */
/* Client block type (btype field in client structure)
* if REDIS_BLOCKED flag is set. */
#define REDIS_BLOCKED_NONE 0 /* Not blocked, no REDIS_BLOCKED flag set. */
#define REDIS_BLOCKED_LIST 1 /* BLPOP & co. */
#define REDIS_BLOCKED_WAIT 2 /* WAIT for synchronous replication. */
/* Client request types */
#define REDIS_REQ_INLINE 1
@@ -256,7 +195,7 @@
#define REDIS_CLIENT_LIMIT_CLASS_PUBSUB 2
#define REDIS_CLIENT_LIMIT_NUM_CLASSES 3
/* Slave replication state - from the point of view of the slave. */
/* Slave replication state - slave side */
#define REDIS_REPL_NONE 0 /* No active replication */
#define REDIS_REPL_CONNECT 1 /* Must connect to master */
#define REDIS_REPL_CONNECTING 2 /* Connecting to master */
@@ -264,18 +203,18 @@
#define REDIS_REPL_TRANSFER 4 /* Receiving .rdb from master */
#define REDIS_REPL_CONNECTED 5 /* Connected to master */
/* Slave replication state - from the point of view of the master.
* In SEND_BULK and ONLINE state the slave receives new updates
* in its output queue. In the WAIT_BGSAVE state instead the server is waiting
* to start the next background saving in order to send updates to it. */
#define REDIS_REPL_WAIT_BGSAVE_START 6 /* We need to produce a new RDB file. */
#define REDIS_REPL_WAIT_BGSAVE_END 7 /* Waiting RDB file creation to finish. */
#define REDIS_REPL_SEND_BULK 8 /* Sending RDB file to slave. */
#define REDIS_REPL_ONLINE 9 /* RDB file transmitted, sending just updates. */
/* Synchronous read timeout - slave side */
#define REDIS_REPL_SYNCIO_TIMEOUT 5
/* Slave replication state - from the point of view of master
* Note that in SEND_BULK and ONLINE state the slave receives new updates
* in its output queue. In the WAIT_BGSAVE state instead the server is waiting
* to start the next background saving in order to send updates to it. */
#define REDIS_REPL_WAIT_BGSAVE_START 3 /* master waits bgsave to start feeding it */
#define REDIS_REPL_WAIT_BGSAVE_END 4 /* master waits bgsave to start bulk DB transmission */
#define REDIS_REPL_SEND_BULK 5 /* master is sending the bulk DB */
#define REDIS_REPL_ONLINE 6 /* bulk DB already transmitted, receive updates */
/* List related stuff */
#define REDIS_HEAD 0
#define REDIS_TAIL 1
@@ -292,7 +231,6 @@
#define REDIS_NOTICE 2
#define REDIS_WARNING 3
#define REDIS_LOG_RAW (1<<10) /* Modifier to log without timestamp */
#define REDIS_DEFAULT_VERBOSITY REDIS_NOTICE
/* Anti-warning macro... */
#define REDIS_NOTUSED(V) ((void) V)
@@ -304,7 +242,6 @@
#define AOF_FSYNC_NO 0
#define AOF_FSYNC_ALWAYS 1
#define AOF_FSYNC_EVERYSEC 2
#define REDIS_DEFAULT_AOF_FSYNC AOF_FSYNC_EVERYSEC
/* Zip structure related defaults */
#define REDIS_HASH_MAX_ZIPLIST_ENTRIES 512
@@ -327,7 +264,6 @@
#define REDIS_MAXMEMORY_ALLKEYS_LRU 3
#define REDIS_MAXMEMORY_ALLKEYS_RANDOM 4
#define REDIS_MAXMEMORY_NO_EVICTION 5
#define REDIS_DEFAULT_MAXMEMORY_POLICY REDIS_MAXMEMORY_NO_EVICTION
/* Scripting */
#define REDIS_LUA_TIME_LIMIT 5000 /* milliseconds */
@@ -353,24 +289,10 @@
#define REDIS_PROPAGATE_AOF 1
#define REDIS_PROPAGATE_REPL 2
/* Keyspace changes notification classes. Every class is associated with a
* character for configuration purposes. */
#define REDIS_NOTIFY_KEYSPACE (1<<0) /* K */
#define REDIS_NOTIFY_KEYEVENT (1<<1) /* E */
#define REDIS_NOTIFY_GENERIC (1<<2) /* g */
#define REDIS_NOTIFY_STRING (1<<3) /* $ */
#define REDIS_NOTIFY_LIST (1<<4) /* l */
#define REDIS_NOTIFY_SET (1<<5) /* s */
#define REDIS_NOTIFY_HASH (1<<6) /* h */
#define REDIS_NOTIFY_ZSET (1<<7) /* z */
#define REDIS_NOTIFY_EXPIRED (1<<8) /* x */
#define REDIS_NOTIFY_EVICTED (1<<9) /* e */
#define REDIS_NOTIFY_ALL (REDIS_NOTIFY_GENERIC | REDIS_NOTIFY_STRING | REDIS_NOTIFY_LIST | REDIS_NOTIFY_SET | REDIS_NOTIFY_HASH | REDIS_NOTIFY_ZSET | REDIS_NOTIFY_EXPIRED | REDIS_NOTIFY_EVICTED) /* A */
/* Using the following macro you can run code inside serverCron() with the
* specified period, specified in milliseconds.
* The actual resolution depends on server.hz. */
#define run_with_period(_ms_) if ((_ms_ <= 1000/server.hz) || !(server.cronloops%((_ms_)/(1000/server.hz))))
* The actual resolution depends on REDIS_HZ. */
#define run_with_period(_ms_) if (!(server.cronloops%((_ms_)/(1000/REDIS_HZ))))
/* We can print the stacktrace, so our assert is defined this way: */
#define redisAssertWithInfo(_c,_o,_e) ((_e)?(void)0 : (_redisAssertWithInfo(_c,_o,#_e,__FILE__,__LINE__),_exit(1)))
@@ -381,29 +303,21 @@
* Data types
*----------------------------------------------------------------------------*/
typedef long long mstime_t; /* millisecond time type. */
/* A redis object, that is a type able to hold a string / list / set */
/* The actual Redis Object */
#define REDIS_LRU_BITS 24
#define REDIS_LRU_CLOCK_MAX ((1<<REDIS_LRU_BITS)-1) /* Max value of obj->lru */
#define REDIS_LRU_CLOCK_RESOLUTION 1000 /* LRU clock resolution in ms */
#define REDIS_LRU_CLOCK_MAX ((1<<21)-1) /* Max value of obj->lru */
#define REDIS_LRU_CLOCK_RESOLUTION 10 /* LRU clock resolution in seconds */
typedef struct redisObject {
unsigned type:4;
unsigned notused:2; /* Not used */
unsigned encoding:4;
unsigned lru:REDIS_LRU_BITS; /* lru time (relative to server.lruclock) */
unsigned lru:22; /* lru time (relative to server.lruclock) */
int refcount;
void *ptr;
} robj;
/* Macro used to obtain the current LRU clock.
* If the current resolution is lower than the frequency we refresh the
* LRU clock (as it should be in production servers) we return the
* precomputed value, otherwise we need to resort to a function call. */
#define LRU_CLOCK() ((1000/server.hz <= REDIS_LRU_CLOCK_RESOLUTION) ? server.lruclock : getLRUClock())
/* Macro used to initialize a Redis object allocated on the stack.
/* Macro used to initalize a Redis object allocated on the stack.
* Note that this macro is taken near the structure definition to make sure
* we'll update it when the structure is changed, to avoid bugs like
* bug #85 introduced exactly in this way. */
@@ -414,31 +328,13 @@ typedef struct redisObject {
_var.ptr = _ptr; \
} while(0);
/* To improve the quality of the LRU approximation we take a set of keys
* that are good candidate for eviction across freeMemoryIfNeeded() calls.
*
* Entries inside the eviciton pool are taken ordered by idle time, putting
* greater idle times to the right (ascending order).
*
* Empty entries have the key pointer set to NULL. */
#define REDIS_EVICTION_POOL_SIZE 16
struct evictionPoolEntry {
unsigned long long idle; /* Object idle time. */
sds key; /* Key name. */
};
/* Redis database representation. There are multiple databases identified
* by integers from 0 (the default database) up to the max configured
* database. The database number is the 'id' field in the structure. */
typedef struct redisDb {
dict *dict; /* The keyspace for this DB */
dict *expires; /* Timeout of keys with a timeout set */
dict *blocking_keys; /* Keys with clients waiting for data (BLPOP) */
dict *ready_keys; /* Blocked keys that received a PUSH */
dict *watched_keys; /* WATCHED keys for MULTI/EXEC CAS */
struct evictionPoolEntry *eviction_pool; /* Eviction pool of keys */
int id; /* Database ID */
long long avg_ttl; /* Average TTL, just for stats */
int id;
} redisDb;
/* Client MULTI/EXEC state */
@@ -451,26 +347,16 @@ typedef struct multiCmd {
typedef struct multiState {
multiCmd *commands; /* Array of MULTI commands */
int count; /* Total number of MULTI commands */
int minreplicas; /* MINREPLICAS for synchronous replication */
time_t minreplicas_timeout; /* MINREPLICAS timeout as unixtime. */
} multiState;
/* This structure holds the blocking operation state for a client.
* The fields used depend on client->btype. */
typedef struct blockingState {
/* Generic fields. */
mstime_t timeout; /* Blocking operation timeout. If UNIX current time
* is > timeout then the operation timed out. */
/* REDIS_BLOCK_LIST */
dict *keys; /* The keys we are waiting to terminate a blocking
robj **keys; /* The key we are waiting to terminate a blocking
* operation such as BLPOP. Otherwise NULL. */
int count; /* Number of blocking keys */
time_t timeout; /* Blocking operation timeout. If UNIX current time
* is >= timeout then the operation timed out. */
robj *target; /* The key that should receive the element,
* for BRPOPLPUSH. */
/* REDIS_BLOCK_WAIT */
int numreplicas; /* Number of replicas we are waiting for ACK. */
long long reploffset; /* Replication offset to reach. */
} blockingState;
/* The following structure represents a node in the server.ready_keys list,
@@ -489,13 +375,12 @@ typedef struct readyList {
robj *key;
} readyList;
/* With multiplexing we need to take per-client state.
/* With multiplexing we need to take per-clinet state.
* Clients are taken in a liked list. */
typedef struct redisClient {
int fd;
redisDb *db;
int dictid;
robj *name; /* As set by CLIENT SETNAME */
sds querybuf;
size_t querybuf_peak; /* Recent (100ms or more) peak of querybuf size */
int argc;
@@ -506,27 +391,22 @@ typedef struct redisClient {
long bulklen; /* length of bulk argument in multi bulk request */
list *reply;
unsigned long reply_bytes; /* Tot bytes of objects in reply list */
int sentlen; /* Amount of bytes already sent in the current
buffer or object being sent. */
int sentlen;
time_t ctime; /* Client creation time */
time_t lastinteraction; /* time of the last interaction, used for timeout */
time_t obuf_soft_limit_reached_time;
int flags; /* REDIS_SLAVE | REDIS_MONITOR | REDIS_MULTI ... */
int slaveseldb; /* slave selected db, if this client is a slave */
int authenticated; /* when requirepass is non-NULL */
int replstate; /* replication state if this is a slave */
int repldbfd; /* replication DB file descriptor */
off_t repldboff; /* replication DB file offset */
long repldboff; /* replication DB file offset */
off_t repldbsize; /* replication DB file size */
sds replpreamble; /* replication DB preamble. */
long long reploff; /* replication offset if this is our master */
long long repl_ack_off; /* replication ack offset, if this is a slave */
long long repl_ack_time;/* replication ack time, if this is a slave */
char replrunid[REDIS_RUN_ID_SIZE+1]; /* master run id if this is a master */
int slave_listening_port; /* As configured with: SLAVECONF listening-port */
multiState mstate; /* MULTI/EXEC state */
int btype; /* Type of blocking op if REDIS_BLOCKED. */
blockingState bpop; /* blocking state */
long long woff; /* Last write global replication offset. */
blockingState bpop; /* blocking state */
list *io_keys; /* Keys this client is waiting to be loaded from the
* swap file in order to continue. */
list *watched_keys; /* Keys WATCHED for MULTI/EXEC CAS */
dict *pubsub_channels; /* channels a client is interested in (SUBSCRIBE) */
list *pubsub_patterns; /* patterns a client is interested in (SUBSCRIBE) */
@@ -546,10 +426,10 @@ struct sharedObjectsStruct {
*colon, *nullbulk, *nullmultibulk, *queued,
*emptymultibulk, *wrongtypeerr, *nokeyerr, *syntaxerr, *sameobjecterr,
*outofrangeerr, *noscripterr, *loadingerr, *slowscripterr, *bgsaveerr,
*masterdownerr, *roslaveerr, *execaborterr, *noautherr, *noreplicaserr,
*masterdownerr, *roslaveerr, *execaborterr,
*oomerr, *plus, *messagebulk, *pmessagebulk, *subscribebulk,
*unsubscribebulk, *psubscribebulk, *punsubscribebulk, *del, *rpop, *lpop,
*lpush, *emptyscan,
*lpush,
*select[REDIS_SHARED_SELECT_CMDS],
*integers[REDIS_SHARED_INTEGERS],
*mbulkhdr[REDIS_SHARED_BULKHDR_LEN], /* "*<value>\r\n" */
@@ -584,8 +464,6 @@ typedef struct clientBufferLimitsConfig {
time_t soft_limit_seconds;
} clientBufferLimitsConfig;
extern clientBufferLimitsConfig clientBufferLimitsDefaults[REDIS_CLIENT_LIMIT_NUM_CLASSES];
/* The redisOp structure defines a Redis Operation, that is an instance of
* a command with an argument vector, database ID, propagation target
* (REDIS_PROPAGATE_*), and command pointer.
@@ -614,17 +492,13 @@ typedef struct redisOpArray {
* Global server state
*----------------------------------------------------------------------------*/
struct clusterState;
struct redisServer {
/* General */
char *configfile; /* Absolute config file path, or NULL */
int hz; /* serverCron() calls frequency in hertz */
redisDb *db;
dict *commands; /* Command table */
dict *orig_commands; /* Command table before command renaming. */
dict *commands; /* Command table hash table */
aeEventLoop *el;
unsigned lruclock:REDIS_LRU_BITS; /* Clock for LRU eviction */
unsigned lruclock:22; /* Clock incrementing every minute, for LRU */
unsigned lruclock_padding:10;
int shutdown_asap; /* SHUTDOWN needed ASAP */
int activerehashing; /* Incremental rehash in serverCron() */
char *requirepass; /* Pass for AUTH command, or NULL */
@@ -635,30 +509,21 @@ struct redisServer {
int sentinel_mode; /* True if this instance is a Sentinel. */
/* Networking */
int port; /* TCP listening port */
int tcp_backlog; /* TCP listen() backlog */
char *bindaddr[REDIS_BINDADDR_MAX]; /* Addresses we should bind to */
int bindaddr_count; /* Number of addresses in server.bindaddr[] */
char *bindaddr; /* Bind address or NULL */
char *unixsocket; /* UNIX socket path */
mode_t unixsocketperm; /* UNIX socket permission */
int ipfd[REDIS_BINDADDR_MAX]; /* TCP socket file descriptors */
int ipfd_count; /* Used slots in ipfd[] */
int ipfd; /* TCP socket file descriptor */
int sofd; /* Unix socket file descriptor */
int cfd[REDIS_BINDADDR_MAX];/* Cluster bus listening socket */
int cfd_count; /* Used slots in cfd[] */
list *clients; /* List of active clients */
list *clients_to_close; /* Clients to close asynchronously */
list *slaves, *monitors; /* List of slaves and MONITORs */
redisClient *current_client; /* Current client, only used on crash report */
int clients_paused; /* True if clients are currently paused */
mstime_t clients_pause_end_time; /* Time when we undo clients_paused */
char neterr[ANET_ERR_LEN]; /* Error buffer for anet.c */
dict *migrate_cached_sockets;/* MIGRATE cached sockets */
char neterr[ANET_ERR_LEN]; /* Error buffer for anet.c */
/* RDB / AOF loading information */
int loading; /* We are loading data from disk if true */
off_t loading_total_bytes;
off_t loading_loaded_bytes;
time_t loading_start_time;
off_t loading_process_events_interval_bytes;
/* Fast pointers to often looked up command */
struct redisCommand *delCommand, *multiCommand, *lpushCommand, *lpopCommand,
*rpopCommand;
@@ -671,16 +536,12 @@ struct redisServer {
long long stat_keyspace_hits; /* Number of successful lookups of keys */
long long stat_keyspace_misses; /* Number of failed lookups of keys */
size_t stat_peak_memory; /* Max used memory record */
long long stat_fork_time; /* Time needed to perform latest fork() */
long long stat_fork_time; /* Time needed to perform latets fork() */
long long stat_rejected_conn; /* Clients rejected because of maxclients */
long long stat_sync_full; /* Number of full resyncs with slaves. */
long long stat_sync_partial_ok; /* Number of accepted PSYNC requests. */
long long stat_sync_partial_err;/* Number of unaccepted PSYNC requests. */
list *slowlog; /* SLOWLOG list of commands */
long long slowlog_entry_id; /* SLOWLOG current entry ID */
long long slowlog_log_slower_than; /* SLOWLOG time limit (to get logged) */
unsigned long slowlog_max_len; /* SLOWLOG max number of items logged */
size_t resident_set_size; /* RSS sampled in serverCron(). */
/* The following two are used to track instantaneous "load" in terms
* of operations per second. */
long long ops_sec_last_sample_time; /* Timestamp of last sample (in ms) */
@@ -690,8 +551,6 @@ struct redisServer {
/* Configuration */
int verbosity; /* Loglevel in redis.conf */
int maxidletime; /* Client timeout in seconds */
int tcpkeepalive; /* Set SO_KEEPALIVE if non-zero. */
int active_expire_enabled; /* Can be disabled for testing purposes. */
size_t client_max_querybuf_len; /* Limit for client query buffer length */
int dbnum; /* Total number of configured DBs */
int daemonize; /* True if running as a daemon */
@@ -717,9 +576,6 @@ struct redisServer {
time_t aof_rewrite_time_start; /* Current AOF rewrite start time. */
int aof_lastbgrewrite_status; /* REDIS_OK or REDIS_ERR */
unsigned long aof_delayed_fsync; /* delayed AOF fsync() counter */
int aof_rewrite_incremental_fsync;/* fsync incrementally while rewriting? */
int aof_last_write_status; /* REDIS_OK or REDIS_ERR */
int aof_last_write_errno; /* Valid if aof_last_write_status is ERR */
/* RDB persistence */
long long dirty; /* Changes to DB from the last save */
long long dirty_before_bgsave; /* Used to restore dirty on failed BGSAVE */
@@ -729,8 +585,7 @@ struct redisServer {
char *rdb_filename; /* Name of RDB file */
int rdb_compression; /* Use compression in RDB? */
int rdb_checksum; /* Use RDB checksum? */
time_t lastsave; /* Unix time of last successful save */
time_t lastbgsave_try; /* Unix time of last attempted bgsave */
time_t lastsave; /* Unix time of last save succeeede */
time_t rdb_save_time_last; /* Time used by last RDB save run. */
time_t rdb_save_time_start; /* Current RDB save start time. */
int lastbgsave_status; /* REDIS_OK or REDIS_ERR */
@@ -742,30 +597,13 @@ struct redisServer {
int syslog_enabled; /* Is syslog enabled? */
char *syslog_ident; /* Syslog ident */
int syslog_facility; /* Syslog facility */
/* Replication (master) */
int slaveseldb; /* Last SELECTed DB in replication output */
long long master_repl_offset; /* Global replication offset */
int repl_ping_slave_period; /* Master pings the slave every N seconds */
char *repl_backlog; /* Replication backlog for partial syncs */
long long repl_backlog_size; /* Backlog circular buffer size */
long long repl_backlog_histlen; /* Backlog actual data length */
long long repl_backlog_idx; /* Backlog circular buffer current offset */
long long repl_backlog_off; /* Replication offset of first byte in the
backlog buffer. */
time_t repl_backlog_time_limit; /* Time without slaves after the backlog
gets released. */
time_t repl_no_slaves_since; /* We have no slaves since that time.
Only valid if server.slaves len is 0. */
int repl_min_slaves_to_write; /* Min number of slaves to write. */
int repl_min_slaves_max_lag; /* Max lag of <count> slaves to write. */
int repl_good_slaves_count; /* Number of slaves with lag <= max_lag. */
/* Replication (slave) */
/* Slave specific fields */
char *masterauth; /* AUTH with this password with master */
char *masterhost; /* Hostname of master */
int masterport; /* Port of master */
int repl_ping_slave_period; /* Master pings the slave every N seconds */
int repl_timeout; /* Timeout after N seconds of master idle */
redisClient *master; /* Client that is master for this slave */
redisClient *cached_master; /* Cached master to be reused for PSYNC. */
int repl_syncio_timeout; /* Timeout for synchronous I/O calls */
int repl_state; /* Replication status if the instance is a slave */
off_t repl_transfer_size; /* Size of RDB to read from master during sync. */
@@ -778,21 +616,11 @@ struct redisServer {
int repl_serve_stale_data; /* Serve stale data when link is down? */
int repl_slave_ro; /* Slave is read only? */
time_t repl_down_since; /* Unix time at which link with master went down */
int repl_disable_tcp_nodelay; /* Disable TCP_NODELAY after SYNC? */
int slave_priority; /* Reported in INFO and used by Sentinel. */
char repl_master_runid[REDIS_RUN_ID_SIZE+1]; /* Master run id for PSYNC. */
long long repl_master_initial_offset; /* Master PSYNC offset. */
/* Replication script cache. */
dict *repl_scriptcache_dict; /* SHA1 all slaves are aware of. */
list *repl_scriptcache_fifo; /* First in, first out LRU eviction. */
int repl_scriptcache_size; /* Max number of elements. */
/* Synchronous replication. */
list *clients_waiting_acks; /* Clients waiting in WAIT command. */
int get_ack_from_slaves; /* If true we send REPLCONF GETACK. */
/* Limits */
int maxclients; /* Max number of simultaneous clients */
unsigned int maxclients; /* Max number of simultaneous clients */
unsigned long long maxmemory; /* Max number of memory bytes to use */
int maxmemory_policy; /* Policy for key eviction */
int maxmemory_policy; /* Policy for key evition */
int maxmemory_samples; /* Pricision of random sampling */
/* Blocked clients */
unsigned int bpop_blocked_clients; /* Number of clients blocked by lists */
@@ -803,7 +631,6 @@ struct redisServer {
int sort_desc;
int sort_alpha;
int sort_bypattern;
int sort_store;
/* Zip structure config, see redis.conf for more information */
size_t hash_max_ziplist_entries;
size_t hash_max_ziplist_value;
@@ -812,28 +639,17 @@ struct redisServer {
size_t set_max_intset_entries;
size_t zset_max_ziplist_entries;
size_t zset_max_ziplist_value;
int mem_compression; /* In memory LZF compression. */
size_t mem_compression_max_size; /* Try to compress up to this size. */
time_t unixtime; /* Unix time sampled every cron cycle. */
long long mstime; /* Like unixtime but in milliseconds. */
time_t unixtime; /* Unix time sampled every second. */
/* Pubsub */
dict *pubsub_channels; /* Map channels to list of subscribed clients */
list *pubsub_patterns; /* A list of pubsub_patterns */
int notify_keyspace_events; /* Events to propagate via Pub/Sub. This is an
xor of REDIS_NOTIFY... flags. */
/* Cluster */
int cluster_enabled; /* Is cluster enabled? */
mstime_t cluster_node_timeout; /* Cluster node timeout. */
char *cluster_configfile; /* Cluster auto-generated config file name. */
struct clusterState *cluster; /* State of the cluster */
int cluster_migration_barrier; /* Cluster replicas migration barrier. */
/* Scripting */
lua_State *lua; /* The Lua interpreter. We use just one for all clients */
redisClient *lua_client; /* The "fake client" to query Redis from Lua */
redisClient *lua_caller; /* The client running EVAL right now, or NULL */
dict *lua_scripts; /* A dictionary of SHA1 -> Lua scripts */
mstime_t lua_time_limit; /* Script timeout in milliseconds */
mstime_t lua_time_start; /* Start time of script, milliseconds time */
long long lua_time_limit; /* Script timeout in seconds */
long long lua_time_start; /* Start time of script */
int lua_write_dirty; /* True if a write command was called during the
execution of the current script. */
int lua_random_dirty; /* True if a random command was called during the
@@ -841,7 +657,7 @@ struct redisServer {
int lua_timedout; /* True if we reached the time limit for script
execution. */
int lua_kill; /* Kill the script if true. */
/* Assert & bug reporting */
/* Assert & bug reportign */
char *assert_failed;
char *assert_file;
int assert_line;
@@ -855,19 +671,18 @@ typedef struct pubsubPattern {
} pubsubPattern;
typedef void redisCommandProc(redisClient *c);
typedef int *redisGetKeysProc(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
typedef int *redisGetKeysProc(struct redisCommand *cmd, robj **argv, int argc, int *numkeys, int flags);
struct redisCommand {
char *name;
redisCommandProc *proc;
int arity;
char *sflags; /* Flags as string representation, one char per flag. */
char *sflags; /* Flags as string represenation, one char per flag. */
int flags; /* The actual flags, obtained from the 'sflags' field. */
/* Use a function to determine keys arguments in a command line.
* Used for Redis Cluster redirect. */
/* Use a function to determine keys arguments in a command line. */
redisGetKeysProc *getkeys_proc;
/* What keys should be loaded in background when calling this command? */
int firstkey; /* The first argument that's a key (0 = no keys) */
int lastkey; /* The last argument that's a key */
int lastkey; /* THe last argument that's a key */
int keystep; /* The step between first and last key */
long long microseconds, calls;
};
@@ -914,7 +729,7 @@ typedef struct {
dictIterator *di;
} setTypeIterator;
/* Structure to hold hash iteration abstraction. Note that iteration over
/* Structure to hold hash iteration abstration. Note that iteration over
* hashes involves both fields and values. Because it is possible that
* not both are required, store pointers in the iterator to avoid
* unnecessary memory allocation for fields/values. */
@@ -939,13 +754,10 @@ extern struct redisServer server;
extern struct sharedObjectsStruct shared;
extern dictType setDictType;
extern dictType zsetDictType;
extern dictType clusterNodesDictType;
extern dictType clusterNodesBlackListDictType;
extern dictType dbDictType;
extern dictType shaScriptObjectDictType;
extern double R_Zero, R_PosInf, R_NegInf, R_Nan;
extern dictType hashDictType;
extern dictType replScriptCacheDictType;
/*-----------------------------------------------------------------------------
* Functions prototypes
@@ -957,14 +769,11 @@ long long mstime(void);
void getRandomHexChars(char *p, unsigned int len);
uint64_t crc64(uint64_t crc, const unsigned char *s, uint64_t l);
void exitFromChild(int retcode);
size_t redisPopcount(void *s, long count);
void redisSetProcTitle(char *title);
/* networking.c -- Networking and Client related operations */
redisClient *createClient(int fd);
void closeTimedoutClients(void);
void freeClient(redisClient *c);
void freeClientAsync(redisClient *c);
void resetClient(redisClient *c);
void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask);
void addReply(redisClient *c, robj *obj);
@@ -991,8 +800,6 @@ void copyClientOutputBuffer(redisClient *dst, redisClient *src);
void *dupClientReplyValue(void *o);
void getClientsMaxBuffers(unsigned long *longest_output_list,
unsigned long *biggest_input_buffer);
void formatPeerId(char *peerid, size_t peerid_len, char *ip, int port);
int getClientPeerId(redisClient *client, char *peerid, size_t peerid_len);
sds getClientInfoString(redisClient *client);
sds getAllClientsInfoString(void);
void rewriteClientCommandVector(redisClient *c, int argc, ...);
@@ -1004,9 +811,6 @@ int getClientLimitClassByName(char *name);
char *getClientLimitClassName(int class);
void flushSlavesOutputBuffers(void);
void disconnectSlaves(void);
int listenToPort(int port, int *fds, int *count);
void pauseClients(mstime_t duration);
int clientsArePaused(void);
#ifdef __GNUC__
void addReplyErrorFormat(redisClient *c, const char *fmt, ...)
@@ -1046,8 +850,7 @@ void discardTransaction(redisClient *c);
void flagTransaction(redisClient *c);
/* Redis object implementation */
void decrRefCount(robj *o);
void decrRefCountVoid(void *o);
void decrRefCount(void *o);
void incrRefCount(robj *o);
robj *resetRefCount(robj *obj);
void freeStringObject(robj *o);
@@ -1057,8 +860,6 @@ void freeZsetObject(robj *o);
void freeHashObject(robj *o);
robj *createObject(int type, void *ptr);
robj *createStringObject(char *ptr, size_t len);
robj *createRawStringObject(char *ptr, size_t len);
robj *createEmbeddedStringObject(char *ptr, size_t len);
robj *dupStringObject(robj *o);
int isObjectRepresentableAsLongLong(robj *o, long long *llongval);
robj *tryObjectEncoding(robj *o);
@@ -1082,13 +883,8 @@ int getLongDoubleFromObject(robj *o, long double *target);
int getLongDoubleFromObjectOrReply(redisClient *c, robj *o, long double *target, const char *msg);
char *strEncoding(int encoding);
int compareStringObjects(robj *a, robj *b);
int collateStringObjects(robj *a, robj *b);
int equalStringObjects(robj *a, robj *b);
unsigned long long estimateObjectIdleTime(robj *o);
#define rawEncodedObject(objptr) (objptr->encoding == REDIS_ENCODING_RAW)
#define lzfEncodedObject(objptr) (objptr->encoding == REDIS_ENCODING_LZF)
#define intEncodedObject(objptr) (objptr->encoding == REDIS_ENCODING_INT)
#define sdsEncodedObject(objptr) (objptr->encoding == REDIS_ENCODING_RAW || objptr->encoding == REDIS_ENCODING_EMBSTR)
unsigned long estimateObjectIdleTime(robj *o);
/* Synchronous I/O with timeout */
ssize_t syncWrite(int fd, char *ptr, ssize_t size, long long timeout);
@@ -1100,21 +896,6 @@ void replicationFeedSlaves(list *slaves, int dictid, robj **argv, int argc);
void replicationFeedMonitors(redisClient *c, list *monitors, int dictid, robj **argv, int argc);
void updateSlavesWaitingBgsave(int bgsaveerr);
void replicationCron(void);
void replicationHandleMasterDisconnection(void);
void replicationCacheMaster(redisClient *c);
void resizeReplicationBacklog(long long newsize);
void replicationSetMaster(char *ip, int port);
void replicationUnsetMaster(void);
void refreshGoodSlavesCount(void);
void replicationScriptCacheInit(void);
void replicationScriptCacheFlush(void);
void replicationScriptCacheAdd(sds sha1);
int replicationScriptCacheExists(sds sha1);
void processClientsWaitingReplicas(void);
void unblockClientWaitingReplicas(redisClient *c);
int replicationCountAcksByOffset(long long offset);
void replicationSendNewlineToMaster(void);
long long replicationGetSlaveOffset(void);
/* Generic persistence functions */
void startLoading(FILE *fp);
@@ -1150,13 +931,11 @@ zskiplistNode *zslInsert(zskiplist *zsl, double score, robj *obj);
unsigned char *zzlInsert(unsigned char *zl, robj *ele, double score);
int zslDelete(zskiplist *zsl, double score, robj *obj);
zskiplistNode *zslFirstInRange(zskiplist *zsl, zrangespec range);
zskiplistNode *zslLastInRange(zskiplist *zsl, zrangespec range);
double zzlGetScore(unsigned char *sptr);
void zzlNext(unsigned char *zl, unsigned char **eptr, unsigned char **sptr);
void zzlPrev(unsigned char *zl, unsigned char **eptr, unsigned char **sptr);
unsigned int zsetLength(robj *zobj);
void zsetConvert(robj *zobj, int encoding);
unsigned long zslGetRank(zskiplist *zsl, double score, robj *o);
/* Core functions */
int freeMemoryIfNeeded(void);
@@ -1164,18 +943,11 @@ int processCommand(redisClient *c);
void setupSignalHandlers(void);
struct redisCommand *lookupCommand(sds name);
struct redisCommand *lookupCommandByCString(char *s);
struct redisCommand *lookupCommandOrOriginal(sds name);
void call(redisClient *c, int flags);
void propagate(struct redisCommand *cmd, int dbid, robj **argv, int argc, int flags);
void alsoPropagate(struct redisCommand *cmd, int dbid, robj **argv, int argc, int target);
void forceCommandPropagation(redisClient *c, int flags);
int prepareForShutdown();
#ifdef __GNUC__
void redisLog(int level, const char *fmt, ...)
__attribute__((format(printf, 2, 3)));
#else
void redisLog(int level, const char *fmt, ...);
#endif
void redisLogRaw(int level, const char *msg);
void redisLogFromHandler(int level, const char *msg);
void usage();
@@ -1184,11 +956,6 @@ int htNeedsResize(dict *dict);
void oom(const char *msg);
void populateCommandTable(void);
void resetCommandTableStats(void);
void adjustOpenFilesLimit(void);
void closeListeningSockets(int unlink_unix_socket);
void updateCachedTime(void);
void resetServerStats(void);
unsigned int getLRUClock(void);
/* Set data type */
robj *setTypeCreate(robj *value);
@@ -1230,18 +997,10 @@ void freePubsubPattern(void *p);
int listMatchPubsubPattern(void *a, void *b);
int pubsubPublishMessage(robj *channel, robj *message);
/* Keyspace events notification */
void notifyKeyspaceEvent(int type, char *event, robj *key, int dbid);
int keyspaceEventsStringToFlags(char *classes);
sds keyspaceEventsFlagsToString(int flags);
/* Configuration */
void loadServerConfig(char *filename, char *options);
void appendServerSaveParams(time_t seconds, int changes);
void resetServerSaveParams();
struct rewriteConfigState; /* Forward declaration to export API. */
void rewriteConfigRewriteLine(struct rewriteConfigState *state, char *option, sds line, int force);
int rewriteConfig(char *path);
/* db.c -- Keyspace access API */
int removeExpire(redisDb *db, robj *key);
@@ -1260,54 +1019,33 @@ void setKey(redisDb *db, robj *key, robj *val);
int dbExists(redisDb *db, robj *key);
robj *dbRandomKey(redisDb *db);
int dbDelete(redisDb *db, robj *key);
robj *dbUnshareStringValue(redisDb *db, robj *key, robj *o);
long long emptyDb(void(callback)(void*));
long long emptyDb();
int selectDb(redisClient *c, int id);
void signalModifiedKey(redisDb *db, robj *key);
void signalFlushedDb(int dbid);
unsigned int getKeysInSlot(unsigned int hashslot, robj **keys, unsigned int count);
unsigned int countKeysInSlot(unsigned int hashslot);
int verifyClusterConfigWithData(void);
void scanGenericCommand(redisClient *c, robj *o, unsigned long cursor);
int parseScanCursorOrReply(redisClient *c, robj *o, unsigned long *cursor);
unsigned int GetKeysInSlot(unsigned int hashslot, robj **keys, unsigned int count);
/* API to get key arguments from commands */
int *getKeysFromCommand(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
#define REDIS_GETKEYS_ALL 0
#define REDIS_GETKEYS_PRELOAD 1
int *getKeysFromCommand(struct redisCommand *cmd, robj **argv, int argc, int *numkeys, int flags);
void getKeysFreeResult(int *result);
int *zunionInterGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *numkeys);
int *evalGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
int *sortGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
/* Cluster */
void clusterInit(void);
unsigned short crc16(const char *buf, int len);
unsigned int keyHashSlot(char *key, int keylen);
void clusterCron(void);
void clusterPropagatePublish(robj *channel, robj *message);
void migrateCloseTimedoutSockets(void);
void clusterBeforeSleep(void);
int *noPreloadGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *numkeys, int flags);
int *renameGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *numkeys, int flags);
int *zunionInterGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *numkeys, int flags);
/* Sentinel */
void initSentinelConfig(void);
void initSentinel(void);
void sentinelTimer(void);
char *sentinelHandleConfiguration(char **argv, int argc);
void sentinelIsRunning(void);
/* Scripting */
void scriptingInit(void);
/* Blocked clients */
void processUnblockedClients(void);
void blockClient(redisClient *c, int btype);
void unblockClient(redisClient *c);
void replyToBlockedClientTimedOut(redisClient *c);
int getTimeoutFromObjectOrReply(redisClient *c, robj *object, mstime_t *timeout, int unit);
/* Git SHA1 */
char *redisGitSHA1(void);
char *redisGitDirty(void);
uint64_t redisBuildId(void);
/* Commands prototypes */
void authCommand(redisClient *c);
@@ -1332,7 +1070,6 @@ void incrbyfloatCommand(redisClient *c);
void selectCommand(redisClient *c);
void randomkeyCommand(redisClient *c);
void keysCommand(redisClient *c);
void scanCommand(redisClient *c);
void dbsizeCommand(redisClient *c);
void lastsaveCommand(redisClient *c);
void saveCommand(redisClient *c);
@@ -1368,7 +1105,6 @@ void sunionCommand(redisClient *c);
void sunionstoreCommand(redisClient *c);
void sdiffCommand(redisClient *c);
void sdiffstoreCommand(redisClient *c);
void sscanCommand(redisClient *c);
void syncCommand(redisClient *c);
void flushdbCommand(redisClient *c);
void flushallCommand(redisClient *c);
@@ -1421,12 +1157,10 @@ void hlenCommand(redisClient *c);
void zremrangebyrankCommand(redisClient *c);
void zunionstoreCommand(redisClient *c);
void zinterstoreCommand(redisClient *c);
void zscanCommand(redisClient *c);
void hkeysCommand(redisClient *c);
void hvalsCommand(redisClient *c);
void hgetallCommand(redisClient *c);
void hexistsCommand(redisClient *c);
void hscanCommand(redisClient *c);
void configCommand(redisClient *c);
void hincrbyCommand(redisClient *c);
void hincrbyfloatCommand(redisClient *c);
@@ -1435,15 +1169,10 @@ void unsubscribeCommand(redisClient *c);
void psubscribeCommand(redisClient *c);
void punsubscribeCommand(redisClient *c);
void publishCommand(redisClient *c);
void pubsubCommand(redisClient *c);
void watchCommand(redisClient *c);
void unwatchCommand(redisClient *c);
void clusterCommand(redisClient *c);
void restoreCommand(redisClient *c);
void migrateCommand(redisClient *c);
void askingCommand(redisClient *c);
void readonlyCommand(redisClient *c);
void readwriteCommand(redisClient *c);
void dumpCommand(redisClient *c);
void objectCommand(redisClient *c);
void clientCommand(redisClient *c);
@@ -1453,14 +1182,7 @@ void scriptCommand(redisClient *c);
void timeCommand(redisClient *c);
void bitopCommand(redisClient *c);
void bitcountCommand(redisClient *c);
void bitposCommand(redisClient *c);
void replconfCommand(redisClient *c);
void waitCommand(redisClient *c);
void pfselftestCommand(redisClient *c);
void pfaddCommand(redisClient *c);
void pfcountCommand(redisClient *c);
void pfmergeCommand(redisClient *c);
void pfgetregCommand(redisClient *c);
#if defined(__GNUC__)
void *calloc(size_t count, size_t size) __attribute__ ((deprecated));
-47
View File
@@ -1,47 +0,0 @@
/* redisassert.h -- Drop in replacemnet assert.h that prints the stack trace
* in the Redis logs.
*
* This file should be included instead of "assert.h" inside libraries used by
* Redis that are using assertions, so instead of Redis disappearing with
* SIGABORT, we get the details and stack trace inside the log file.
*
* ----------------------------------------------------------------------------
*
* Copyright (c) 2006-2012, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __REDIS_ASSERT_H__
#define __REDIS_ASSERT_H__
#include <unistd.h> /* for _exit() */
#define assert(_e) ((_e)?(void)0 : (_redisAssert(#_e,__FILE__,__LINE__),_exit(1)))
void _redisAssert(char *estr, char *file, int line);
#endif
+2 -12
View File
@@ -27,15 +27,11 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
/* Every time the Redis Git SHA1 or Dirty status changes only this small
* file is recompiled, as we access this information in all the other
/* Every time the Redis Git SHA1 or Dirty status changes only this file
* small file is recompiled, as we access this information in all the other
* files using this functions. */
#include <string.h>
#include "release.h"
#include "version.h"
#include "crc64.h"
char *redisGitSHA1(void) {
return REDIS_GIT_SHA1;
@@ -44,9 +40,3 @@ char *redisGitSHA1(void) {
char *redisGitDirty(void) {
return REDIS_GIT_DIRTY;
}
uint64_t redisBuildId(void) {
char *buildid = REDIS_VERSION REDIS_BUILD_ID REDIS_GIT_DIRTY REDIS_GIT_SHA1;
return crc64(0,(unsigned char*)buildid,strlen(buildid));
}
+104 -1126
View File
File diff suppressed because it is too large Load Diff
+3 -36
View File
@@ -48,12 +48,10 @@
#include "fmacros.h"
#include <string.h>
#include <stdio.h>
#include <unistd.h>
#include "rio.h"
#include "util.h"
#include "crc64.h"
#include "config.h"
#include "redis.h"
uint64_t crc64(uint64_t crc, const unsigned char *s, uint64_t l);
/* Returns 1 or 0 for success/failure. */
static size_t rioBufferWrite(rio *r, const void *buf, size_t len) {
@@ -78,19 +76,7 @@ static off_t rioBufferTell(rio *r) {
/* Returns 1 or 0 for success/failure. */
static size_t rioFileWrite(rio *r, const void *buf, size_t len) {
size_t retval;
retval = fwrite(buf,len,1,r->io.file.fp);
r->io.file.buffered += len;
if (r->io.file.autosync &&
r->io.file.buffered >= r->io.file.autosync)
{
fflush(r->io.file.fp);
aof_fsync(fileno(r->io.file.fp));
r->io.file.buffered = 0;
}
return retval;
return fwrite(buf,len,1,r->io.file.fp);
}
/* Returns 1 or 0 for success/failure. */
@@ -109,8 +95,6 @@ static const rio rioBufferIO = {
rioBufferTell,
NULL, /* update_checksum */
0, /* current checksum */
0, /* bytes read or written */
0, /* read/write chunk size */
{ { NULL, 0 } } /* union for io-specific vars */
};
@@ -120,16 +104,12 @@ static const rio rioFileIO = {
rioFileTell,
NULL, /* update_checksum */
0, /* current checksum */
0, /* bytes read or written */
0, /* read/write chunk size */
{ { NULL, 0 } } /* union for io-specific vars */
};
void rioInitWithFile(rio *r, FILE *fp) {
*r = rioFileIO;
r->io.file.fp = fp;
r->io.file.buffered = 0;
r->io.file.autosync = 0;
}
void rioInitWithBuffer(rio *r, sds s) {
@@ -144,19 +124,6 @@ void rioGenericUpdateChecksum(rio *r, const void *buf, size_t len) {
r->cksum = crc64(r->cksum,buf,len);
}
/* Set the file-based rio object to auto-fsync every 'bytes' file written.
* By default this is set to zero that means no automatic file sync is
* performed.
*
* This feature is useful in a few contexts since when we rely on OS write
* buffers sometimes the OS buffers way too much, resulting in too many
* disk I/O concentrated in very little time. When we fsync in an explicit
* way instead the I/O pressure is more distributed across time. */
void rioSetAutoSync(rio *r, off_t bytes) {
redisAssert(r->read == rioFileIO.read);
r->io.file.autosync = bytes;
}
/* ------------------------------ Higher level interface ---------------------------
* The following higher level functions use lower level rio.c functions to help
* generating the Redis protocol for the Append Only File. */
+10 -33
View File
@@ -43,22 +43,15 @@ struct _rio {
size_t (*read)(struct _rio *, void *buf, size_t len);
size_t (*write)(struct _rio *, const void *buf, size_t len);
off_t (*tell)(struct _rio *);
/* The update_cksum method if not NULL is used to compute the checksum of
* all the data that was read or written so far. The method should be
* designed so that can be called with the current checksum, and the buf
* and len fields pointing to the new block of data to add to the checksum
* computation. */
/* The update_cksum method if not NULL is used to compute the checksum of all the
* data that was read or written so far. The method should be designed so that
* can be called with the current checksum, and the buf and len fields pointing
* to the new block of data to add to the checksum computation. */
void (*update_cksum)(struct _rio *, const void *buf, size_t len);
/* The current checksum */
uint64_t cksum;
/* number of bytes read or written */
size_t processed_bytes;
/* maximum single read or write chunk size */
size_t max_processing_chunk;
/* Backend-specific vars. */
union {
struct {
@@ -67,8 +60,6 @@ struct _rio {
} buffer;
struct {
FILE *fp;
off_t buffered; /* Bytes written since last fsync. */
off_t autosync; /* fsync after 'autosync' bytes written. */
} file;
} io;
};
@@ -80,29 +71,16 @@ typedef struct _rio rio;
* if needed. */
static inline size_t rioWrite(rio *r, const void *buf, size_t len) {
while (len) {
size_t bytes_to_write = (r->max_processing_chunk && r->max_processing_chunk < len) ? r->max_processing_chunk : len;
if (r->update_cksum) r->update_cksum(r,buf,bytes_to_write);
if (r->write(r,buf,bytes_to_write) == 0)
return 0;
buf = (char*)buf + bytes_to_write;
len -= bytes_to_write;
r->processed_bytes += bytes_to_write;
}
return 1;
if (r->update_cksum) r->update_cksum(r,buf,len);
return r->write(r,buf,len);
}
static inline size_t rioRead(rio *r, void *buf, size_t len) {
while (len) {
size_t bytes_to_read = (r->max_processing_chunk && r->max_processing_chunk < len) ? r->max_processing_chunk : len;
if (r->read(r,buf,bytes_to_read) == 0)
return 0;
if (r->update_cksum) r->update_cksum(r,buf,bytes_to_read);
buf = (char*)buf + bytes_to_read;
len -= bytes_to_read;
r->processed_bytes += bytes_to_read;
if (r->read(r,buf,len) == 1) {
if (r->update_cksum) r->update_cksum(r,buf,len);
return 1;
}
return 1;
return 0;
}
static inline off_t rioTell(rio *r) {
@@ -118,6 +96,5 @@ size_t rioWriteBulkLongLong(rio *r, long long l);
size_t rioWriteBulkDouble(rio *r, double d);
void rioGenericUpdateChecksum(rio *r, const void *buf, size_t len);
void rioSetAutoSync(rio *r, off_t bytes);
#endif

Some files were not shown because too many files have changed in this diff Show More