Redis 2.5.7 (2.6 RC1)

An user reported a crash with Redis scripting (see issue #480 on
github), inspection of the kindly provided strack trace showed that
server.lua_caller was probably set to NULL. The stack trace also slowed
that the call to the hook was originating from a point where we just
used to set/get a few global variables in the Lua state.

What was happening is that we did not set the timeout hook selectively
only when the user script was called. Now we set it more selectively,
specifically only in the context of the lua_pcall() call, and make sure
to remove the hook when the call returns. Otherwise the hook can get
called in random contexts every time we do something with the Lua
state.

00-RELEASENOTES

@@ -32,12 +32,17 @@ Also the following redis.conf and CONFIG GET / SET parameters changed name:
 CHANGELOG
 ---------
 
-What's new in Redis 2.6.0
-=========================
+What's new in Redis 2.5.7 (aka 2.6 Release Candidate 1)
+=======================================================
 
-UPGRADE URGENCY: We suggest new users to start with 2.6.0, and old users to
-                 upgrade after some testing of the application with the new
-                 Redis version.
+UPGRADE URGENCY: upgrade not recommended because this is an RC release.
+
+* This is the first release candidate for Redis 2.6. We are not aware of
+  bugs, but part of this code is young and was never tested in production
+  environments, so handle with care.
+
+An overview of new features and changes in Redis 2.6.x
+======================================================
 
 * Server side Lua scripting, see http://redis.io/commands/eval
 * Virtual Memory removed (was deprecated in 2.4)
@@ -46,7 +51,9 @@ UPGRADE URGENCY: We suggest new users to start with 2.6.0, and old users to
   in slaves.
 * Milliseconds resolution expires, also added new commands with milliseconds
   precision (PEXPIRE, PTTL, ...).
-* Clinets max output buffer soft and hard limits. You can specifiy different
+* Better memory usage for "small" lists, ziplists and hashes when fields or
+  values contain small integers.
+* Clients max output buffer soft and hard limits. You can specifiy different
   limits for different classes of clients (normal,pubsub,slave).
 * AOF is now able to rewrite aggregate data types using variadic commands,
   often producing an AOF that is faster to save, load, and is smaller in size.
@@ -54,6 +61,12 @@ UPGRADE URGENCY: We suggest new users to start with 2.6.0, and old users to
   redis-server binary, with the same name and number of arguments.
 * Hash table seed randomization for protection against collisions attacks.
 * Performances improved when writing large objects to Redis.
+* Integrated memory test, see redis-server --test-memory.
+* INCRBYFLOAT and HINCRBYFLOAT commands.
+* New DUMP, RESTORE, MIGRATE commands (back ported from Redis Cluster to 2.6).
+* CRC64 checksump in RDB files.
+* Better MONITOR output and behavior (now commands are logged before execution).
+* "Software Watchdog" feature to debug latency issues.
 * Significant parts of the core refactored or rewritten. New internal APIs
   and core changes allowed to develop Redis Cluster on top of the new code,
   however for 2.6 all the cluster code was removed, and will be released with
@@ -70,7 +83,8 @@ UPGRADE URGENCY: We suggest new users to start with 2.6.0, and old users to
 * New statistics about how many time a command was called, and how much
   execution time it used (INFO commandstats).
 * More predictable SORT behavior in edge cases.
-* INCRBYFLOAT and HINCRBYFLOAT commands.
+* Better support for big endian and *BSD systems.
+* Build system improved.
 
 --------------------------------------------------------------------------------
 

README

@@ -7,6 +7,13 @@ documentation at http://redis.io
 Building Redis
 --------------
 
+Redis can be compiled and used on Linux, OSX, OpenBSD, NetBSD, FreeBSD.
+We support big endian and little endian architectures.
+
+It may compile on Solaris derived systems (for instance SmartOS) but our
+support for this platform is "best effort" and Redis is not guaranteed to
+work as well as in Linux, OSX, and *BSD there.
+
 It is as simple as:
 
     % make

deps/Makefile

@@ -1,17 +1,6 @@
 # Redis dependency Makefile
 
-UNAME_S:=$(shell sh -c 'uname -s 2> /dev/null || echo not')
-
-LUA_CFLAGS=-O2 -Wall $(ARCH)
-ifeq ($(UNAME_S),SunOS)
-  # Make isinf() available
-  LUA_CFLAGS+= -D__C99FEATURES__=1
-endif
-
-JEMALLOC_CFLAGS=
-ifeq ($(ARCH),-m32)
-  JEMALLOC_CFLAGS+=CFLAGS="-std=gnu99 -Wall -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -m32"
-endif
+uname_S:= $(shell sh -c 'uname -s 2>/dev/null || echo not')
 
 CCCOLOR="\033[34m"
 LINKCOLOR="\033[34;1m"
@@ -23,37 +12,67 @@ ENDCOLOR="\033[0m"
 default:
 	@echo "Explicit target required"
 
-# Clean everything when ARCH is different
-ifneq ($(shell sh -c '[ -f .make-arch ] && cat .make-arch'), $(ARCH))
-.make-arch: distclean
-else
-.make-arch:
+.PHONY: default
+
+# Prerequisites target
+.make-prerequisites:
+	@touch $@
+
+# Clean everything when CFLAGS is different
+ifneq ($(shell sh -c '[ -f .make-cflags ] && cat .make-cflags || echo none'), $(CFLAGS))
+.make-cflags: distclean
+	-(echo "$(CFLAGS)" > .make-cflags)
+.make-prerequisites: .make-cflags
 endif
 
-.make-arch:
-	-(echo $(ARCH) > .make-arch)
+# Clean everything when LDFLAGS is different
+ifneq ($(shell sh -c '[ -f .make-ldflags ] && cat .make-ldflags || echo none'), $(LDFLAGS))
+.make-ldflags: distclean
+	-(echo "$(LDFLAGS)" > .make-ldflags)
+.make-prerequisites: .make-ldflags
+endif
 
 distclean:
 	-(cd hiredis && $(MAKE) clean) > /dev/null || true
 	-(cd linenoise && $(MAKE) clean) > /dev/null || true
 	-(cd lua && $(MAKE) clean) > /dev/null || true
 	-(cd jemalloc && [ -f Makefile ] && $(MAKE) distclean) > /dev/null || true
-	-(rm -f .make-arch)
+	-(rm -f .make-*)
 
-hiredis: .make-arch
-	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)hiredis$(ENDCOLOR)
-	cd hiredis && $(MAKE) static ARCH="$(ARCH)"
+.PHONY: distclean
 
-linenoise: .make-arch
-	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)linenoise$(ENDCOLOR)
-	cd linenoise && $(MAKE) ARCH="$(ARCH)"
+hiredis: .make-prerequisites
+	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
+	cd hiredis && $(MAKE) static
 
-lua: .make-arch
-	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)lua$(ENDCOLOR)
-	cd lua && $(MAKE) CFLAGS="$(LUA_CFLAGS)" MYLDFLAGS="$(ARCH)" ansi
+.PHONY: hiredis
 
-jemalloc: .make-arch
-	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)jemalloc$(ENDCOLOR)
-	cd jemalloc && ./configure $(JEMALLOC_CFLAGS) --with-jemalloc-prefix=je_ --enable-cc-silence && $(MAKE) lib/libjemalloc.a
+linenoise: .make-prerequisites
+	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
+	cd linenoise && $(MAKE)
 
-.PHONY: default conditional_clean hiredis linenoise lua jemalloc
+.PHONY: linenoise
+
+ifeq ($(uname_S),SunOS)
+  # Make isinf() available
+  LUA_CFLAGS= -D__C99FEATURES__=1
+endif
+
+LUA_CFLAGS+= -O2 -Wall -DLUA_ANSI $(CFLAGS)
+LUA_LDFLAGS+= $(LDFLAGS)
+
+lua: .make-prerequisites
+	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
+	cd lua/src && $(MAKE) all CFLAGS="$(LUA_CFLAGS)" MYLDFLAGS="$(LUA_LDFLAGS)"
+
+.PHONY: lua
+
+JEMALLOC_CFLAGS= -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops $(CFLAGS)
+JEMALLOC_LDFLAGS= $(LDFLAGS)
+
+jemalloc: .make-prerequisites
+	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
+	cd jemalloc && ./configure --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)"
+	cd jemalloc && $(MAKE) lib/libjemalloc.a
+
+.PHONY: jemalloc

deps/linenoise/Makefile

@@ -1,10 +1,21 @@
-linenoise_example: linenoise.h linenoise.c
+STD=
+WARN= -Wall
+OPT= -Os
+
+R_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS)
+R_LDFLAGS= $(LDFLAGS)
+DEBUG= -g
+
+R_CC=$(CC) $(R_CFLAGS)
+R_LD=$(CC) $(R_LDFLAGS)
+
+linenoise.o: linenoise.h linenoise.c
 
 linenoise_example: linenoise.o example.o
-	$(CC) $(ARCH) -Wall -W -Os -g -o linenoise_example linenoise.o example.o
+	$(R_LD) -o $@ $^
 
 .c.o:
-	$(CC) $(ARCH) -c -Wall -W -Os -g $<
+	$(R_CC) -c $<
 
 clean:
 	rm -f linenoise_example *.o

redis.conf

@@ -114,6 +114,15 @@ stop-writes-on-bgsave-error yes
 # the dataset will likely be bigger if you have compressible values or keys.
 rdbcompression yes
 
+# Since verison 5 of RDB a CRC64 checksum is placed at the end of the file.
+# This makes the format more resistant to corruption but there is a performance
+# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
+# for maximum performances.
+#
+# RDB files created with checksum disabled have a checksum of zero that will
+# tell the loading code to skip the check.
+rdbchecksum yes
+
 # The filename where to dump the DB
 dbfilename dump.rdb
 
@@ -156,6 +165,22 @@ dir ./
 #
 slave-serve-stale-data yes
 
+# You can configure a slave instance to accept writes or not. Writing against
+# a slave instance may be useful to store some ephemeral data (because data
+# written on a slave will be easily deleted after resync with the master) but
+# may also cause problems if clients are writing to it because of a
+# misconfiguration.
+#
+# Since Redis 2.6 by default slaves are read-only.
+#
+# Note: read only slaves are not designed to be exposed to untrusted clients
+# on the internet. It's just a protection layer against misuse of the instance.
+# Still a read only slave exports by default all the administrative commands
+# such as CONFIG, DEBUG, and so forth. To a limited extend you can improve
+# security of read only slaves using 'rename-command' to shadow all the
+# administrative / dangerous commands.
+slave-read-only yes
+
 # Slaves send PINGs to server in a predefined interval. It's possible to change
 # this interval with the repl_ping_slave_period option. The default value is 10
 # seconds.
@@ -246,7 +271,7 @@ slave-serve-stale-data yes
 # volatile-lru -> remove the key with an expire set using an LRU algorithm
 # allkeys-lru -> remove any key accordingly to the LRU algorithm
 # volatile-random -> remove a random key with an expire set
-# allkeys->random -> remove a random key, any key
+# allkeys-random -> remove a random key, any key
 # volatile-ttl -> remove the key with the nearest expire time (minor TTL)
 # noeviction -> don't expire at all, just return an error on write operations
 # 
@@ -273,21 +298,23 @@ slave-serve-stale-data yes
 
 ############################## APPEND ONLY MODE ###############################
 
-# By default Redis asynchronously dumps the dataset on disk. If you can live
-# with the idea that the latest records will be lost if something like a crash
-# happens this is the preferred way to run Redis. If instead you care a lot
-# about your data and don't want to that a single record can get lost you should
-# enable the append only mode: when this mode is enabled Redis will append
-# every write operation received in the file appendonly.aof. This file will
-# be read on startup in order to rebuild the full dataset in memory.
+# By default Redis asynchronously dumps the dataset on disk. This mode is
+# good enough in many applications, but an issue with the Redis process or
+# a power outage may result into a few minutes of writes lost (depending on
+# the configured save points).
 #
-# Note that you can have both the async dumps and the append only file if you
-# like (you have to comment the "save" statements above to disable the dumps).
-# Still if append only mode is enabled Redis will load the data from the
-# log file at startup ignoring the dump.rdb file.
+# The Append Only File is an alternative persistence mode that provides
+# much better durability. For instance using the default data fsync policy
+# (see later in the config file) Redis can lose just one second of writes in a
+# dramatic event like a server power outage, or a single write if something
+# wrong with the Redis process itself happens, but the operating system is
+# still running correctly.
 #
-# IMPORTANT: Check the BGREWRITEAOF to check how to rewrite the append
-# log file in background when it gets too big.
+# AOF and RDB persistence can be enabled at the same time without problems.
+# If the AOF is enabled on startup Redis will load the AOF, that is the file
+# with the better durability guarantees.
+#
+# Please check http://redis.io/topics/persistence for more information.
 
 appendonly no
 
@@ -302,7 +329,7 @@ appendonly no
 #
 # no: don't fsync, just let the OS flush the data when it wants. Faster.
 # always: fsync after every write to the append only log . Slow, Safest.
-# everysec: fsync only if one second passed since the last fsync. Compromise.
+# everysec: fsync only one time every second. Compromise.
 #
 # The default is "everysec" that's usually the right compromise between
 # speed and data safety. It's up to you to understand if you can relax this to
@@ -312,6 +339,9 @@ appendonly no
 # or on the contrary, use "always" that's very slow but a bit safer than
 # everysec.
 #
+# More details please check the following article:
+# http://antirez.com/post/redis-persistence-demystified.html
+#
 # If unsure, use "everysec".
 
 # appendfsync always
@@ -376,25 +406,6 @@ auto-aof-rewrite-min-size 64mb
 # Set it to 0 or a negative value for unlimited execution without warnings.
 lua-time-limit 5000
 
-################################ REDIS CLUSTER  ###############################
-#
-# Normal Redis instances can't be part of a Redis Cluster, only nodes that are
-# started as cluster nodes can. In order to start a Redis instance as a
-# cluster node enable the cluster support uncommenting the following:
-#
-# cluster-enabled yes
-
-# Every cluster node has a cluster configuration file. This file is not
-# intended to be edited by hand. It is created and updated by Redis nodes.
-# Every Redis Cluster node requires a different cluster configuration file.
-# Make sure that instances running in the same system does not have
-# overlapping cluster configuration file names.
-#
-# cluster-config-file nodes-6379.conf
-
-# In order to setup your cluster make sure to read the documentation
-# available at http://redis.io web site.
-
 ################################## SLOW LOG ###################################
 
 # The Redis Slow Log is a system to log queries that exceeded a specified
@@ -417,7 +428,7 @@ slowlog-log-slower-than 10000
 
 # There is no limit to this length. Just be aware that it will consume memory.
 # You can reclaim memory used by the slow log with SLOWLOG RESET.
-slowlog-max-len 1024
+slowlog-max-len 128
 
 ############################### ADVANCED CONFIG ###############################
 

src/.gitignore

@@ -0,0 +1,5 @@
+*.gcda
+*.gcno
+*.gcov
+redis.info
+lcov-html

src/Makefile

@@ -1,27 +1,32 @@
 # Redis Makefile
 # Copyright (C) 2009 Salvatore Sanfilippo <antirez at gmail dot com>
 # This file is released under the BSD license, see the COPYING file
+#
+# The Makefile composes the final FINAL_CFLAGS and FINAL_LDFLAGS using
+# what is needed for Redis plus the standard CFLAGS and LDFLAGS passed.
+# However when building the dependencies (Jemalloc, Lua, Hiredis, ...)
+# CFLAGS and LDFLAGS are propagated to the dependencies, so to pass
+# flags only to be used when compiling / linking Redis itself REDIS_CFLAGS
+# and REDIS_LDFLAGS are used instead (this is the case of 'make gcov').
+#
+# Dependencies are stored in the Makefile.dep file. To rebuild this file
+# Just use 'make dep', but this is only needed by developers.
 
 release_hdr := $(shell sh -c './mkreleasehdr.sh')
 uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
 OPTIMIZATION?=-O2
 DEPENDENCY_TARGETS=hiredis linenoise lua
 
-ifeq ($(uname_S),SunOS)
-  CFLAGS?=-std=c99 -pedantic $(OPTIMIZATION) -Wall -W -D__EXTENSIONS__ -D_XPG6
-  CCLINK?=-ldl -lnsl -lsocket -lm -lpthread
-  DEBUG?=-g -ggdb 
-else
-  CFLAGS?=-std=c99 -pedantic $(OPTIMIZATION) -Wall -W $(ARCH) $(PROF)
-  CCLINK?=-lm -pthread
-  DEBUG?=-g -rdynamic -ggdb 
-endif
+# Default settings
+STD= -std=c99 -pedantic
+WARN= -Wall
+OPT= $(OPTIMIZATION)
 
 # Default allocator
 ifeq ($(uname_S),Linux)
-  MALLOC?=jemalloc
+  MALLOC=jemalloc
 else
-  MALLOC?=libc
+  MALLOC=libc
 endif
 
 # Backwards compatibility for selecting an allocator
@@ -37,25 +42,42 @@ ifeq ($(USE_JEMALLOC),yes)
   MALLOC=jemalloc
 endif
 
+# Override default settings if possible
+-include .make-settings
+
+ifeq ($(uname_S),SunOS)
+  FINAL_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS) -D__EXTENSIONS__ -D_XPG6
+  FINAL_LDFLAGS= $(LDFLAGS) $(REDIS_LDFLAGS) -g -ggdb
+  FINAL_LIBS= -ldl -lnsl -lsocket -lm -lpthread
+  DEBUG= -g -ggdb
+else
+  FINAL_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS)
+  FINAL_LDFLAGS= $(LDFLAGS) $(REDIS_LDFLAGS) -g -rdynamic -ggdb
+  FINAL_LIBS= -lm -pthread
+  DEBUG= -g -rdynamic -ggdb
+endif
+
+# Include paths to dependencies
+FINAL_CFLAGS+= -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src
+
 ifeq ($(MALLOC),tcmalloc)
-  ALLOC_LINK=-ltcmalloc
-  ALLOC_FLAGS=-DUSE_TCMALLOC
+  FINAL_CFLAGS+= -DUSE_TCMALLOC
+  FINAL_LIBS+= -ltcmalloc
 endif
 
 ifeq ($(MALLOC),tcmalloc_minimal)
-  ALLOC_LINK=-ltcmalloc_minimal
-  ALLOC_FLAGS=-DUSE_TCMALLOC
+  FINAL_CFLAGS+= -DUSE_TCMALLOC
+  FINAL_LIBS+= -ltcmalloc_minimal
 endif
 
 ifeq ($(MALLOC),jemalloc)
-  ALLOC_LINK=../deps/jemalloc/lib/libjemalloc.a -ldl
-  ALLOC_FLAGS=-DUSE_JEMALLOC -I../deps/jemalloc/include
   DEPENDENCY_TARGETS+= jemalloc
+  FINAL_CFLAGS+= -DUSE_JEMALLOC -I../deps/jemalloc/include
+  FINAL_LIBS+= ../deps/jemalloc/lib/libjemalloc.a -ldl
 endif
 
-CCLINK+= $(ALLOC_LINK)
-CFLAGS+= $(ALLOC_FLAGS)
-CCOPT= $(CFLAGS) $(ARCH) $(PROF)
+REDIS_CC=$(QUIET_CC)$(CC) $(FINAL_CFLAGS)
+REDIS_LD=$(QUIET_LINK)$(CC) $(FINAL_LDFLAGS)
 
 PREFIX= /usr/local
 INSTALL_BIN= $(PREFIX)/bin
@@ -69,202 +91,134 @@ MAKECOLOR="\033[32;1m"
 ENDCOLOR="\033[0m"
 
 ifndef V
-QUIET_CC = @printf '    %b %b\n' $(CCCOLOR)CC$(ENDCOLOR) $(SRCCOLOR)$@$(ENDCOLOR);
-QUIET_LINK = @printf '    %b %b\n' $(LINKCOLOR)LINK$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR);
+QUIET_CC = @printf '    %b %b\n' $(CCCOLOR)CC$(ENDCOLOR) $(SRCCOLOR)$@$(ENDCOLOR) 1>&2;
+QUIET_LINK = @printf '    %b %b\n' $(LINKCOLOR)LINK$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR) 1>&2;
 endif
 
-OBJ = adlist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o
-BENCHOBJ = ae.o anet.o redis-benchmark.o sds.o adlist.o zmalloc.o
-CLIOBJ = anet.o sds.o adlist.o redis-cli.o zmalloc.o release.o
-CHECKDUMPOBJ = redis-check-dump.o lzf_c.o lzf_d.o
-CHECKAOFOBJ = redis-check-aof.o
+REDIS_SERVER_NAME= redis-server
+REDIS_SERVER_OBJ= adlist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o migrate.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o
+REDIS_CLI_NAME= redis-cli
+REDIS_CLI_OBJ= anet.o sds.o adlist.o redis-cli.o zmalloc.o release.o
+REDIS_BENCHMARK_NAME= redis-benchmark
+REDIS_BENCHMARK_OBJ= ae.o anet.o redis-benchmark.o sds.o adlist.o zmalloc.o redis-benchmark.o
+REDIS_CHECK_DUMP_NAME= redis-check-dump
+REDIS_CHECK_DUMP_OBJ= redis-check-dump.o lzf_c.o lzf_d.o crc64.o
+REDIS_CHECK_AOF_NAME= redis-check-aof
+REDIS_CHECK_AOF_OBJ= redis-check-aof.o
 
-PRGNAME = redis-server
-BENCHPRGNAME = redis-benchmark
-CLIPRGNAME = redis-cli
-CHECKDUMPPRGNAME = redis-check-dump
-CHECKAOFPRGNAME = redis-check-aof
-
-all: redis-benchmark redis-cli redis-check-dump redis-check-aof redis-server
+all: $(REDIS_SERVER_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_DUMP_NAME) $(REDIS_CHECK_AOF_NAME)
 	@echo ""
 	@echo "Hint: To run 'make test' is a good idea ;)"
 	@echo ""
 
+.PHONY: all
+
 # Deps (use make dep to generate this)
-adlist.o: adlist.c adlist.h zmalloc.h
-ae.o: ae.c ae.h zmalloc.h config.h ae_kqueue.c
-ae_epoll.o: ae_epoll.c
-ae_kqueue.o: ae_kqueue.c
-ae_select.o: ae_select.c
-anet.o: anet.c fmacros.h anet.h
-aof.o: aof.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-bio.o: bio.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h bio.h
-cluster.o: cluster.c redis.h fmacros.h config.h ae.h sds.h dict.h \
-  adlist.h zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-config.o: config.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-crc16.o: crc16.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-db.o: db.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-debug.o: debug.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h sha1.h
-dict.o: dict.c fmacros.h dict.h zmalloc.h
-endianconv.o: endianconv.c
-intset.o: intset.c intset.h zmalloc.h endianconv.h
-lzf_c.o: lzf_c.c lzfP.h
-lzf_d.o: lzf_d.c lzfP.h
-multi.o: multi.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-networking.o: networking.c redis.h fmacros.h config.h ae.h sds.h dict.h \
-  adlist.h zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-object.o: object.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-pqsort.o: pqsort.c
-pubsub.o: pubsub.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-rdb.o: rdb.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h lzf.h
-redis-benchmark.o: redis-benchmark.c fmacros.h ae.h \
-  ../deps/hiredis/hiredis.h sds.h adlist.h zmalloc.h
-redis-check-aof.o: redis-check-aof.c fmacros.h config.h
-redis-check-dump.o: redis-check-dump.c lzf.h
-redis-cli.o: redis-cli.c fmacros.h version.h ../deps/hiredis/hiredis.h \
-  sds.h zmalloc.h ../deps/linenoise/linenoise.h help.h
-redis.o: redis.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h slowlog.h \
-  bio.h asciilogo.h
-release.o: release.c release.h
-replication.o: replication.c redis.h fmacros.h config.h ae.h sds.h dict.h \
-  adlist.h zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-scripting.o: scripting.c redis.h fmacros.h config.h ae.h sds.h dict.h \
-  adlist.h zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h \
-  sha1.h rand.h
-rio.o: rio.c sds.h
-sds.o: sds.c sds.h zmalloc.h
-sha1.o: sha1.c sha1.h config.h
-slowlog.o: slowlog.c redis.h fmacros.h config.h ae.h sds.h dict.h \
-  adlist.h zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h \
-  slowlog.h
-sort.o: sort.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h pqsort.h
-syncio.o: syncio.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-t_hash.o: t_hash.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-t_list.o: t_list.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-t_set.o: t_set.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-t_string.o: t_string.c redis.h fmacros.h config.h ae.h sds.h dict.h \
-  adlist.h zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-t_zset.o: t_zset.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
-  zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
-util.o: util.c fmacros.h util.h
-ziplist.o: ziplist.c zmalloc.h util.h ziplist.h endianconv.h
-zipmap.o: zipmap.c zmalloc.h endianconv.h
-zmalloc.o: zmalloc.c config.h zmalloc.h
+include Makefile.dep
 
-# Clean local objects when ARCH is different
-ifneq ($(shell sh -c '[ -f .make-arch ] && cat .make-arch'), $(ARCH))
-.make-arch: clean
-else
-.make-arch:
-endif
+dep:
+	$(REDIS_CC) -MM *.c > Makefile.dep
 
-.make-arch:
-	-(cd ../deps && $(MAKE) $(DEPENDENCY_TARGETS) ARCH="$(ARCH)")
-	-(echo $(ARCH) > .make-arch)
+.PHONY: dep
 
-# Clean local objects when allocator changes
-ifneq ($(shell sh -c '[ -f .make-malloc ] && cat .make-malloc'), $(MALLOC))
-.make-malloc: clean
-else
-.make-malloc:
-endif
+persist-settings: distclean
+	echo STD=$(STD) >> .make-settings
+	echo WARN=$(WARN) >> .make-settings
+	echo OPT=$(OPT) >> .make-settings
+	echo MALLOC=$(MALLOC) >> .make-settings
+	echo CFLAGS=$(CFLAGS) >> .make-settings
+	echo LDFLAGS=$(LDFLAGS) >> .make-settings
+	echo REDIS_CFLAGS=$(REDIS_CFLAGS) >> .make-settings
+	echo REDIS_LDFLAGS=$(REDIS_LDFLAGS) >> .make-settings
+	echo PREV_FINAL_CFLAGS=$(FINAL_CFLAGS) >> .make-settings
+	echo PREV_FINAL_LDFLAGS=$(FINAL_LDFLAGS) >> .make-settings
+	-(cd ../deps && $(MAKE) $(DEPENDENCY_TARGETS))
 
-.make-malloc:
-	-(echo $(MALLOC) > .make-malloc)
+.PHONY: persist-settings
 
-# Union of make-prerequisites
-.make-prerequisites: .make-arch .make-malloc
+# Prerequisites target
+.make-prerequisites:
 	@touch $@
 
-redis-server: .make-prerequisites $(OBJ)
-	$(QUIET_LINK)$(CC) -o $(PRGNAME) $(CCOPT) $(DEBUG) $(OBJ) ../deps/lua/src/liblua.a $(CCLINK)
+# Clean everything, persist settings and build dependencies if anything changed
+ifneq ($(strip $(PREV_FINAL_CFLAGS)), $(strip $(FINAL_CFLAGS)))
+.make-prerequisites: persist-settings
+endif
 
-redis-benchmark: .make-prerequisites $(BENCHOBJ)
-	$(QUIET_LINK)$(CC) -o $(BENCHPRGNAME) $(CCOPT) $(DEBUG) $(BENCHOBJ) ../deps/hiredis/libhiredis.a $(CCLINK)
+ifneq ($(strip $(PREV_FINAL_LDFLAGS)), $(strip $(FINAL_LDFLAGS)))
+.make-prerequisites: persist-settings
+endif
 
-redis-benchmark.o: redis-benchmark.c .make-prerequisites
-	$(QUIET_CC)$(CC) -c $(CFLAGS) -I../deps/hiredis $(DEBUG) $(COMPILE_TIME) $<
+# redis-server
+$(REDIS_SERVER_NAME): $(REDIS_SERVER_OBJ)
+	$(REDIS_LD) -o $@ $^ ../deps/lua/src/liblua.a $(FINAL_LIBS)
 
-redis-cli: .make-prerequisites $(CLIOBJ)
-	$(QUIET_LINK)$(CC) -o $(CLIPRGNAME) $(CCOPT) $(DEBUG) $(CLIOBJ) ../deps/hiredis/libhiredis.a ../deps/linenoise/linenoise.o $(CCLINK)
+# redis-cli
+$(REDIS_CLI_NAME): $(REDIS_CLI_OBJ)
+	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/linenoise/linenoise.o $(FINAL_LIBS)
 
-redis-cli.o: redis-cli.c .make-prerequisites
-	$(QUIET_CC)$(CC) -c $(CFLAGS) -I../deps/hiredis -I../deps/linenoise $(DEBUG) $(COMPILE_TIME) $<
+# redis-benchmark
+$(REDIS_BENCHMARK_NAME): $(REDIS_BENCHMARK_OBJ)
+	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a $(FINAL_LIBS)
 
-redis-check-dump: .make-prerequisites $(CHECKDUMPOBJ)
-	$(QUIET_LINK)$(CC) -o $(CHECKDUMPPRGNAME) $(CCOPT) $(DEBUG) $(CHECKDUMPOBJ) $(CCLINK)
+# redis-check-dump
+$(REDIS_CHECK_DUMP_NAME): $(REDIS_CHECK_DUMP_OBJ)
+	$(REDIS_LD) -o $@ $^ $(FINAL_LIBS)
 
-redis-check-aof: .make-prerequisites $(CHECKAOFOBJ)
-	$(QUIET_LINK)$(CC) -o $(CHECKAOFPRGNAME) $(CCOPT) $(DEBUG) $(CHECKAOFOBJ) $(CCLINK)
+# redis-check-aof
+$(REDIS_CHECK_AOF_NAME): $(REDIS_CHECK_AOF_OBJ)
+	$(REDIS_LD) -o $@ $^ $(FINAL_LIBS)
 
-# Because the jemalloc.h header is generated as a part of the jemalloc build
-# process, building it should complete before building any other object. Instead of
-# depending on a single artifact, simply build all dependencies first.
+# Because the jemalloc.h header is generated as a part of the jemalloc build,
+# building it should complete before building any other object. Instead of
+# depending on a single artifact, build all dependencies first.
 %.o: %.c .make-prerequisites
-	$(QUIET_CC)$(CC) -c $(CFLAGS) $(DEBUG) $(COMPILE_TIME) -I../deps/lua/src $<
-
-.PHONY: all clean distclean
+	$(REDIS_CC) -c $<
 
 clean:
-	rm -rf $(PRGNAME) $(BENCHPRGNAME) $(CLIPRGNAME) $(CHECKDUMPPRGNAME) $(CHECKAOFPRGNAME) *.o *.gcda *.gcno *.gcov
+	rm -rf $(REDIS_SERVER_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_DUMP_NAME) $(REDIS_CHECK_AOF_NAME) *.o *.gcda *.gcno *.gcov redis.info lcov-html
+
+.PHONY: clean
 
 distclean: clean
 	-(cd ../deps && $(MAKE) distclean)
-	-(rm -f .make-arch .make-malloc)
+	-(rm -f .make-*)
 
-dep:
-	$(CC) -MM *.c -I ../deps/hiredis -I ../deps/linenoise
+.PHONY: distclean
 
-test: redis-server redis-check-aof
+test: $(REDIS_SERVER_NAME) $(REDIS_CHECK_AOF_NAME)
 	@(cd ..; ./runtest)
 
-bench:
-	./redis-benchmark
+lcov:
+	$(MAKE) gcov
+	@(set -e; cd ..; ./runtest --clients 1)
+	@geninfo -o redis.info .
+	@genhtml --legend -o lcov-html redis.info
 
-log:
-	git log '--pretty=format:%ad %s (%cn)' --date=short > ../Changelog
+.PHONY: lcov
+
+bench: $(REDIS_BENCHMARK_NAME)
+	./$(REDIS_BENCHMARK_NAME)
 
 32bit:
 	@echo ""
 	@echo "WARNING: if it fails under Linux you probably need to install libc6-dev-i386"
 	@echo ""
-	$(MAKE) ARCH="-m32" JEMALLOC_CFLAGS='CFLAGS="-std=gnu99 -Wall -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -m32"'
-
-gprof:
-	$(MAKE) PROF="-pg"
+	$(MAKE) CFLAGS="-m32" LDFLAGS="-m32"
 
 gcov:
-	$(MAKE) PROF="-fprofile-arcs -ftest-coverage"
+	$(MAKE) REDIS_CFLAGS="-fprofile-arcs -ftest-coverage -DCOVERAGE_TEST" REDIS_LDFLAGS="-fprofile-arcs -ftest-coverage"
 
 noopt:
-	$(MAKE) OPTIMIZATION=""
-
-32bitgprof:
-	$(MAKE) PROF="-pg" ARCH="-arch i386"
+	$(MAKE) OPT="-O0"
 
 src/help.h:
 	@../utils/generate-command-help.rb > help.h
 
 install: all
 	mkdir -p $(INSTALL_BIN)
-	$(INSTALL) $(PRGNAME) $(INSTALL_BIN)
-	$(INSTALL) $(BENCHPRGNAME) $(INSTALL_BIN)
-	$(INSTALL) $(CLIPRGNAME) $(INSTALL_BIN)
-	$(INSTALL) $(CHECKDUMPPRGNAME) $(INSTALL_BIN)
-	$(INSTALL) $(CHECKAOFPRGNAME) $(INSTALL_BIN)
+	$(INSTALL) $(REDIS_SERVER_NAME) $(INSTALL_BIN)
+	$(INSTALL) $(REDIS_BENCHMARK_NAME) $(INSTALL_BIN)
+	$(INSTALL) $(REDIS_CLI_NAME) $(INSTALL_BIN)
+	$(INSTALL) $(REDIS_CHECK_DUMP_NAME) $(INSTALL_BIN)
+	$(INSTALL) $(REDIS_CHECK_AOF_NAME) $(INSTALL_BIN)

src/Makefile.dep

@@ -0,0 +1,101 @@
+adlist.o: adlist.c adlist.h zmalloc.h
+ae.o: ae.c ae.h zmalloc.h config.h ae_kqueue.c
+ae_epoll.o: ae_epoll.c
+ae_kqueue.o: ae_kqueue.c
+ae_select.o: ae_select.c
+anet.o: anet.c fmacros.h anet.h
+aof.o: aof.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h bio.h
+bio.o: bio.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h bio.h
+config.o: config.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+crc64.o: crc64.c
+db.o: db.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+debug.o: debug.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h sha1.h
+dict.o: dict.c fmacros.h dict.h zmalloc.h
+endianconv.o: endianconv.c
+intset.o: intset.c intset.h zmalloc.h endianconv.h
+lzf_c.o: lzf_c.c lzfP.h
+lzf_d.o: lzf_d.c lzfP.h
+memtest.o: memtest.c
+migrate.o: migrate.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h endianconv.h
+multi.o: multi.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+networking.o: networking.c redis.h fmacros.h config.h \
+  ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+  adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h rdb.h \
+  rio.h
+object.o: object.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+pqsort.o: pqsort.c
+pubsub.o: pubsub.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+rand.o: rand.c
+rdb.o: rdb.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h lzf.h zipmap.h \
+  endianconv.h
+redis-benchmark.o: redis-benchmark.c fmacros.h ae.h \
+  ../deps/hiredis/hiredis.h sds.h adlist.h zmalloc.h
+redis-check-aof.o: redis-check-aof.c fmacros.h config.h
+redis-check-dump.o: redis-check-dump.c lzf.h
+redis-cli.o: redis-cli.c fmacros.h version.h ../deps/hiredis/hiredis.h \
+  sds.h zmalloc.h ../deps/linenoise/linenoise.h help.h
+redis.o: redis.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h slowlog.h bio.h \
+  asciilogo.h
+release.o: release.c release.h
+replication.o: replication.c redis.h fmacros.h config.h \
+  ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h ae.h sds.h dict.h \
+  adlist.h zmalloc.h anet.h ziplist.h intset.h version.h util.h rdb.h \
+  rio.h
+rio.o: rio.c fmacros.h rio.h sds.h util.h
+scripting.o: scripting.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h sha1.h rand.h \
+  ../deps/lua/src/lauxlib.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/lualib.h
+sds.o: sds.c sds.h zmalloc.h
+sha1.o: sha1.c sha1.h config.h
+slowlog.o: slowlog.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h slowlog.h
+sort.o: sort.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h pqsort.h
+syncio.o: syncio.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+t_hash.o: t_hash.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+t_list.o: t_list.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+t_set.o: t_set.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+t_string.o: t_string.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+t_zset.o: t_zset.c redis.h fmacros.h config.h ../deps/lua/src/lua.h \
+  ../deps/lua/src/luaconf.h ae.h sds.h dict.h adlist.h zmalloc.h anet.h \
+  ziplist.h intset.h version.h util.h rdb.h rio.h
+util.o: util.c fmacros.h util.h
+ziplist.o: ziplist.c zmalloc.h util.h ziplist.h endianconv.h
+zipmap.o: zipmap.c zmalloc.h endianconv.h
+zmalloc.o: zmalloc.c config.h zmalloc.h

src/adlist.c

@@ -323,3 +323,19 @@ listNode *listIndex(list *list, long index) {
     }
     return n;
 }
+
+/* Rotate the list removing the tail node and inserting it to the head. */
+void listRotate(list *list) {
+    listNode *tail = list->tail;
+
+    if (listLength(list) <= 1) return;
+
+    /* Detatch current tail */
+    list->tail = tail->prev;
+    list->tail->next = NULL;
+    /* Move it as head */
+    list->head->prev = tail;
+    tail->prev = NULL;
+    tail->next = list->head;
+    list->head = tail;
+}

src/adlist.h

@@ -84,6 +84,7 @@ listNode *listSearchKey(list *list, void *key);
 listNode *listIndex(list *list, long index);
 void listRewind(list *list, listIter *li);
 void listRewindTail(list *list, listIter *li);
+void listRotate(list *list);
 
 /* Directions for iterators */
 #define AL_START_HEAD 0

src/ae.c

@@ -35,6 +35,7 @@
 #include <sys/types.h>
 #include <unistd.h>
 #include <stdlib.h>
+#include <poll.h>
 #include <string.h>
 
 #include "ae.h"
@@ -369,21 +370,17 @@ int aeProcessEvents(aeEventLoop *eventLoop, int flags)
 /* Wait for millseconds until the given file descriptor becomes
  * writable/readable/exception */
 int aeWait(int fd, int mask, long long milliseconds) {
-    struct timeval tv;
-    fd_set rfds, wfds, efds;
+    struct pollfd pfd;
     int retmask = 0, retval;
 
-    tv.tv_sec = milliseconds/1000;
-    tv.tv_usec = (milliseconds%1000)*1000;
-    FD_ZERO(&rfds);
-    FD_ZERO(&wfds);
-    FD_ZERO(&efds);
+    memset(&pfd, 0, sizeof(pfd));
+    pfd.fd = fd;
+    if (mask & AE_READABLE) pfd.events |= POLLIN;
+    if (mask & AE_WRITABLE) pfd.events |= POLLOUT;
 
-    if (mask & AE_READABLE) FD_SET(fd,&rfds);
-    if (mask & AE_WRITABLE) FD_SET(fd,&wfds);
-    if ((retval = select(fd+1, &rfds, &wfds, &efds, &tv)) > 0) {
-        if (FD_ISSET(fd,&rfds)) retmask |= AE_READABLE;
-        if (FD_ISSET(fd,&wfds)) retmask |= AE_WRITABLE;
+    if ((retval = poll(&pfd, 1, milliseconds))== 1) {
+        if (pfd.revents & POLLIN) retmask |= AE_READABLE;
+        if (pfd.revents & POLLOUT) retmask |= AE_WRITABLE;
         return retmask;
     } else {
         return retval;

src/anet.c

@@ -262,7 +262,11 @@ static int anetListen(char *err, int s, struct sockaddr *sa, socklen_t len) {
         close(s);
         return ANET_ERR;
     }
-    if (listen(s, 511) == -1) { /* the magic 511 constant is from nginx */
+
+    /* Use a backlog of 512 entries. We pass 511 to the listen() call because
+     * the kernel does: backlogsize = roundup_pow_of_two(backlogsize + 1);
+     * which will thus give us a backlog of 512 entries */
+    if (listen(s, 511) == -1) {
         anetSetError(err, "listen: %s", strerror(errno));
         close(s);
         return ANET_ERR;

src/aof.c

@@ -46,7 +46,7 @@ void stopAppendOnly(void) {
 /* Called when the user switches from "appendonly no" to "appendonly yes"
  * at runtime using the CONFIG command. */
 int startAppendOnly(void) {
-    server.aof_last_fsync = time(NULL);
+    server.aof_last_fsync = server.unixtime;
     server.aof_fd = open(server.aof_filename,O_WRONLY|O_APPEND|O_CREAT,0644);
     redisAssert(server.aof_state == REDIS_AOF_OFF);
     if (server.aof_fd == -1) {
@@ -108,6 +108,7 @@ void flushAppendOnlyFile(int force) {
             }
             /* Otherwise fall trough, and go write since we can't wait
              * over two seconds. */
+            server.aof_delayed_fsync++;
             redisLog(REDIS_NOTICE,"Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.");
         }
     }
@@ -287,6 +288,7 @@ struct redisClient *createFakeClient(void) {
     selectDb(c,0);
     c->fd = -1;
     c->querybuf = sdsempty();
+    c->querybuf_peak = 0;
     c->argc = 0;
     c->argv = NULL;
     c->bufpos = 0;
@@ -801,9 +803,9 @@ int rewriteAppendOnlyFileBackground(void) {
         if (server.sofd > 0) close(server.sofd);
         snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) getpid());
         if (rewriteAppendOnlyFile(tmpfile) == REDIS_OK) {
-            _exit(0);
+            exitFromChild(0);
         } else {
-            _exit(1);
+            exitFromChild(1);
         }
     } else {
         /* Parent */

src/bio.c

@@ -108,9 +108,18 @@ void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3) {
 void *bioProcessBackgroundJobs(void *arg) {
     struct bio_job *job;
     unsigned long type = (unsigned long) arg;
+    sigset_t sigset;
 
     pthread_detach(pthread_self());
     pthread_mutex_lock(&bio_mutex[type]);
+    /* Block SIGALRM so we are sure that only the main thread will
+     * receive the watchdog signal. */
+    sigemptyset(&sigset);
+    sigaddset(&sigset, SIGALRM);
+    if (pthread_sigmask(SIG_BLOCK, &sigset, NULL))
+        redisLog(REDIS_WARNING,
+            "Warning: can't mask SIGALRM in bio.c thread: %s", strerror(errno));
+
     while(1) {
         listNode *ln;
 

src/config.c

@@ -155,6 +155,9 @@ void loadServerConfigFromString(char *config) {
             loadServerConfig(argv[1],NULL);
         } else if (!strcasecmp(argv[0],"maxclients") && argc == 2) {
             server.maxclients = atoi(argv[1]);
+            if (server.maxclients < 1) {
+                err = "Invalid max clients limit"; goto loaderr;
+            }
         } else if (!strcasecmp(argv[0],"maxmemory") && argc == 2) {
             server.maxmemory = memtoll(argv[1],NULL);
         } else if (!strcasecmp(argv[0],"maxmemory-policy") && argc == 2) {
@@ -202,10 +205,18 @@ void loadServerConfigFromString(char *config) {
             if ((server.repl_serve_stale_data = yesnotoi(argv[1])) == -1) {
                 err = "argument must be 'yes' or 'no'"; goto loaderr;
             }
+        } else if (!strcasecmp(argv[0],"slave-read-only") && argc == 2) {
+            if ((server.repl_slave_ro = yesnotoi(argv[1])) == -1) {
+                err = "argument must be 'yes' or 'no'"; goto loaderr;
+            }
         } else if (!strcasecmp(argv[0],"rdbcompression") && argc == 2) {
             if ((server.rdb_compression = yesnotoi(argv[1])) == -1) {
                 err = "argument must be 'yes' or 'no'"; goto loaderr;
             }
+        } else if (!strcasecmp(argv[0],"rdbchecksum") && argc == 2) {
+            if ((server.rdb_checksum = yesnotoi(argv[1])) == -1) {
+                err = "argument must be 'yes' or 'no'"; goto loaderr;
+            }
         } else if (!strcasecmp(argv[0],"activerehashing") && argc == 2) {
             if ((server.activerehashing = yesnotoi(argv[1])) == -1) {
                 err = "argument must be 'yes' or 'no'"; goto loaderr;
@@ -298,13 +309,6 @@ void loadServerConfigFromString(char *config) {
                     err = "Target command name already exists"; goto loaderr;
                 }
             }
-        } else if (!strcasecmp(argv[0],"cluster-enabled") && argc == 2) {
-            if ((server.cluster_enabled = yesnotoi(argv[1])) == -1) {
-                err = "argument must be 'yes' or 'no'"; goto loaderr;
-            }
-        } else if (!strcasecmp(argv[0],"cluster-config-file") && argc == 2) {
-            zfree(server.cluster.configfile);
-            server.cluster.configfile = zstrdup(argv[1]);
         } else if (!strcasecmp(argv[0],"lua-time-limit") && argc == 2) {
             server.lua_time_limit = strtoll(argv[1],NULL,10);
         } else if (!strcasecmp(argv[0],"slowlog-log-slower-than") &&
@@ -514,6 +518,11 @@ void configSetCommand(redisClient *c) {
 
         if (yn == -1) goto badfmt;
         server.repl_serve_stale_data = yn;
+    } else if (!strcasecmp(c->argv[2]->ptr,"slave-read-only")) {
+        int yn = yesnotoi(o->ptr);
+
+        if (yn == -1) goto badfmt;
+        server.repl_slave_ro = yn;
     } else if (!strcasecmp(c->argv[2]->ptr,"dir")) {
         if (chdir((char*)o->ptr) == -1) {
             addReplyErrorFormat(c,"Changing directory: %s", strerror(errno));
@@ -618,6 +627,22 @@ void configSetCommand(redisClient *c) {
     } else if (!strcasecmp(c->argv[2]->ptr,"repl-timeout")) {
         if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll <= 0) goto badfmt;
         server.repl_timeout = ll;
+    } else if (!strcasecmp(c->argv[2]->ptr,"watchdog-period")) {
+        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
+        if (ll)
+            enableWatchdog(ll);
+        else
+            disableWatchdog();
+    } else if (!strcasecmp(c->argv[2]->ptr,"rdbcompression")) {
+        int yn = yesnotoi(o->ptr);
+
+        if (yn == -1) goto badfmt;
+        server.rdb_compression = yn;
+    } else if (!strcasecmp(c->argv[2]->ptr,"rdbchecksum")) {
+        int yn = yesnotoi(o->ptr);
+
+        if (yn == -1) goto badfmt;
+        server.rdb_checksum = yn;
     } else {
         addReplyErrorFormat(c,"Unsupported CONFIG parameter: %s",
             (char*)c->argv[2]->ptr);
@@ -706,16 +731,20 @@ void configGetCommand(redisClient *c) {
     config_get_numerical_field("repl-ping-slave-period",server.repl_ping_slave_period);
     config_get_numerical_field("repl-timeout",server.repl_timeout);
     config_get_numerical_field("maxclients",server.maxclients);
+    config_get_numerical_field("watchdog-period",server.watchdog_period);
 
     /* Bool (yes/no) values */
     config_get_bool_field("no-appendfsync-on-rewrite",
             server.aof_no_fsync_on_rewrite);
     config_get_bool_field("slave-serve-stale-data",
             server.repl_serve_stale_data);
+    config_get_bool_field("slave-read-only",
+            server.repl_slave_ro);
     config_get_bool_field("stop-writes-on-bgsave-error",
             server.stop_writes_on_bgsave_err);
     config_get_bool_field("daemonize", server.daemonize);
     config_get_bool_field("rdbcompression", server.rdb_compression);
+    config_get_bool_field("rdbchecksum", server.rdb_checksum);
     config_get_bool_field("activerehashing", server.activerehashing);
 
     /* Everything we can't handle with macros follows. */
@@ -848,6 +877,9 @@ void configCommand(redisClient *c) {
         server.stat_numcommands = 0;
         server.stat_numconnections = 0;
         server.stat_expiredkeys = 0;
+        server.stat_rejected_conn = 0;
+        server.stat_fork_time = 0;
+        server.aof_delayed_fsync = 0;
         resetCommandTableStats();
         addReply(c,shared.ok);
     } else {

src/config.h

@@ -59,10 +59,11 @@
 #define	BIG_ENDIAN	4321	/* most-significant byte first (IBM, net) */
 #define	PDP_ENDIAN	3412	/* LSB first in word, MSW first in long (pdp)*/
 
-#if defined(vax) || defined(ns32000) || defined(sun386) || defined(__i386__) || \
-    defined(MIPSEL) || defined(_MIPSEL) || defined(BIT_ZERO_ON_RIGHT) || \
-    defined(__alpha__) || defined(__alpha)
-#define BYTE_ORDER	LITTLE_ENDIAN
+#if defined(__i386__) || defined(__x86_64__) || defined(__amd64__) || \
+   defined(vax) || defined(ns32000) || defined(sun386) || \
+   defined(MIPSEL) || defined(_MIPSEL) || defined(BIT_ZERO_ON_RIGHT) || \
+   defined(__alpha__) || defined(__alpha)
+#define BYTE_ORDER    LITTLE_ENDIAN
 #endif
 
 #if defined(sel) || defined(pyr) || defined(mc68000) || defined(sparc) || \

src/crc16.c

@@ -1,87 +0,0 @@
-#include "redis.h"
-
-/*      
- * Copyright 2001-2010 Georges Menie (www.menie.org)
- * Copyright 2010 Salvatore Sanfilippo (adapted to Redis coding style)
- * All rights reserved.
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- *     * Redistributions of source code must retain the above copyright
- *       notice, this list of conditions and the following disclaimer.
- *     * Redistributions in binary form must reproduce the above copyright
- *       notice, this list of conditions and the following disclaimer in the
- *       documentation and/or other materials provided with the distribution.
- *     * Neither the name of the University of California, Berkeley nor the
- *       names of its contributors may be used to endorse or promote products
- *       derived from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND ANY
- * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
- * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
- * DISCLAIMED. IN NO EVENT SHALL THE REGENTS AND CONTRIBUTORS BE LIABLE FOR ANY
- * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
- * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
- * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
- * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-/* CRC16 implementation acording to CCITT standards.
- *
- * Note by @antirez: this is actually the XMODEM CRC 16 algorithm, using the
- * following parameters:
- *
- * Name                       : "XMODEM", also known as "ZMODEM", "CRC-16/ACORN"
- * Width                      : 16 bit
- * Poly                       : 1021 (That is actually x^16 + x^12 + x^5 + 1)
- * Initialization             : 0000
- * Reflect Input byte         : False
- * Reflect Output CRC         : False
- * Xor constant to output CRC : 0000
- * Output for "123456789"     : 31C3
- */
-
-static const uint16_t crc16tab[256]= {
-    0x0000,0x1021,0x2042,0x3063,0x4084,0x50a5,0x60c6,0x70e7,
-    0x8108,0x9129,0xa14a,0xb16b,0xc18c,0xd1ad,0xe1ce,0xf1ef,
-    0x1231,0x0210,0x3273,0x2252,0x52b5,0x4294,0x72f7,0x62d6,
-    0x9339,0x8318,0xb37b,0xa35a,0xd3bd,0xc39c,0xf3ff,0xe3de,
-    0x2462,0x3443,0x0420,0x1401,0x64e6,0x74c7,0x44a4,0x5485,
-    0xa56a,0xb54b,0x8528,0x9509,0xe5ee,0xf5cf,0xc5ac,0xd58d,
-    0x3653,0x2672,0x1611,0x0630,0x76d7,0x66f6,0x5695,0x46b4,
-    0xb75b,0xa77a,0x9719,0x8738,0xf7df,0xe7fe,0xd79d,0xc7bc,
-    0x48c4,0x58e5,0x6886,0x78a7,0x0840,0x1861,0x2802,0x3823,
-    0xc9cc,0xd9ed,0xe98e,0xf9af,0x8948,0x9969,0xa90a,0xb92b,
-    0x5af5,0x4ad4,0x7ab7,0x6a96,0x1a71,0x0a50,0x3a33,0x2a12,
-    0xdbfd,0xcbdc,0xfbbf,0xeb9e,0x9b79,0x8b58,0xbb3b,0xab1a,
-    0x6ca6,0x7c87,0x4ce4,0x5cc5,0x2c22,0x3c03,0x0c60,0x1c41,
-    0xedae,0xfd8f,0xcdec,0xddcd,0xad2a,0xbd0b,0x8d68,0x9d49,
-    0x7e97,0x6eb6,0x5ed5,0x4ef4,0x3e13,0x2e32,0x1e51,0x0e70,
-    0xff9f,0xefbe,0xdfdd,0xcffc,0xbf1b,0xaf3a,0x9f59,0x8f78,
-    0x9188,0x81a9,0xb1ca,0xa1eb,0xd10c,0xc12d,0xf14e,0xe16f,
-    0x1080,0x00a1,0x30c2,0x20e3,0x5004,0x4025,0x7046,0x6067,
-    0x83b9,0x9398,0xa3fb,0xb3da,0xc33d,0xd31c,0xe37f,0xf35e,
-    0x02b1,0x1290,0x22f3,0x32d2,0x4235,0x5214,0x6277,0x7256,
-    0xb5ea,0xa5cb,0x95a8,0x8589,0xf56e,0xe54f,0xd52c,0xc50d,
-    0x34e2,0x24c3,0x14a0,0x0481,0x7466,0x6447,0x5424,0x4405,
-    0xa7db,0xb7fa,0x8799,0x97b8,0xe75f,0xf77e,0xc71d,0xd73c,
-    0x26d3,0x36f2,0x0691,0x16b0,0x6657,0x7676,0x4615,0x5634,
-    0xd94c,0xc96d,0xf90e,0xe92f,0x99c8,0x89e9,0xb98a,0xa9ab,
-    0x5844,0x4865,0x7806,0x6827,0x18c0,0x08e1,0x3882,0x28a3,
-    0xcb7d,0xdb5c,0xeb3f,0xfb1e,0x8bf9,0x9bd8,0xabbb,0xbb9a,
-    0x4a75,0x5a54,0x6a37,0x7a16,0x0af1,0x1ad0,0x2ab3,0x3a92,
-    0xfd2e,0xed0f,0xdd6c,0xcd4d,0xbdaa,0xad8b,0x9de8,0x8dc9,
-    0x7c26,0x6c07,0x5c64,0x4c45,0x3ca2,0x2c83,0x1ce0,0x0cc1,
-    0xef1f,0xff3e,0xcf5d,0xdf7c,0xaf9b,0xbfba,0x8fd9,0x9ff8,
-    0x6e17,0x7e36,0x4e55,0x5e74,0x2e93,0x3eb2,0x0ed1,0x1ef0
-};
-  
-uint16_t crc16(const char *buf, int len) {
-    int counter;
-    uint16_t crc = 0;
-    for (counter = 0; counter < len; counter++)
-            crc = (crc<<8) ^ crc16tab[((crc>>8) ^ *buf++)&0x00FF];
-    return crc;
-}

src/crc64.c

@@ -0,0 +1,191 @@
+/* Redis uses the CRC64 variant with "Jones" coefficients and init value of 0.
+ *
+ * Specification of this CRC64 variant follows:
+ * Name: crc-64-jones
+ * Width: 64 bites
+ * Poly: 0xad93d23594c935a9
+ * Reflected In: True
+ * Xor_In: 0xffffffffffffffff
+ * Reflected_Out: True
+ * Xor_Out: 0x0
+ * Check("123456789"): 0xe9c6d914c4b8d9ca
+ *
+ * Copyright (c) 2012, Salvatore Sanfilippo <antirez at gmail dot com>
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ *   * Redistributions of source code must retain the above copyright notice,
+ *     this list of conditions and the following disclaimer.
+ *   * Redistributions in binary form must reproduce the above copyright
+ *     notice, this list of conditions and the following disclaimer in the
+ *     documentation and/or other materials provided with the distribution.
+ *   * Neither the name of Redis nor the names of its contributors may be used
+ *     to endorse or promote products derived from this software without
+ *     specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE. */
+
+#include <stdint.h>
+
+static const uint64_t crc64_tab[256] = {
+    UINT64_C(0x0000000000000000), UINT64_C(0x7ad870c830358979),
+    UINT64_C(0xf5b0e190606b12f2), UINT64_C(0x8f689158505e9b8b),
+    UINT64_C(0xc038e5739841b68f), UINT64_C(0xbae095bba8743ff6),
+    UINT64_C(0x358804e3f82aa47d), UINT64_C(0x4f50742bc81f2d04),
+    UINT64_C(0xab28ecb46814fe75), UINT64_C(0xd1f09c7c5821770c),
+    UINT64_C(0x5e980d24087fec87), UINT64_C(0x24407dec384a65fe),
+    UINT64_C(0x6b1009c7f05548fa), UINT64_C(0x11c8790fc060c183),
+    UINT64_C(0x9ea0e857903e5a08), UINT64_C(0xe478989fa00bd371),
+    UINT64_C(0x7d08ff3b88be6f81), UINT64_C(0x07d08ff3b88be6f8),
+    UINT64_C(0x88b81eabe8d57d73), UINT64_C(0xf2606e63d8e0f40a),
+    UINT64_C(0xbd301a4810ffd90e), UINT64_C(0xc7e86a8020ca5077),
+    UINT64_C(0x4880fbd87094cbfc), UINT64_C(0x32588b1040a14285),
+    UINT64_C(0xd620138fe0aa91f4), UINT64_C(0xacf86347d09f188d),
+    UINT64_C(0x2390f21f80c18306), UINT64_C(0x594882d7b0f40a7f),
+    UINT64_C(0x1618f6fc78eb277b), UINT64_C(0x6cc0863448deae02),
+    UINT64_C(0xe3a8176c18803589), UINT64_C(0x997067a428b5bcf0),
+    UINT64_C(0xfa11fe77117cdf02), UINT64_C(0x80c98ebf2149567b),
+    UINT64_C(0x0fa11fe77117cdf0), UINT64_C(0x75796f2f41224489),
+    UINT64_C(0x3a291b04893d698d), UINT64_C(0x40f16bccb908e0f4),
+    UINT64_C(0xcf99fa94e9567b7f), UINT64_C(0xb5418a5cd963f206),
+    UINT64_C(0x513912c379682177), UINT64_C(0x2be1620b495da80e),
+    UINT64_C(0xa489f35319033385), UINT64_C(0xde51839b2936bafc),
+    UINT64_C(0x9101f7b0e12997f8), UINT64_C(0xebd98778d11c1e81),
+    UINT64_C(0x64b116208142850a), UINT64_C(0x1e6966e8b1770c73),
+    UINT64_C(0x8719014c99c2b083), UINT64_C(0xfdc17184a9f739fa),
+    UINT64_C(0x72a9e0dcf9a9a271), UINT64_C(0x08719014c99c2b08),
+    UINT64_C(0x4721e43f0183060c), UINT64_C(0x3df994f731b68f75),
+    UINT64_C(0xb29105af61e814fe), UINT64_C(0xc849756751dd9d87),
+    UINT64_C(0x2c31edf8f1d64ef6), UINT64_C(0x56e99d30c1e3c78f),
+    UINT64_C(0xd9810c6891bd5c04), UINT64_C(0xa3597ca0a188d57d),
+    UINT64_C(0xec09088b6997f879), UINT64_C(0x96d1784359a27100),
+    UINT64_C(0x19b9e91b09fcea8b), UINT64_C(0x636199d339c963f2),
+    UINT64_C(0xdf7adabd7a6e2d6f), UINT64_C(0xa5a2aa754a5ba416),
+    UINT64_C(0x2aca3b2d1a053f9d), UINT64_C(0x50124be52a30b6e4),
+    UINT64_C(0x1f423fcee22f9be0), UINT64_C(0x659a4f06d21a1299),
+    UINT64_C(0xeaf2de5e82448912), UINT64_C(0x902aae96b271006b),
+    UINT64_C(0x74523609127ad31a), UINT64_C(0x0e8a46c1224f5a63),
+    UINT64_C(0x81e2d7997211c1e8), UINT64_C(0xfb3aa75142244891),
+    UINT64_C(0xb46ad37a8a3b6595), UINT64_C(0xceb2a3b2ba0eecec),
+    UINT64_C(0x41da32eaea507767), UINT64_C(0x3b024222da65fe1e),
+    UINT64_C(0xa2722586f2d042ee), UINT64_C(0xd8aa554ec2e5cb97),
+    UINT64_C(0x57c2c41692bb501c), UINT64_C(0x2d1ab4dea28ed965),
+    UINT64_C(0x624ac0f56a91f461), UINT64_C(0x1892b03d5aa47d18),
+    UINT64_C(0x97fa21650afae693), UINT64_C(0xed2251ad3acf6fea),
+    UINT64_C(0x095ac9329ac4bc9b), UINT64_C(0x7382b9faaaf135e2),
+    UINT64_C(0xfcea28a2faafae69), UINT64_C(0x8632586aca9a2710),
+    UINT64_C(0xc9622c4102850a14), UINT64_C(0xb3ba5c8932b0836d),
+    UINT64_C(0x3cd2cdd162ee18e6), UINT64_C(0x460abd1952db919f),
+    UINT64_C(0x256b24ca6b12f26d), UINT64_C(0x5fb354025b277b14),
+    UINT64_C(0xd0dbc55a0b79e09f), UINT64_C(0xaa03b5923b4c69e6),
+    UINT64_C(0xe553c1b9f35344e2), UINT64_C(0x9f8bb171c366cd9b),
+    UINT64_C(0x10e3202993385610), UINT64_C(0x6a3b50e1a30ddf69),
+    UINT64_C(0x8e43c87e03060c18), UINT64_C(0xf49bb8b633338561),
+    UINT64_C(0x7bf329ee636d1eea), UINT64_C(0x012b592653589793),
+    UINT64_C(0x4e7b2d0d9b47ba97), UINT64_C(0x34a35dc5ab7233ee),
+    UINT64_C(0xbbcbcc9dfb2ca865), UINT64_C(0xc113bc55cb19211c),
+    UINT64_C(0x5863dbf1e3ac9dec), UINT64_C(0x22bbab39d3991495),
+    UINT64_C(0xadd33a6183c78f1e), UINT64_C(0xd70b4aa9b3f20667),
+    UINT64_C(0x985b3e827bed2b63), UINT64_C(0xe2834e4a4bd8a21a),
+    UINT64_C(0x6debdf121b863991), UINT64_C(0x1733afda2bb3b0e8),
+    UINT64_C(0xf34b37458bb86399), UINT64_C(0x8993478dbb8deae0),
+    UINT64_C(0x06fbd6d5ebd3716b), UINT64_C(0x7c23a61ddbe6f812),
+    UINT64_C(0x3373d23613f9d516), UINT64_C(0x49aba2fe23cc5c6f),
+    UINT64_C(0xc6c333a67392c7e4), UINT64_C(0xbc1b436e43a74e9d),
+    UINT64_C(0x95ac9329ac4bc9b5), UINT64_C(0xef74e3e19c7e40cc),
+    UINT64_C(0x601c72b9cc20db47), UINT64_C(0x1ac40271fc15523e),
+    UINT64_C(0x5594765a340a7f3a), UINT64_C(0x2f4c0692043ff643),
+    UINT64_C(0xa02497ca54616dc8), UINT64_C(0xdafce7026454e4b1),
+    UINT64_C(0x3e847f9dc45f37c0), UINT64_C(0x445c0f55f46abeb9),
+    UINT64_C(0xcb349e0da4342532), UINT64_C(0xb1eceec59401ac4b),
+    UINT64_C(0xfebc9aee5c1e814f), UINT64_C(0x8464ea266c2b0836),
+    UINT64_C(0x0b0c7b7e3c7593bd), UINT64_C(0x71d40bb60c401ac4),
+    UINT64_C(0xe8a46c1224f5a634), UINT64_C(0x927c1cda14c02f4d),
+    UINT64_C(0x1d148d82449eb4c6), UINT64_C(0x67ccfd4a74ab3dbf),
+    UINT64_C(0x289c8961bcb410bb), UINT64_C(0x5244f9a98c8199c2),
+    UINT64_C(0xdd2c68f1dcdf0249), UINT64_C(0xa7f41839ecea8b30),
+    UINT64_C(0x438c80a64ce15841), UINT64_C(0x3954f06e7cd4d138),
+    UINT64_C(0xb63c61362c8a4ab3), UINT64_C(0xcce411fe1cbfc3ca),
+    UINT64_C(0x83b465d5d4a0eece), UINT64_C(0xf96c151de49567b7),
+    UINT64_C(0x76048445b4cbfc3c), UINT64_C(0x0cdcf48d84fe7545),
+    UINT64_C(0x6fbd6d5ebd3716b7), UINT64_C(0x15651d968d029fce),
+    UINT64_C(0x9a0d8ccedd5c0445), UINT64_C(0xe0d5fc06ed698d3c),
+    UINT64_C(0xaf85882d2576a038), UINT64_C(0xd55df8e515432941),
+    UINT64_C(0x5a3569bd451db2ca), UINT64_C(0x20ed197575283bb3),
+    UINT64_C(0xc49581ead523e8c2), UINT64_C(0xbe4df122e51661bb),
+    UINT64_C(0x3125607ab548fa30), UINT64_C(0x4bfd10b2857d7349),
+    UINT64_C(0x04ad64994d625e4d), UINT64_C(0x7e7514517d57d734),
+    UINT64_C(0xf11d85092d094cbf), UINT64_C(0x8bc5f5c11d3cc5c6),
+    UINT64_C(0x12b5926535897936), UINT64_C(0x686de2ad05bcf04f),
+    UINT64_C(0xe70573f555e26bc4), UINT64_C(0x9ddd033d65d7e2bd),
+    UINT64_C(0xd28d7716adc8cfb9), UINT64_C(0xa85507de9dfd46c0),
+    UINT64_C(0x273d9686cda3dd4b), UINT64_C(0x5de5e64efd965432),
+    UINT64_C(0xb99d7ed15d9d8743), UINT64_C(0xc3450e196da80e3a),
+    UINT64_C(0x4c2d9f413df695b1), UINT64_C(0x36f5ef890dc31cc8),
+    UINT64_C(0x79a59ba2c5dc31cc), UINT64_C(0x037deb6af5e9b8b5),
+    UINT64_C(0x8c157a32a5b7233e), UINT64_C(0xf6cd0afa9582aa47),
+    UINT64_C(0x4ad64994d625e4da), UINT64_C(0x300e395ce6106da3),
+    UINT64_C(0xbf66a804b64ef628), UINT64_C(0xc5bed8cc867b7f51),
+    UINT64_C(0x8aeeace74e645255), UINT64_C(0xf036dc2f7e51db2c),
+    UINT64_C(0x7f5e4d772e0f40a7), UINT64_C(0x05863dbf1e3ac9de),
+    UINT64_C(0xe1fea520be311aaf), UINT64_C(0x9b26d5e88e0493d6),
+    UINT64_C(0x144e44b0de5a085d), UINT64_C(0x6e963478ee6f8124),
+    UINT64_C(0x21c640532670ac20), UINT64_C(0x5b1e309b16452559),
+    UINT64_C(0xd476a1c3461bbed2), UINT64_C(0xaeaed10b762e37ab),
+    UINT64_C(0x37deb6af5e9b8b5b), UINT64_C(0x4d06c6676eae0222),
+    UINT64_C(0xc26e573f3ef099a9), UINT64_C(0xb8b627f70ec510d0),
+    UINT64_C(0xf7e653dcc6da3dd4), UINT64_C(0x8d3e2314f6efb4ad),
+    UINT64_C(0x0256b24ca6b12f26), UINT64_C(0x788ec2849684a65f),
+    UINT64_C(0x9cf65a1b368f752e), UINT64_C(0xe62e2ad306bafc57),
+    UINT64_C(0x6946bb8b56e467dc), UINT64_C(0x139ecb4366d1eea5),
+    UINT64_C(0x5ccebf68aecec3a1), UINT64_C(0x2616cfa09efb4ad8),
+    UINT64_C(0xa97e5ef8cea5d153), UINT64_C(0xd3a62e30fe90582a),
+    UINT64_C(0xb0c7b7e3c7593bd8), UINT64_C(0xca1fc72bf76cb2a1),
+    UINT64_C(0x45775673a732292a), UINT64_C(0x3faf26bb9707a053),
+    UINT64_C(0x70ff52905f188d57), UINT64_C(0x0a2722586f2d042e),
+    UINT64_C(0x854fb3003f739fa5), UINT64_C(0xff97c3c80f4616dc),
+    UINT64_C(0x1bef5b57af4dc5ad), UINT64_C(0x61372b9f9f784cd4),
+    UINT64_C(0xee5fbac7cf26d75f), UINT64_C(0x9487ca0fff135e26),
+    UINT64_C(0xdbd7be24370c7322), UINT64_C(0xa10fceec0739fa5b),
+    UINT64_C(0x2e675fb4576761d0), UINT64_C(0x54bf2f7c6752e8a9),
+    UINT64_C(0xcdcf48d84fe75459), UINT64_C(0xb71738107fd2dd20),
+    UINT64_C(0x387fa9482f8c46ab), UINT64_C(0x42a7d9801fb9cfd2),
+    UINT64_C(0x0df7adabd7a6e2d6), UINT64_C(0x772fdd63e7936baf),
+    UINT64_C(0xf8474c3bb7cdf024), UINT64_C(0x829f3cf387f8795d),
+    UINT64_C(0x66e7a46c27f3aa2c), UINT64_C(0x1c3fd4a417c62355),
+    UINT64_C(0x935745fc4798b8de), UINT64_C(0xe98f353477ad31a7),
+    UINT64_C(0xa6df411fbfb21ca3), UINT64_C(0xdc0731d78f8795da),
+    UINT64_C(0x536fa08fdfd90e51), UINT64_C(0x29b7d047efec8728),
+};
+
+uint64_t crc64(uint64_t crc, const unsigned char *s, uint64_t l) {
+    uint64_t j;
+
+    for (j = 0; j < l; j++) {
+        uint8_t byte = s[j];
+        crc = crc64_tab[(uint8_t)crc ^ byte] ^ (crc >> 8);
+    }
+    return crc;
+}
+
+/* Test main */
+#ifdef TEST_MAIN
+#include <stdio.h>
+int main(void) {
+    printf("e9c6d914c4b8d9ca == %016llx\n",
+        (unsigned long long) crc64(0,(unsigned char*)"123456789",9));
+    return 0;
+}
+#endif

src/db.c

@@ -10,28 +10,6 @@ void SlotToKeyDel(robj *key);
  * C-level DB API
  *----------------------------------------------------------------------------*/
 
-/* Important notes on lookup and disk store.
- *
- * When disk store is enabled on lookup we can have different cases.
- *
- * a) The key is in memory:
- *    - If the key is not in IO_SAVEINPROG state we can access it.
- *      As if it's just IO_SAVE this means we have the key in the IO queue
- *      but can't be accessed by the IO thread (it requires to be
- *      translated into an IO Job by the cache cron function.)
- *    - If the key is in IO_SAVEINPROG we can't touch the key and have
- *      to blocking wait completion of operations.
- * b) The key is not in memory:
- *    - If it's marked as non existing on disk as well (negative cache)
- *      we don't need to perform the disk access.
- *    - if the key MAY EXIST, but is not in memory, and it is marked as IO_SAVE
- *      then the key can only be a deleted one. As IO_SAVE keys are never
- *      evicted (dirty state), so the only possibility is that key was deleted.
- *    - if the key MAY EXIST we need to blocking load it.
- *      We check that the key is not in IO_SAVEINPROG state before accessing
- *      the disk object. If it is in this state, we wait.
- */
-
 robj *lookupKey(redisDb *db, robj *key) {
     dictEntry *de = dictFind(db->dict,key->ptr);
     if (de) {
@@ -86,7 +64,6 @@ void dbAdd(redisDb *db, robj *key, robj *val) {
     int retval = dictAdd(db->dict, copy, val);
 
     redisAssertWithInfo(NULL,key,retval == REDIS_OK);
-    if (server.cluster_enabled) SlotToKeyAdd(key);
  }
 
 /* Overwrite an existing key with a new value. Incrementing the reference
@@ -154,15 +131,12 @@ int dbDelete(redisDb *db, robj *key) {
      * the key, because it is shared with the main dictionary. */
     if (dictSize(db->expires) > 0) dictDelete(db->expires,key->ptr);
     if (dictDelete(db->dict,key->ptr) == DICT_OK) {
-        if (server.cluster_enabled) SlotToKeyDel(key);
         return 1;
     } else {
         return 0;
     }
 }
 
-/* Empty the whole database.
- * If diskstore is enabled this function will just flush the in-memory cache. */
 long long emptyDb() {
     int j;
     long long removed = 0;
@@ -254,10 +228,6 @@ void existsCommand(redisClient *c) {
 void selectCommand(redisClient *c) {
     int id = atoi(c->argv[1]->ptr);
 
-    if (server.cluster_enabled && id != 0) {
-        addReplyError(c,"SELECT is not allowed in cluster mode");
-        return;
-    }
     if (selectDb(c,id) == REDIS_ERR) {
         addReplyError(c,"invalid DB index");
     } else {
@@ -398,11 +368,6 @@ void moveCommand(redisClient *c) {
     redisDb *src, *dst;
     int srcid;
 
-    if (server.cluster_enabled) {
-        addReplyError(c,"MOVE is not allowed in cluster mode");
-        return;
-    }
-
     /* Obtain source and target DB pointers */
     src = c->db;
     srcid = c->db->id;
@@ -518,7 +483,7 @@ int expireIfNeeded(redisDb *db, robj *key) {
      * that is, 0 if we think the key should be still valid, 1 if
      * we think the key is expired at this time. */
     if (server.masterhost != NULL) {
-        return time(NULL) > when;
+        return mstime() > when;
     }
 
     /* Return when this key has not expired */
@@ -534,24 +499,23 @@ int expireIfNeeded(redisDb *db, robj *key) {
  * Expires Commands
  *----------------------------------------------------------------------------*/
 
-/* Given an string object return true if it contains exactly the "ms"
- * or "MS" string. This is used in order to check if the last argument
- * of EXPIRE, EXPIREAT or TTL is "ms" to switch into millisecond input/output */
-int stringObjectEqualsMs(robj *a) {
-    char *arg = a->ptr;
-    return tolower(arg[0]) == 'm' && tolower(arg[1]) == 's' && arg[2] == '\0';
-}
-
-void expireGenericCommand(redisClient *c, long long offset, int unit) {
+/* This is the generic command implementation for EXPIRE, PEXPIRE, EXPIREAT
+ * and PEXPIREAT. Because the commad second argument may be relative or absolute
+ * the "basetime" argument is used to signal what the base time is (either 0
+ * for *AT variants of the command, or the current time for relative expires).
+ *
+ * unit is either UNIT_SECONDS or UNIT_MILLISECONDS, and is only used for
+ * the argv[2] parameter. The basetime is always specified in milliesconds. */
+void expireGenericCommand(redisClient *c, long long basetime, int unit) {
     dictEntry *de;
     robj *key = c->argv[1], *param = c->argv[2];
-    long long milliseconds;
+    long long when; /* unix time in milliseconds when the key will expire. */
 
-    if (getLongLongFromObjectOrReply(c, param, &milliseconds, NULL) != REDIS_OK)
+    if (getLongLongFromObjectOrReply(c, param, &when, NULL) != REDIS_OK)
         return;
 
-    if (unit == UNIT_SECONDS) milliseconds *= 1000;
-    milliseconds -= offset;
+    if (unit == UNIT_SECONDS) when *= 1000;
+    when += basetime;
 
     de = dictFind(c->db->dict,key->ptr);
     if (de == NULL) {
@@ -564,7 +528,7 @@ void expireGenericCommand(redisClient *c, long long offset, int unit) {
      *
      * Instead we take the other branch of the IF statement setting an expire
      * (possibly in the past) and wait for an explicit DEL from the master. */
-    if (milliseconds <= 0 && !server.loading && !server.masterhost) {
+    if (when <= mstime() && !server.loading && !server.masterhost) {
         robj *aux;
 
         redisAssertWithInfo(c,key,dbDelete(c->db,key));
@@ -578,7 +542,6 @@ void expireGenericCommand(redisClient *c, long long offset, int unit) {
         addReply(c, shared.cone);
         return;
     } else {
-        long long when = mstime()+milliseconds;
         setExpire(c->db,key,when);
         addReply(c,shared.cone);
         signalModifiedKey(c->db,key);
@@ -588,19 +551,19 @@ void expireGenericCommand(redisClient *c, long long offset, int unit) {
 }
 
 void expireCommand(redisClient *c) {
-    expireGenericCommand(c,0,UNIT_SECONDS);
-}
-
-void expireatCommand(redisClient *c) {
     expireGenericCommand(c,mstime(),UNIT_SECONDS);
 }
 
+void expireatCommand(redisClient *c) {
+    expireGenericCommand(c,0,UNIT_SECONDS);
+}
+
 void pexpireCommand(redisClient *c) {
-    expireGenericCommand(c,0,UNIT_MILLISECONDS);
+    expireGenericCommand(c,mstime(),UNIT_MILLISECONDS);
 }
 
 void pexpireatCommand(redisClient *c) {
-    expireGenericCommand(c,mstime(),UNIT_MILLISECONDS);
+    expireGenericCommand(c,0,UNIT_MILLISECONDS);
 }
 
 void ttlGenericCommand(redisClient *c, int output_ms) {
@@ -714,35 +677,3 @@ int *zunionInterGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *num
     *numkeys = num;
     return keys;
 }
-
-/* Slot to Key API. This is used by Redis Cluster in order to obtain in
- * a fast way a key that belongs to a specified hash slot. This is useful
- * while rehashing the cluster. */
-void SlotToKeyAdd(robj *key) {
-    unsigned int hashslot = keyHashSlot(key->ptr,sdslen(key->ptr));
-
-    zslInsert(server.cluster.slots_to_keys,hashslot,key);
-    incrRefCount(key);
-}
-
-void SlotToKeyDel(robj *key) {
-    unsigned int hashslot = keyHashSlot(key->ptr,sdslen(key->ptr));
-
-    zslDelete(server.cluster.slots_to_keys,hashslot,key);
-}
-
-unsigned int GetKeysInSlot(unsigned int hashslot, robj **keys, unsigned int count) {
-    zskiplistNode *n;
-    zrangespec range;
-    int j = 0;
-
-    range.min = range.max = hashslot;
-    range.minex = range.maxex = 0;
-    
-    n = zslFirstInRange(server.cluster.slots_to_keys, range);
-    while(n && n->score == hashslot && count--) {
-        keys[j++] = n->obj;
-        n = n->level[0].forward;
-    }
-    return j;
-}

src/debug.c

@@ -7,6 +7,7 @@
 #ifdef HAVE_BACKTRACE
 #include <execinfo.h>
 #include <ucontext.h>
+#include <fcntl.h>
 #endif /* HAVE_BACKTRACE */
 
 /* ================================= Debugging ============================== */
@@ -105,7 +106,6 @@ void computeDatasetDigest(unsigned char *final) {
 
             mixDigest(digest,key,sdslen(key));
 
-            /* Make sure the key is loaded if VM is active */
             o = dictGetVal(de);
 
             aux = htonl(o->type);
@@ -399,30 +399,31 @@ void bugReportStart(void) {
 
 #ifdef HAVE_BACKTRACE
 static void *getMcontextEip(ucontext_t *uc) {
-#if defined(__FreeBSD__)
-    return (void*) uc->uc_mcontext.mc_eip;
-#elif defined(__dietlibc__)
-    return (void*) uc->uc_mcontext.eip;
-#elif defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
-  #if __x86_64__
+#if defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
+    /* OSX < 10.6 */
+    #if defined(__x86_64__)
     return (void*) uc->uc_mcontext->__ss.__rip;
-  #elif __i386__
+    #elif defined(__i386__)
     return (void*) uc->uc_mcontext->__ss.__eip;
-  #else
+    #else
     return (void*) uc->uc_mcontext->__ss.__srr0;
-  #endif
+    #endif
 #elif defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)
-  #if defined(_STRUCT_X86_THREAD_STATE64) && !defined(__i386__)
+    /* OSX >= 10.6 */
+    #if defined(_STRUCT_X86_THREAD_STATE64) && !defined(__i386__)
     return (void*) uc->uc_mcontext->__ss.__rip;
-  #else
+    #else
     return (void*) uc->uc_mcontext->__ss.__eip;
-  #endif
-#elif defined(__i386__)
+    #endif
+#elif defined(__linux__)
+    /* Linux */
+    #if defined(__i386__)
     return (void*) uc->uc_mcontext.gregs[14]; /* Linux 32 */
-#elif defined(__X86_64__) || defined(__x86_64__)
+    #elif defined(__X86_64__) || defined(__x86_64__)
     return (void*) uc->uc_mcontext.gregs[16]; /* Linux 64 */
-#elif defined(__ia64__) /* Linux IA64 */
+    #elif defined(__ia64__) /* Linux IA64 */
     return (void*) uc->uc_mcontext.sc_ip;
+    #endif
 #else
     return NULL;
 #endif
@@ -440,8 +441,11 @@ void logStackContent(void **sp) {
 
 void logRegisters(ucontext_t *uc) {
     redisLog(REDIS_WARNING, "--- REGISTERS");
+
+/* OSX */
 #if defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)
-  #if defined(_STRUCT_X86_THREAD_STATE64) && !defined(__i386__)
+  /* OSX AMD64 */
+    #if defined(_STRUCT_X86_THREAD_STATE64) && !defined(__i386__)
     redisLog(REDIS_WARNING,
     "\n"
     "RAX:%016lx RBX:%016lx\nRCX:%016lx RDX:%016lx\n"
@@ -472,7 +476,8 @@ void logRegisters(ucontext_t *uc) {
         uc->uc_mcontext->__ss.__gs
     );
     logStackContent((void**)uc->uc_mcontext->__ss.__rsp);
-  #else
+    #else
+    /* OSX x86 */
     redisLog(REDIS_WARNING,
     "\n"
     "EAX:%08lx EBX:%08lx ECX:%08lx EDX:%08lx\n"
@@ -497,8 +502,11 @@ void logRegisters(ucontext_t *uc) {
         uc->uc_mcontext->__ss.__gs
     );
     logStackContent((void**)uc->uc_mcontext->__ss.__esp);
-  #endif
-#elif defined(__i386__)
+    #endif
+/* Linux */
+#elif defined(__linux__)
+    /* Linux x86 */
+    #if defined(__i386__)
     redisLog(REDIS_WARNING,
     "\n"
     "EAX:%08lx EBX:%08lx ECX:%08lx EDX:%08lx\n"
@@ -523,7 +531,8 @@ void logRegisters(ucontext_t *uc) {
         uc->uc_mcontext.gregs[0]
     );
     logStackContent((void**)uc->uc_mcontext.gregs[7]);
-#elif defined(__X86_64__) || defined(__x86_64__)
+    #elif defined(__X86_64__) || defined(__x86_64__)
+    /* Linux AMD64 */
     redisLog(REDIS_WARNING,
     "\n"
     "RAX:%016lx RBX:%016lx\nRCX:%016lx RDX:%016lx\n"
@@ -552,16 +561,78 @@ void logRegisters(ucontext_t *uc) {
         uc->uc_mcontext.gregs[18]
     );
     logStackContent((void**)uc->uc_mcontext.gregs[15]);
+    #endif
 #else
     redisLog(REDIS_WARNING,
         "  Dumping of registers not supported for this OS/arch");
 #endif
 }
 
-void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
+/* Logs the stack trace using the backtrace() call. This function is designed
+ * to be called from signal handlers safely. */
+void logStackTrace(ucontext_t *uc) {
     void *trace[100];
-    char **messages = NULL;
-    int i, trace_size = 0;
+    int trace_size = 0, fd;
+
+    /* Open the log file in append mode. */
+    fd = server.logfile ?
+        open(server.logfile, O_APPEND|O_CREAT|O_WRONLY, 0644) :
+        STDOUT_FILENO;
+    if (fd == -1) return;
+
+    /* Generate the stack trace */
+    trace_size = backtrace(trace, 100);
+
+    /* overwrite sigaction with caller's address */
+    if (getMcontextEip(uc) != NULL)
+        trace[1] = getMcontextEip(uc);
+
+    /* Write symbols to log file */
+    backtrace_symbols_fd(trace, trace_size, fd);
+
+    /* Cleanup */
+    if (server.logfile) close(fd);
+}
+
+/* Log information about the "current" client, that is, the client that is
+ * currently being served by Redis. May be NULL if Redis is not serving a
+ * client right now. */
+void logCurrentClient(void) {
+    if (server.current_client == NULL) return;
+
+    redisClient *cc = server.current_client;
+    sds client;
+    int j;
+
+    redisLog(REDIS_WARNING, "--- CURRENT CLIENT INFO");
+    client = getClientInfoString(cc);
+    redisLog(REDIS_WARNING,"client: %s", client);
+    sdsfree(client);
+    for (j = 0; j < cc->argc; j++) {
+        robj *decoded;
+
+        decoded = getDecodedObject(cc->argv[j]);
+        redisLog(REDIS_WARNING,"argv[%d]: '%s'", j, (char*)decoded->ptr);
+        decrRefCount(decoded);
+    }
+    /* Check if the first argument, usually a key, is found inside the
+     * selected DB, and if so print info about the associated object. */
+    if (cc->argc >= 1) {
+        robj *val, *key;
+        dictEntry *de;
+
+        key = getDecodedObject(cc->argv[1]);
+        de = dictFind(cc->db->dict, key->ptr);
+        if (de) {
+            val = dictGetVal(de);
+            redisLog(REDIS_WARNING,"key '%s' found in DB containing the following object:", key->ptr);
+            redisLogObjectDebugInfo(val);
+        }
+        decrRefCount(key);
+    }
+}
+
+void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
     ucontext_t *uc = (ucontext_t*) secret;
     sds infostring, clients;
     struct sigaction act;
@@ -574,17 +645,9 @@ void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
         "    Failed assertion: %s (%s:%d)", server.assert_failed,
                         server.assert_file, server.assert_line);
 
-    /* Generate the stack trace */
-    trace_size = backtrace(trace, 100);
-
-    /* overwrite sigaction with caller's address */
-    if (getMcontextEip(uc) != NULL) {
-        trace[1] = getMcontextEip(uc);
-    }
-    messages = backtrace_symbols(trace, trace_size);
+    /* Log the stack trace */
     redisLog(REDIS_WARNING, "--- STACK TRACE");
-    for (i=1; i<trace_size; ++i)
-        redisLog(REDIS_WARNING,"%s", messages[i]);
+    logStackTrace(uc);
 
     /* Log INFO and CLIENT LIST */
     redisLog(REDIS_WARNING, "--- INFO OUTPUT");
@@ -595,49 +658,20 @@ void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
     redisLog(REDIS_WARNING, "--- CLIENT LIST OUTPUT");
     clients = getAllClientsInfoString();
     redisLogRaw(REDIS_WARNING, clients);
-    /* Don't sdsfree() strings to avoid a crash. Memory may be corrupted. */
+    sdsfree(infostring);
+    sdsfree(clients);
 
-    /* Log CURRENT CLIENT info */
-    if (server.current_client) {
-        redisClient *cc = server.current_client;
-        sds client;
-        int j;
-
-        redisLog(REDIS_WARNING, "--- CURRENT CLIENT INFO");
-        client = getClientInfoString(cc);
-        redisLog(REDIS_WARNING,"client: %s", client);
-        /* Missing sdsfree(client) to avoid crash if memory is corrupted. */
-        for (j = 0; j < cc->argc; j++) {
-            robj *decoded;
-
-            decoded = getDecodedObject(cc->argv[j]);
-            redisLog(REDIS_WARNING,"argv[%d]: '%s'", j, (char*)decoded->ptr);
-            decrRefCount(decoded);
-        }
-        /* Check if the first argument, usually a key, is found inside the
-         * selected DB, and if so print info about the associated object. */
-        if (cc->argc >= 1) {
-            robj *val, *key;
-            dictEntry *de;
-
-            key = getDecodedObject(cc->argv[1]);
-            de = dictFind(cc->db->dict, key->ptr);
-            if (de) {
-                val = dictGetVal(de);
-                redisLog(REDIS_WARNING,"key '%s' found in DB containing the following object:", key->ptr);
-                redisLogObjectDebugInfo(val);
-            }
-            decrRefCount(key);
-        }
-    }
+    /* Log the current client */
+    logCurrentClient();
 
     /* Log dump of processor registers */
     logRegisters(uc);
 
     redisLog(REDIS_WARNING,
 "\n=== REDIS BUG REPORT END. Make sure to include from START to END. ===\n\n"
-"    Please report the crash opening an issue on github:\n\n"
-"        http://github.com/antirez/redis/issues\n\n"
+"       Please report the crash opening an issue on github:\n\n"
+"           http://github.com/antirez/redis/issues\n\n"
+"  Suspect RAM error? Use redis-server --test-memory to veryfy it.\n\n"
 );
     /* free(messages); Don't call free() with possibly corrupted memory. */
     if (server.daemonize) unlink(server.pidfile);
@@ -645,11 +679,75 @@ void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
     /* Make sure we exit with the right signal at the end. So for instance
      * the core will be dumped if enabled. */
     sigemptyset (&act.sa_mask);
-    /* When the SA_SIGINFO flag is set in sa_flags then sa_sigaction
-     * is used. Otherwise, sa_handler is used */
     act.sa_flags = SA_NODEFER | SA_ONSTACK | SA_RESETHAND;
     act.sa_handler = SIG_DFL;
     sigaction (sig, &act, NULL);
     kill(getpid(),sig);
 }
 #endif /* HAVE_BACKTRACE */
+
+/* =========================== Software Watchdog ============================ */
+#include <sys/time.h>
+
+void watchdogSignalHandler(int sig, siginfo_t *info, void *secret) {
+#ifdef HAVE_BACKTRACE
+    ucontext_t *uc = (ucontext_t*) secret;
+#endif
+    REDIS_NOTUSED(info);
+    REDIS_NOTUSED(sig);
+
+    redisLogFromHandler(REDIS_WARNING,"\n--- WATCHDOG TIMER EXPIRED ---");
+#ifdef HAVE_BACKTRACE
+    logStackTrace(uc);
+#else
+    redisLogFromHandler(REDIS_WARNING,"Sorry: no support for backtrace().");
+#endif
+    redisLogFromHandler(REDIS_WARNING,"--------\n");
+}
+
+/* Schedule a SIGALRM delivery after the specified period in milliseconds.
+ * If a timer is already scheduled, this function will re-schedule it to the
+ * specified time. If period is 0 the current timer is disabled. */
+void watchdogScheduleSignal(int period) {
+    struct itimerval it;
+
+    /* Will stop the timer if period is 0. */
+    it.it_value.tv_sec = period/1000;
+    it.it_value.tv_usec = (period%1000)*1000;
+    /* Don't automatically restart. */
+    it.it_interval.tv_sec = 0;
+    it.it_interval.tv_usec = 0;
+    setitimer(ITIMER_REAL, &it, NULL);
+}
+
+/* Enable the software watchdong with the specified period in milliseconds. */
+void enableWatchdog(int period) {
+    if (server.watchdog_period == 0) {
+        struct sigaction act;
+
+        /* Watchdog was actually disabled, so we have to setup the signal
+         * handler. */
+        sigemptyset(&act.sa_mask);
+        act.sa_flags = SA_NODEFER | SA_ONSTACK | SA_SIGINFO;
+        act.sa_sigaction = watchdogSignalHandler;
+        sigaction(SIGALRM, &act, NULL);
+    }
+    if (period < 200) period = 200; /* We don't accept periods < 200 ms. */
+    watchdogScheduleSignal(period); /* Adjust the current timer. */
+    server.watchdog_period = period;
+}
+
+/* Disable the software watchdog. */
+void disableWatchdog(void) {
+    struct sigaction act;
+    if (server.watchdog_period == 0) return; /* Already disabled. */
+    watchdogScheduleSignal(0); /* Stop the current timer. */
+
+    /* Set the signal handler to SIG_IGN, this will also remove pending
+     * signals from the queue. */
+    sigemptyset(&act.sa_mask);
+    act.sa_flags = 0;
+    act.sa_handler = SIG_IGN;
+    sigaction(SIGALRM, &act, NULL);
+    server.watchdog_period = 0;
+}

src/dict.c

@@ -150,7 +150,7 @@ int _dictInit(dict *d, dictType *type,
 }
 
 /* Resize the table to the minimal size that contains all the elements,
- * but with the invariant of a USER/BUCKETS ratio near to <= 1 */
+ * but with the invariant of a USED/BUCKETS ratio near to <= 1 */
 int dictResize(dict *d)
 {
     int minimal;
@@ -633,6 +633,21 @@ void dictEmpty(dict *d) {
     d->iterators = 0;
 }
 
+void dictEnableResize(void) {
+    dict_can_resize = 1;
+}
+
+void dictDisableResize(void) {
+    dict_can_resize = 0;
+}
+
+#if 0
+
+/* The following is code that we don't use for Redis currently, but that is part
+of the library. */
+
+/* ----------------------- Debugging ------------------------*/
+
 #define DICT_STATS_VECTLEN 50
 static void _dictPrintStatsHt(dictht *ht) {
     unsigned long i, slots = 0, chainlen, maxchainlen = 0;
@@ -686,20 +701,6 @@ void dictPrintStats(dict *d) {
     }
 }
 
-void dictEnableResize(void) {
-    dict_can_resize = 1;
-}
-
-void dictDisableResize(void) {
-    dict_can_resize = 0;
-}
-
-#if 0
-
-/* The following are just example hash table types implementations.
- * Not useful for Redis so they are commented out.
- */
-
 /* ----------------------- StringCopy Hash Table Type ------------------------*/
 
 static unsigned int _dictStringCopyHTHashFunction(const void *key)

src/help.h

@@ -1,4 +1,4 @@
-/* Automatically generated by generate-command-help.rb, do not edit. */
+/* Automatically generated by utils/generate-command-help.rb, do not edit. */
 
 #ifndef __REDIS_HELP_H
 #define __REDIS_HELP_H
@@ -13,7 +13,8 @@ static char *commandGroups[] = {
     "pubsub",
     "transactions",
     "connection",
-    "server"
+    "server",
+    "scripting"
 };
 
 struct commandHelp {
@@ -27,612 +28,697 @@ struct commandHelp {
     "key value",
     "Append a value to a key",
     1,
-    "1.3.3" },
+    "2.0.0" },
     { "AUTH",
     "password",
     "Authenticate to the server",
     8,
-    "0.08" },
+    "1.0.0" },
     { "BGREWRITEAOF",
     "-",
     "Asynchronously rewrite the append-only file",
     9,
-    "1.07" },
+    "1.0.0" },
     { "BGSAVE",
     "-",
     "Asynchronously save the dataset to disk",
     9,
-    "0.07" },
+    "1.0.0" },
     { "BLPOP",
     "key [key ...] timeout",
     "Remove and get the first element in a list, or block until one is available",
     2,
-    "1.3.1" },
+    "2.0.0" },
     { "BRPOP",
     "key [key ...] timeout",
     "Remove and get the last element in a list, or block until one is available",
     2,
-    "1.3.1" },
+    "2.0.0" },
     { "BRPOPLPUSH",
     "source destination timeout",
     "Pop a value from a list, push it to another list and return it; or block until one is available",
     2,
-    "2.1.7" },
+    "2.2.0" },
     { "CONFIG GET",
     "parameter",
     "Get the value of a configuration parameter",
     9,
-    "2.0" },
+    "2.0.0" },
     { "CONFIG RESETSTAT",
     "-",
     "Reset the stats returned by INFO",
     9,
-    "2.0" },
+    "2.0.0" },
     { "CONFIG SET",
     "parameter value",
     "Set a configuration parameter to the given value",
     9,
-    "2.0" },
+    "2.0.0" },
     { "DBSIZE",
     "-",
     "Return the number of keys in the selected database",
     9,
-    "0.07" },
+    "1.0.0" },
     { "DEBUG OBJECT",
     "key",
     "Get debugging information about a key",
     9,
-    "0.101" },
+    "1.0.0" },
     { "DEBUG SEGFAULT",
     "-",
     "Make the server crash",
     9,
-    "0.101" },
+    "1.0.0" },
     { "DECR",
     "key",
     "Decrement the integer value of a key by one",
     1,
-    "0.07" },
+    "1.0.0" },
     { "DECRBY",
     "key decrement",
     "Decrement the integer value of a key by the given number",
     1,
-    "0.07" },
+    "1.0.0" },
     { "DEL",
     "key [key ...]",
     "Delete a key",
     0,
-    "0.07" },
+    "1.0.0" },
     { "DISCARD",
     "-",
     "Discard all commands issued after MULTI",
     7,
-    "1.3.3" },
+    "2.0.0" },
+    { "DUMP",
+    "key",
+    "Return a serialized verison of the value stored at the specified key.",
+    0,
+    "2.6.0" },
     { "ECHO",
     "message",
     "Echo the given string",
     8,
-    "0.07" },
+    "1.0.0" },
+    { "EVAL",
+    "script numkeys key [key ...] arg [arg ...]",
+    "Execute a Lua script server side",
+    10,
+    "2.6.0" },
     { "EXEC",
     "-",
     "Execute all commands issued after MULTI",
     7,
-    "1.1.95" },
+    "1.2.0" },
     { "EXISTS",
     "key",
     "Determine if a key exists",
-    9,
-    "0.07" },
+    0,
+    "1.0.0" },
     { "EXPIRE",
     "key seconds",
     "Set a key's time to live in seconds",
     0,
-    "0.09" },
+    "1.0.0" },
     { "EXPIREAT",
     "key timestamp",
     "Set the expiration for a key as a UNIX timestamp",
     0,
-    "1.1" },
+    "1.2.0" },
     { "FLUSHALL",
     "-",
     "Remove all keys from all databases",
     9,
-    "0.07" },
+    "1.0.0" },
     { "FLUSHDB",
     "-",
     "Remove all keys from the current database",
     9,
-    "0.07" },
+    "1.0.0" },
     { "GET",
     "key",
     "Get the value of a key",
     1,
-    "0.07" },
+    "1.0.0" },
     { "GETBIT",
     "key offset",
     "Returns the bit value at offset in the string value stored at key",
     1,
-    "2.1.8" },
+    "2.2.0" },
+    { "GETRANGE",
+    "key start end",
+    "Get a substring of the string stored at a key",
+    1,
+    "2.4.0" },
     { "GETSET",
     "key value",
     "Set the string value of a key and return its old value",
     1,
-    "0.091" },
+    "1.0.0" },
     { "HDEL",
-    "key field",
-    "Delete a hash field",
+    "key field [field ...]",
+    "Delete one or more hash fields",
     5,
-    "1.3.10" },
+    "2.0.0" },
     { "HEXISTS",
     "key field",
     "Determine if a hash field exists",
     5,
-    "1.3.10" },
+    "2.0.0" },
     { "HGET",
     "key field",
     "Get the value of a hash field",
     5,
-    "1.3.10" },
+    "2.0.0" },
     { "HGETALL",
     "key",
     "Get all the fields and values in a hash",
     5,
-    "1.3.10" },
+    "2.0.0" },
     { "HINCRBY",
     "key field increment",
     "Increment the integer value of a hash field by the given number",
     5,
-    "1.3.10" },
+    "2.0.0" },
+    { "HINCRBYFLOAT",
+    "key field increment",
+    "Increment the float value of a hash field by the given amount",
+    5,
+    "2.6.0" },
     { "HKEYS",
     "key",
     "Get all the fields in a hash",
     5,
-    "1.3.10" },
+    "2.0.0" },
     { "HLEN",
     "key",
     "Get the number of fields in a hash",
     5,
-    "1.3.10" },
+    "2.0.0" },
     { "HMGET",
     "key field [field ...]",
     "Get the values of all the given hash fields",
     5,
-    "1.3.10" },
+    "2.0.0" },
     { "HMSET",
     "key field value [field value ...]",
     "Set multiple hash fields to multiple values",
     5,
-    "1.3.8" },
+    "2.0.0" },
     { "HSET",
     "key field value",
     "Set the string value of a hash field",
     5,
-    "1.3.10" },
+    "2.0.0" },
     { "HSETNX",
     "key field value",
     "Set the value of a hash field, only if the field does not exist",
     5,
-    "1.3.8" },
+    "2.0.0" },
     { "HVALS",
     "key",
     "Get all the values in a hash",
     5,
-    "1.3.10" },
+    "2.0.0" },
     { "INCR",
     "key",
     "Increment the integer value of a key by one",
     1,
-    "0.07" },
+    "1.0.0" },
     { "INCRBY",
     "key increment",
-    "Increment the integer value of a key by the given number",
+    "Increment the integer value of a key by the given amount",
     1,
-    "0.07" },
+    "1.0.0" },
+    { "INCRBYFLOAT",
+    "key increment",
+    "Increment the float value of a key by the given amount",
+    1,
+    "2.6.0" },
     { "INFO",
     "-",
     "Get information and statistics about the server",
     9,
-    "0.07" },
+    "1.0.0" },
     { "KEYS",
     "pattern",
     "Find all keys matching the given pattern",
     0,
-    "0.07" },
+    "1.0.0" },
     { "LASTSAVE",
     "-",
     "Get the UNIX time stamp of the last successful save to disk",
     9,
-    "0.07" },
+    "1.0.0" },
     { "LINDEX",
     "key index",
     "Get an element from a list by its index",
     2,
-    "0.07" },
+    "1.0.0" },
     { "LINSERT",
     "key BEFORE|AFTER pivot value",
     "Insert an element before or after another element in a list",
     2,
-    "2.1.1" },
+    "2.2.0" },
     { "LLEN",
     "key",
     "Get the length of a list",
     2,
-    "0.07" },
+    "1.0.0" },
     { "LPOP",
     "key",
     "Remove and get the first element in a list",
     2,
-    "0.07" },
+    "1.0.0" },
     { "LPUSH",
-    "key value",
-    "Prepend a value to a list",
+    "key value [value ...]",
+    "Prepend one or multiple values to a list",
     2,
-    "0.07" },
+    "1.0.0" },
     { "LPUSHX",
     "key value",
     "Prepend a value to a list, only if the list exists",
     2,
-    "2.1.1" },
+    "2.2.0" },
     { "LRANGE",
     "key start stop",
     "Get a range of elements from a list",
     2,
-    "0.07" },
+    "1.0.0" },
     { "LREM",
     "key count value",
     "Remove elements from a list",
     2,
-    "0.07" },
+    "1.0.0" },
     { "LSET",
     "key index value",
     "Set the value of an element in a list by its index",
     2,
-    "0.07" },
+    "1.0.0" },
     { "LTRIM",
     "key start stop",
     "Trim a list to the specified range",
     2,
-    "0.07" },
+    "1.0.0" },
     { "MGET",
     "key [key ...]",
     "Get the values of all the given keys",
     1,
-    "0.07" },
+    "1.0.0" },
+    { "MIGRATE",
+    "host port key destination-db timeout",
+    "Atomically transfer a key from a Redis instance to another one.",
+    0,
+    "2.6.0" },
     { "MONITOR",
     "-",
     "Listen for all requests received by the server in real time",
     9,
-    "0.07" },
+    "1.0.0" },
     { "MOVE",
     "key db",
     "Move a key to another database",
     0,
-    "0.07" },
+    "1.0.0" },
     { "MSET",
     "key value [key value ...]",
     "Set multiple keys to multiple values",
     1,
-    "1.001" },
+    "1.0.1" },
     { "MSETNX",
     "key value [key value ...]",
     "Set multiple keys to multiple values, only if none of the keys exist",
     1,
-    "1.001" },
+    "1.0.1" },
     { "MULTI",
     "-",
     "Mark the start of a transaction block",
     7,
-    "1.1.95" },
+    "1.2.0" },
+    { "OBJECT",
+    "subcommand [arguments [arguments ...]]",
+    "Inspect the internals of Redis objects",
+    0,
+    "2.2.3" },
     { "PERSIST",
     "key",
     "Remove the expiration from a key",
     0,
-    "2.1.2" },
+    "2.2.0" },
+    { "PEXPIRE",
+    "key milliseconds",
+    "Set a key's time to live in milliseconds",
+    0,
+    "2.6.0" },
+    { "PEXPIREAT",
+    "key milliseconds-timestamp",
+    "Set the expiration for a key as a UNIX timestamp specified in milliseconds",
+    0,
+    "2.6.0" },
     { "PING",
     "-",
     "Ping the server",
     8,
-    "0.07" },
+    "1.0.0" },
+    { "PSETEX",
+    "key milliseconds value",
+    "Set the value and expiration in milliseconds of a key",
+    1,
+    "2.6.0" },
     { "PSUBSCRIBE",
-    "pattern",
+    "pattern [pattern ...]",
     "Listen for messages published to channels matching the given patterns",
     6,
-    "1.3.8" },
+    "2.0.0" },
+    { "PTTL",
+    "key",
+    "Get the time to live for a key in milliseconds",
+    0,
+    "2.6.0" },
     { "PUBLISH",
     "channel message",
     "Post a message to a channel",
     6,
-    "1.3.8" },
+    "2.0.0" },
     { "PUNSUBSCRIBE",
     "[pattern [pattern ...]]",
     "Stop listening for messages posted to channels matching the given patterns",
     6,
-    "1.3.8" },
+    "2.0.0" },
     { "QUIT",
     "-",
     "Close the connection",
     8,
-    "0.07" },
+    "1.0.0" },
     { "RANDOMKEY",
     "-",
     "Return a random key from the keyspace",
     0,
-    "0.07" },
+    "1.0.0" },
     { "RENAME",
     "key newkey",
     "Rename a key",
     0,
-    "0.07" },
+    "1.0.0" },
     { "RENAMENX",
     "key newkey",
     "Rename a key, only if the new key does not exist",
     0,
-    "0.07" },
+    "1.0.0" },
+    { "RESTORE",
+    "key ttl serialized-value",
+    "Create a key using the provided serialized value, previously obtained using DUMP.",
+    0,
+    "2.6.0" },
     { "RPOP",
     "key",
     "Remove and get the last element in a list",
     2,
-    "0.07" },
+    "1.0.0" },
     { "RPOPLPUSH",
     "source destination",
     "Remove the last element in a list, append it to another list and return it",
     2,
-    "1.1" },
+    "1.2.0" },
     { "RPUSH",
-    "key value",
-    "Append a value to a list",
+    "key value [value ...]",
+    "Append one or multiple values to a list",
     2,
-    "0.07" },
+    "1.0.0" },
     { "RPUSHX",
     "key value",
     "Append a value to a list, only if the list exists",
     2,
-    "2.1.1" },
+    "2.2.0" },
     { "SADD",
-    "key member",
-    "Add a member to a set",
+    "key member [member ...]",
+    "Add one or more members to a set",
     3,
-    "0.07" },
+    "1.0.0" },
     { "SAVE",
     "-",
     "Synchronously save the dataset to disk",
     9,
-    "0.07" },
+    "1.0.0" },
     { "SCARD",
     "key",
     "Get the number of members in a set",
     3,
-    "0.07" },
+    "1.0.0" },
+    { "SCRIPT EXISTS",
+    "script [script ...]",
+    "Check existence of scripts in the script cache.",
+    10,
+    "2.6.0" },
+    { "SCRIPT FLUSH",
+    "-",
+    "Remove all the scripts from the script cache.",
+    10,
+    "2.6.0" },
+    { "SCRIPT KILL",
+    "-",
+    "Kill the script currently in execution.",
+    10,
+    "2.6.0" },
+    { "SCRIPT LOAD",
+    "script",
+    "Load the specified Lua script into the script cache.",
+    10,
+    "2.6.0" },
     { "SDIFF",
     "key [key ...]",
     "Subtract multiple sets",
     3,
-    "0.100" },
+    "1.0.0" },
     { "SDIFFSTORE",
     "destination key [key ...]",
     "Subtract multiple sets and store the resulting set in a key",
     3,
-    "0.100" },
+    "1.0.0" },
     { "SELECT",
     "index",
     "Change the selected database for the current connection",
     8,
-    "0.07" },
+    "1.0.0" },
     { "SET",
     "key value",
     "Set the string value of a key",
     1,
-    "0.07" },
+    "1.0.0" },
     { "SETBIT",
     "key offset value",
     "Sets or clears the bit at offset in the string value stored at key",
     1,
-    "2.1.8" },
+    "2.2.0" },
     { "SETEX",
     "key seconds value",
     "Set the value and expiration of a key",
     1,
-    "1.3.10" },
+    "2.0.0" },
     { "SETNX",
     "key value",
     "Set the value of a key, only if the key does not exist",
     1,
-    "0.07" },
+    "1.0.0" },
     { "SETRANGE",
     "key offset value",
     "Overwrite part of a string at key starting at the specified offset",
     1,
-    "2.1.8" },
+    "2.2.0" },
     { "SHUTDOWN",
-    "-",
+    "[NOSAVE] [SAVE]",
     "Synchronously save the dataset to disk and then shut down the server",
     9,
-    "0.07" },
+    "1.0.0" },
     { "SINTER",
     "key [key ...]",
     "Intersect multiple sets",
     3,
-    "0.07" },
+    "1.0.0" },
     { "SINTERSTORE",
     "destination key [key ...]",
     "Intersect multiple sets and store the resulting set in a key",
     3,
-    "0.07" },
+    "1.0.0" },
     { "SISMEMBER",
     "key member",
     "Determine if a given value is a member of a set",
     3,
-    "0.07" },
+    "1.0.0" },
     { "SLAVEOF",
     "host port",
     "Make the server a slave of another instance, or promote it as master",
     9,
-    "0.100" },
+    "1.0.0" },
+    { "SLOWLOG",
+    "subcommand [argument]",
+    "Manages the Redis slow queries log",
+    9,
+    "2.2.12" },
     { "SMEMBERS",
     "key",
     "Get all the members in a set",
     3,
-    "0.07" },
+    "1.0.0" },
     { "SMOVE",
     "source destination member",
     "Move a member from one set to another",
     3,
-    "0.091" },
+    "1.0.0" },
     { "SORT",
     "key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination]",
     "Sort the elements in a list, set or sorted set",
     0,
-    "0.07" },
+    "1.0.0" },
     { "SPOP",
     "key",
     "Remove and return a random member from a set",
     3,
-    "0.101" },
+    "1.0.0" },
     { "SRANDMEMBER",
     "key",
     "Get a random member from a set",
     3,
-    "1.001" },
+    "1.0.0" },
     { "SREM",
-    "key member",
-    "Remove a member from a set",
+    "key member [member ...]",
+    "Remove one or more members from a set",
     3,
-    "0.07" },
+    "1.0.0" },
     { "STRLEN",
     "key",
     "Get the length of the value stored in a key",
     1,
-    "2.1.2" },
+    "2.2.0" },
     { "SUBSCRIBE",
-    "channel",
+    "channel [channel ...]",
     "Listen for messages published to the given channels",
     6,
-    "1.3.8" },
-    { "SUBSTR",
-    "key start end",
-    "Get a substring of the string stored at a key",
-    1,
-    "1.3.4" },
+    "2.0.0" },
     { "SUNION",
     "key [key ...]",
     "Add multiple sets",
     3,
-    "0.091" },
+    "1.0.0" },
     { "SUNIONSTORE",
     "destination key [key ...]",
     "Add multiple sets and store the resulting set in a key",
     3,
-    "0.091" },
+    "1.0.0" },
     { "SYNC",
     "-",
     "Internal command used for replication",
     9,
-    "0.07" },
+    "1.0.0" },
+    { "TIME",
+    "-",
+    "Return the current server time",
+    9,
+    "2.6.0" },
     { "TTL",
     "key",
     "Get the time to live for a key",
     0,
-    "0.100" },
+    "1.0.0" },
     { "TYPE",
     "key",
     "Determine the type stored at key",
     0,
-    "0.07" },
+    "1.0.0" },
     { "UNSUBSCRIBE",
     "[channel [channel ...]]",
     "Stop listening for messages posted to the given channels",
     6,
-    "1.3.8" },
+    "2.0.0" },
     { "UNWATCH",
     "-",
     "Forget about all watched keys",
     7,
-    "2.1.0" },
+    "2.2.0" },
     { "WATCH",
     "key [key ...]",
     "Watch the given keys to determine execution of the MULTI/EXEC block",
     7,
-    "2.1.0" },
+    "2.2.0" },
     { "ZADD",
-    "key score member",
-    "Add a member to a sorted set, or update its score if it already exists",
+    "key score member [score] [member]",
+    "Add one or more members to a sorted set, or update its score if it already exists",
     4,
-    "1.1" },
+    "1.2.0" },
     { "ZCARD",
     "key",
     "Get the number of members in a sorted set",
     4,
-    "1.1" },
+    "1.2.0" },
     { "ZCOUNT",
     "key min max",
     "Count the members in a sorted set with scores within the given values",
     4,
-    "1.3.3" },
+    "2.0.0" },
     { "ZINCRBY",
     "key increment member",
     "Increment the score of a member in a sorted set",
     4,
-    "1.1" },
+    "1.2.0" },
     { "ZINTERSTORE",
     "destination numkeys key [key ...] [WEIGHTS weight] [AGGREGATE SUM|MIN|MAX]",
     "Intersect multiple sorted sets and store the resulting sorted set in a new key",
     4,
-    "1.3.10" },
+    "2.0.0" },
     { "ZRANGE",
     "key start stop [WITHSCORES]",
     "Return a range of members in a sorted set, by index",
     4,
-    "1.1" },
+    "1.2.0" },
     { "ZRANGEBYSCORE",
     "key min max [WITHSCORES] [LIMIT offset count]",
     "Return a range of members in a sorted set, by score",
     4,
-    "1.050" },
+    "1.0.5" },
     { "ZRANK",
     "key member",
     "Determine the index of a member in a sorted set",
     4,
-    "1.3.4" },
+    "2.0.0" },
     { "ZREM",
-    "key member",
-    "Remove a member from a sorted set",
+    "key member [member ...]",
+    "Remove one or more members from a sorted set",
     4,
-    "1.1" },
+    "1.2.0" },
     { "ZREMRANGEBYRANK",
     "key start stop",
     "Remove all members in a sorted set within the given indexes",
     4,
-    "1.3.4" },
+    "2.0.0" },
     { "ZREMRANGEBYSCORE",
     "key min max",
     "Remove all members in a sorted set within the given scores",
     4,
-    "1.1" },
+    "1.2.0" },
     { "ZREVRANGE",
     "key start stop [WITHSCORES]",
     "Return a range of members in a sorted set, by index, with scores ordered from high to low",
     4,
-    "1.1" },
+    "1.2.0" },
     { "ZREVRANGEBYSCORE",
     "key max min [WITHSCORES] [LIMIT offset count]",
     "Return a range of members in a sorted set, by score, with scores ordered from high to low",
     4,
-    "2.1.6" },
+    "2.2.0" },
     { "ZREVRANK",
     "key member",
     "Determine the index of a member in a sorted set, with scores ordered from high to low",
     4,
-    "1.3.4" },
+    "2.0.0" },
     { "ZSCORE",
     "key member",
     "Get the score associated with the given member in a sorted set",
     4,
-    "1.1" },
+    "1.2.0" },
     { "ZUNIONSTORE",
     "destination numkeys key [key ...] [WEIGHTS weight] [AGGREGATE SUM|MIN|MAX]",
     "Add multiple sorted sets and store the resulting sorted set in a new key",
     4,
-    "1.3.10" }
+    "2.0.0" }
 };
 
 #endif

src/memtest.c

@@ -0,0 +1,225 @@
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <assert.h>
+#include <limits.h>
+#include <errno.h>
+#include <termios.h>
+#include <sys/ioctl.h>
+
+#if (ULONG_MAX == 4294967295UL)
+#define MEMTEST_32BIT
+#elif (ULONG_MAX == 18446744073709551615ULL)
+#define MEMTEST_64BIT
+#else
+#error "ULONG_MAX value not supported."
+#endif
+
+#ifdef MEMTEST_32BIT
+#define ULONG_ONEZERO 0xaaaaaaaaUL
+#define ULONG_ZEROONE 0x55555555UL
+#else
+#define ULONG_ONEZERO 0xaaaaaaaaaaaaaaaaUL
+#define ULONG_ZEROONE 0x5555555555555555UL
+#endif
+
+static struct winsize ws;
+size_t progress_printed; /* Printed chars in screen-wide progress bar. */
+size_t progress_full; /* How many chars to write to fill the progress bar. */
+
+void memtest_progress_start(char *title, int pass) {
+    int j;
+
+    printf("\x1b[H\x1b[2J");    /* Cursor home, clear screen. */
+    /* Fill with dots. */
+    for (j = 0; j < ws.ws_col*(ws.ws_row-2); j++) printf(".");
+    printf("Please keep the test running several minutes per GB of memory.\n");
+    printf("Also check http://www.memtest86.com/ and http://pyropus.ca/software/memtester/");
+    printf("\x1b[H\x1b[2K");          /* Cursor home, clear current line.  */
+    printf("%s [%d]\n", title, pass); /* Print title. */
+    progress_printed = 0;
+    progress_full = ws.ws_col*(ws.ws_row-3);
+    fflush(stdout);
+}
+
+void memtest_progress_end(void) {
+    printf("\x1b[H\x1b[2J");    /* Cursor home, clear screen. */
+}
+
+void memtest_progress_step(size_t curr, size_t size, char c) {
+    size_t chars = ((unsigned long long)curr*progress_full)/size, j;
+
+    for (j = 0; j < chars-progress_printed; j++) {
+        printf("%c",c);
+        progress_printed++;
+    }
+    fflush(stdout);
+}
+
+/* Test that addressing is fine. Every location is populated with its own
+ * address, and finally verified. This test is very fast but may detect
+ * ASAP big issues with the memory subsystem. */
+void memtest_addressing(unsigned long *l, size_t bytes) {
+    unsigned long words = bytes/sizeof(unsigned long);
+    unsigned long j, *p;
+
+    /* Fill */
+    p = l;
+    for (j = 0; j < words; j++) {
+        *p = (unsigned long)p;
+        p++;
+        if ((j & 0xffff) == 0) memtest_progress_step(j,words*2,'A');
+    }
+    /* Test */
+    p = l;
+    for (j = 0; j < words; j++) {
+        if (*p != (unsigned long)p) {
+            printf("\n*** MEMORY ADDRESSING ERROR: %p contains %lu\n",
+                (void*) p, *p);
+            exit(1);
+        }
+        p++;
+        if ((j & 0xffff) == 0) memtest_progress_step(j+words,words*2,'A');
+    }
+}
+
+/* Fill words stepping a single page at every write, so we continue to
+ * touch all the pages in the smallest amount of time reducing the
+ * effectiveness of caches, and making it hard for the OS to transfer
+ * pages on the swap. */
+void memtest_fill_random(unsigned long *l, size_t bytes) {
+    unsigned long step = 4096/sizeof(unsigned long);
+    unsigned long words = bytes/sizeof(unsigned long)/2;
+    unsigned long iwords = words/step;  /* words per iteration */
+    unsigned long off, w, *l1, *l2;
+
+    assert((bytes & 4095) == 0);
+    for (off = 0; off < step; off++) {
+        l1 = l+off;
+        l2 = l1+words;
+        for (w = 0; w < iwords; w++) {
+#ifdef MEMTEST_32BIT
+            *l1 = *l2 = ((unsigned long)     (rand()&0xffff)) |
+                        (((unsigned long)    (rand()&0xffff)) << 16);
+#else
+            *l1 = *l2 = ((unsigned long)     (rand()&0xffff)) |
+                        (((unsigned long)    (rand()&0xffff)) << 16) |
+                        (((unsigned long)    (rand()&0xffff)) << 32) |
+                        (((unsigned long)    (rand()&0xffff)) << 48);
+#endif
+            l1 += step;
+            l2 += step;
+            if ((w & 0xffff) == 0)
+                memtest_progress_step(w+iwords*off,words,'R');
+        }
+    }
+}
+
+/* Like memtest_fill_random() but uses the two specified values to fill
+ * memory, in an alternated way (v1|v2|v1|v2|...) */
+void memtest_fill_value(unsigned long *l, size_t bytes, unsigned long v1,
+                        unsigned long v2, char sym)
+{
+    unsigned long step = 4096/sizeof(unsigned long);
+    unsigned long words = bytes/sizeof(unsigned long)/2;
+    unsigned long iwords = words/step;  /* words per iteration */
+    unsigned long off, w, *l1, *l2, v;
+
+    assert((bytes & 4095) == 0);
+    for (off = 0; off < step; off++) {
+        l1 = l+off;
+        l2 = l1+words;
+        v = (off & 1) ? v2 : v1;
+        for (w = 0; w < iwords; w++) {
+#ifdef MEMTEST_32BIT
+            *l1 = *l2 = ((unsigned long)     v) |
+                        (((unsigned long)    v) << 16);
+#else
+            *l1 = *l2 = ((unsigned long)     v) |
+                        (((unsigned long)    v) << 16) |
+                        (((unsigned long)    v) << 32) |
+                        (((unsigned long)    v) << 48);
+#endif
+            l1 += step;
+            l2 += step;
+            if ((w & 0xffff) == 0)
+                memtest_progress_step(w+iwords*off,words,sym);
+        }
+    }
+}
+
+void memtest_compare(unsigned long *l, size_t bytes) {
+    unsigned long words = bytes/sizeof(unsigned long)/2;
+    unsigned long w, *l1, *l2;
+
+    assert((bytes & 4095) == 0);
+    l1 = l;
+    l2 = l1+words;
+    for (w = 0; w < words; w++) {
+        if (*l1 != *l2) {
+            printf("\n*** MEMORY ERROR DETECTED: %p != %p (%lu vs %lu)\n",
+                (void*)l1, (void*)l2, *l1, *l2);
+            exit(1);
+        }
+        l1 ++;
+        l2 ++;
+        if ((w & 0xffff) == 0) memtest_progress_step(w,words,'=');
+    }
+}
+
+void memtest_compare_times(unsigned long *m, size_t bytes, int pass, int times) {
+    int j;
+
+    for (j = 0; j < times; j++) {
+        memtest_progress_start("Compare",pass);
+        memtest_compare(m,bytes);
+        memtest_progress_end();
+    }
+}
+
+void memtest_test(size_t megabytes, int passes) {
+    size_t bytes = megabytes*1024*1024;
+    unsigned long *m = malloc(bytes);
+    int pass = 0;
+
+    if (m == NULL) {
+        fprintf(stderr,"Unable to allocate %zu megabytes: %s",
+            megabytes, strerror(errno));
+        exit(1);
+    }
+    while (pass != passes) {
+        pass++;
+
+        memtest_progress_start("Addressing test",pass);
+        memtest_addressing(m,bytes);
+        memtest_progress_end();
+
+        memtest_progress_start("Random fill",pass);
+        memtest_fill_random(m,bytes);
+        memtest_progress_end();
+        memtest_compare_times(m,bytes,pass,4);
+
+        memtest_progress_start("Solid fill",pass);
+        memtest_fill_value(m,bytes,0,(unsigned long)-1,'S');
+        memtest_progress_end();
+        memtest_compare_times(m,bytes,pass,4);
+
+        memtest_progress_start("Checkerboard fill",pass);
+        memtest_fill_value(m,bytes,ULONG_ONEZERO,ULONG_ZEROONE,'C');
+        memtest_progress_end();
+        memtest_compare_times(m,bytes,pass,4);
+    }
+}
+
+void memtest(size_t megabytes, int passes) {
+    if (ioctl(1, TIOCGWINSZ, &ws) == -1) {
+        ws.ws_col = 80;
+        ws.ws_row = 20;
+    }
+    memtest_test(megabytes,passes);
+    printf("\nYour memory passed this test.\n");
+    printf("Please if you are still in doubt use the following two tools:\n");
+    printf("1) memtest86: http://www.memtest86.com/\n");
+    printf("2) memtester: http://pyropus.ca/software/memtester/\n");
+    exit(0);
+}

src/migrate.c

@@ -0,0 +1,246 @@
+#include "redis.h"
+#include "endianconv.h"
+
+/* -----------------------------------------------------------------------------
+ * DUMP, RESTORE and MIGRATE commands
+ * -------------------------------------------------------------------------- */
+
+/* Generates a DUMP-format representation of the object 'o', adding it to the
+ * io stream pointed by 'rio'. This function can't fail. */
+void createDumpPayload(rio *payload, robj *o) {
+    unsigned char buf[2];
+    uint64_t crc;
+
+    /* Serialize the object in a RDB-like format. It consist of an object type
+     * byte followed by the serialized object. This is understood by RESTORE. */
+    rioInitWithBuffer(payload,sdsempty());
+    redisAssert(rdbSaveObjectType(payload,o));
+    redisAssert(rdbSaveObject(payload,o));
+
+    /* Write the footer, this is how it looks like:
+     * ----------------+---------------------+---------------+
+     * ... RDB payload | 2 bytes RDB version | 8 bytes CRC64 |
+     * ----------------+---------------------+---------------+
+     * RDB version and CRC are both in little endian.
+     */
+
+    /* RDB version */
+    buf[0] = REDIS_RDB_VERSION & 0xff;
+    buf[1] = (REDIS_RDB_VERSION >> 8) & 0xff;
+    payload->io.buffer.ptr = sdscatlen(payload->io.buffer.ptr,buf,2);
+
+    /* CRC64 */
+    crc = crc64(0,(unsigned char*)payload->io.buffer.ptr,
+                sdslen(payload->io.buffer.ptr));
+    memrev64ifbe(&crc);
+    payload->io.buffer.ptr = sdscatlen(payload->io.buffer.ptr,&crc,8);
+}
+
+/* Verify that the RDB version of the dump payload matches the one of this Redis
+ * instance and that the checksum is ok.
+ * If the DUMP payload looks valid REDIS_OK is returned, otherwise REDIS_ERR
+ * is returned. */
+int verifyDumpPayload(unsigned char *p, size_t len) {
+    unsigned char *footer;
+    uint16_t rdbver;
+    uint64_t crc;
+
+    /* At least 2 bytes of RDB version and 8 of CRC64 should be present. */
+    if (len < 10) return REDIS_ERR;
+    footer = p+(len-10);
+
+    /* Verify RDB version */
+    rdbver = (footer[1] << 8) | footer[0];
+    if (rdbver != REDIS_RDB_VERSION) return REDIS_ERR;
+
+    /* Verify CRC64 */
+    crc = crc64(0,p,len-8);
+    memrev64ifbe(&crc);
+    return (memcmp(&crc,footer+2,8) == 0) ? REDIS_OK : REDIS_ERR;
+}
+
+/* DUMP keyname
+ * DUMP is actually not used by Redis Cluster but it is the obvious
+ * complement of RESTORE and can be useful for different applications. */
+void dumpCommand(redisClient *c) {
+    robj *o, *dumpobj;
+    rio payload;
+
+    /* Check if the key is here. */
+    if ((o = lookupKeyRead(c->db,c->argv[1])) == NULL) {
+        addReply(c,shared.nullbulk);
+        return;
+    }
+
+    /* Create the DUMP encoded representation. */
+    createDumpPayload(&payload,o);
+
+    /* Transfer to the client */
+    dumpobj = createObject(REDIS_STRING,payload.io.buffer.ptr);
+    addReplyBulk(c,dumpobj);
+    decrRefCount(dumpobj);
+    return;
+}
+
+/* RESTORE key ttl serialized-value */
+void restoreCommand(redisClient *c) {
+    long ttl;
+    rio payload;
+    int type;
+    robj *obj;
+
+    /* Make sure this key does not already exist here... */
+    if (lookupKeyWrite(c->db,c->argv[1]) != NULL) {
+        addReplyError(c,"Target key name is busy.");
+        return;
+    }
+
+    /* Check if the TTL value makes sense */
+    if (getLongFromObjectOrReply(c,c->argv[2],&ttl,NULL) != REDIS_OK) {
+        return;
+    } else if (ttl < 0) {
+        addReplyError(c,"Invalid TTL value, must be >= 0");
+        return;
+    }
+
+    /* Verify RDB version and data checksum. */
+    if (verifyDumpPayload(c->argv[3]->ptr,sdslen(c->argv[3]->ptr)) == REDIS_ERR) {
+        addReplyError(c,"DUMP payload version or checksum are wrong");
+        return;
+    }
+
+    rioInitWithBuffer(&payload,c->argv[3]->ptr);
+    if (((type = rdbLoadObjectType(&payload)) == -1) ||
+        ((obj = rdbLoadObject(type,&payload)) == NULL))
+    {
+        addReplyError(c,"Bad data format");
+        return;
+    }
+
+    /* Create the key and set the TTL if any */
+    dbAdd(c->db,c->argv[1],obj);
+    if (ttl) setExpire(c->db,c->argv[1],mstime()+ttl);
+    signalModifiedKey(c->db,c->argv[1]);
+    addReply(c,shared.ok);
+    server.dirty++;
+}
+
+/* MIGRATE host port key dbid timeout */
+void migrateCommand(redisClient *c) {
+    int fd;
+    long timeout;
+    long dbid;
+    long long ttl = 0, expireat;
+    robj *o;
+    rio cmd, payload;
+
+    /* Sanity check */
+    if (getLongFromObjectOrReply(c,c->argv[5],&timeout,NULL) != REDIS_OK)
+        return;
+    if (getLongFromObjectOrReply(c,c->argv[4],&dbid,NULL) != REDIS_OK)
+        return;
+    if (timeout <= 0) timeout = 1;
+
+    /* Check if the key is here. If not we reply with success as there is
+     * nothing to migrate (for instance the key expired in the meantime), but
+     * we include such information in the reply string. */
+    if ((o = lookupKeyRead(c->db,c->argv[3])) == NULL) {
+        addReplySds(c,sdsnew("+NOKEY\r\n"));
+        return;
+    }
+    
+    /* Connect */
+    fd = anetTcpNonBlockConnect(server.neterr,c->argv[1]->ptr,
+                atoi(c->argv[2]->ptr));
+    if (fd == -1) {
+        addReplyErrorFormat(c,"Can't connect to target node: %s",
+            server.neterr);
+        return;
+    }
+    if ((aeWait(fd,AE_WRITABLE,timeout*1000) & AE_WRITABLE) == 0) {
+        addReplySds(c,sdsnew("-IOERR error or timeout connecting to the client\r\n"));
+        return;
+    }
+
+    /* Create RESTORE payload and generate the protocol to call the command. */
+    rioInitWithBuffer(&cmd,sdsempty());
+    redisAssertWithInfo(c,NULL,rioWriteBulkCount(&cmd,'*',2));
+    redisAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,"SELECT",6));
+    redisAssertWithInfo(c,NULL,rioWriteBulkLongLong(&cmd,dbid));
+
+    expireat = getExpire(c->db,c->argv[3]);
+    if (expireat != -1) {
+        ttl = expireat-mstime();
+        if (ttl < 1) ttl = 1;
+    }
+    redisAssertWithInfo(c,NULL,rioWriteBulkCount(&cmd,'*',4));
+    redisAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,"RESTORE",7));
+    redisAssertWithInfo(c,NULL,c->argv[3]->encoding == REDIS_ENCODING_RAW);
+    redisAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,c->argv[3]->ptr,sdslen(c->argv[3]->ptr)));
+    redisAssertWithInfo(c,NULL,rioWriteBulkLongLong(&cmd,ttl));
+
+    /* Finally the last argument that is the serailized object payload
+     * in the DUMP format. */
+    createDumpPayload(&payload,o);
+    redisAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,payload.io.buffer.ptr,
+                                sdslen(payload.io.buffer.ptr)));
+    sdsfree(payload.io.buffer.ptr);
+
+    /* Tranfer the query to the other node in 64K chunks. */
+    {
+        sds buf = cmd.io.buffer.ptr;
+        size_t pos = 0, towrite;
+        int nwritten = 0;
+
+        while ((towrite = sdslen(buf)-pos) > 0) {
+            towrite = (towrite > (64*1024) ? (64*1024) : towrite);
+            nwritten = syncWrite(fd,buf+pos,towrite,timeout);
+            if (nwritten != (signed)towrite) goto socket_wr_err;
+            pos += nwritten;
+        }
+    }
+
+    /* Read back the reply. */
+    {
+        char buf1[1024];
+        char buf2[1024];
+
+        /* Read the two replies */
+        if (syncReadLine(fd, buf1, sizeof(buf1), timeout) <= 0)
+            goto socket_rd_err;
+        if (syncReadLine(fd, buf2, sizeof(buf2), timeout) <= 0)
+            goto socket_rd_err;
+        if (buf1[0] == '-' || buf2[0] == '-') {
+            addReplyErrorFormat(c,"Target instance replied with error: %s",
+                (buf1[0] == '-') ? buf1+1 : buf2+1);
+        } else {
+            robj *aux;
+
+            dbDelete(c->db,c->argv[3]);
+            signalModifiedKey(c->db,c->argv[3]);
+            addReply(c,shared.ok);
+            server.dirty++;
+
+            /* Translate MIGRATE as DEL for replication/AOF. */
+            aux = createStringObject("DEL",3);
+            rewriteClientCommandVector(c,2,aux,c->argv[3]);
+            decrRefCount(aux);
+        }
+    }
+
+    sdsfree(cmd.io.buffer.ptr);
+    close(fd);
+    return;
+
+socket_wr_err:
+    addReplySds(c,sdsnew("-IOERR error or timeout writing to target instance\r\n"));
+    sdsfree(cmd.io.buffer.ptr);
+    close(fd);
+    return;
+
+socket_rd_err:
+    addReplySds(c,sdsnew("-IOERR error or timeout reading from target node\r\n"));
+    sdsfree(cmd.io.buffer.ptr);
+    close(fd);
+    return;
+}

src/multi.c

@@ -40,6 +40,13 @@ void queueMultiCommand(redisClient *c) {
     c->mstate.count++;
 }
 
+void discardTransaction(redisClient *c) {
+    freeClientMultiState(c);
+    initClientMultiState(c);
+    c->flags &= ~(REDIS_MULTI|REDIS_DIRTY_CAS);;
+    unwatchAllKeys(c);
+}
+
 void multiCommand(redisClient *c) {
     if (c->flags & REDIS_MULTI) {
         addReplyError(c,"MULTI calls can not be nested");
@@ -54,11 +61,7 @@ void discardCommand(redisClient *c) {
         addReplyError(c,"DISCARD without MULTI");
         return;
     }
-
-    freeClientMultiState(c);
-    initClientMultiState(c);
-    c->flags &= ~(REDIS_MULTI|REDIS_DIRTY_CAS);;
-    unwatchAllKeys(c);
+    discardTransaction(c);
     addReply(c,shared.ok);
 }
 

src/networking.c

@@ -22,7 +22,6 @@ int listMatchObjects(void *a, void *b) {
 
 redisClient *createClient(int fd) {
     redisClient *c = zmalloc(sizeof(redisClient));
-    c->bufpos = 0;
 
     /* passing -1 as fd it is possible to create a non connected client.
      * This is useful since all the Redis commands needs to be executed
@@ -42,7 +41,9 @@ redisClient *createClient(int fd) {
 
     selectDb(c,0);
     c->fd = fd;
+    c->bufpos = 0;
     c->querybuf = sdsempty();
+    c->querybuf_peak = 0;
     c->reqtype = 0;
     c->argc = 0;
     c->argv = NULL;
@@ -51,7 +52,7 @@ redisClient *createClient(int fd) {
     c->bulklen = -1;
     c->sentlen = 0;
     c->flags = 0;
-    c->lastinteraction = time(NULL);
+    c->ctime = c->lastinteraction = server.unixtime;
     c->authenticated = 0;
     c->replstate = REDIS_REPL_NONE;
     c->reply = listCreate();
@@ -546,6 +547,16 @@ static void freeClientArgv(redisClient *c) {
     c->cmd = NULL;
 }
 
+/* Close all the slaves connections. This is useful in chained replication
+ * when we resync with our own master and want to force all our slaves to
+ * resync with us as well. */
+void disconnectSlaves(void) {
+    while (listLength(server.slaves)) {
+        listNode *ln = listFirst(server.slaves);
+        freeClient((redisClient*)ln->value);
+    }
+}
+
 void freeClient(redisClient *c) {
     listNode *ln;
 
@@ -603,22 +614,13 @@ void freeClient(redisClient *c) {
     if (c->flags & REDIS_MASTER) {
         server.master = NULL;
         server.repl_state = REDIS_REPL_CONNECT;
-        server.repl_down_since = time(NULL);
-        /* Since we lost the connection with the master, we should also
-         * close the connection with all our slaves if we have any, so
-         * when we'll resync with the master the other slaves will sync again
-         * with us as well. Note that also when the slave is not connected
-         * to the master it will keep refusing connections by other slaves.
+        server.repl_down_since = server.unixtime;
+        /* We lost connection with our master, force our slaves to resync
+         * with us as well to load the new data set.
          *
-         * We do this only if server.masterhost != NULL. If it is NULL this
-         * means the user called SLAVEOF NO ONE and we are freeing our
-         * link with the master, so no need to close link with slaves. */
-        if (server.masterhost != NULL) {
-            while (listLength(server.slaves)) {
-                ln = listFirst(server.slaves);
-                freeClient((redisClient*)ln->value);
-            }
-        }
+         * If server.masterhost is NULL the user called SLAVEOF NO ONE so
+         * slave resync is not needed. */
+        if (server.masterhost != NULL) disconnectSlaves();
     }
 
     /* If this client was scheduled for async freeing we need to remove it
@@ -731,7 +733,7 @@ void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask) {
             return;
         }
     }
-    if (totwritten > 0) c->lastinteraction = time(NULL);
+    if (totwritten > 0) c->lastinteraction = server.unixtime;
     if (c->bufpos == 0 && listLength(c->reply) == 0) {
         c->sentlen = 0;
         aeDeleteFileEvent(server.el,c->fd,AE_WRITABLE);
@@ -751,34 +753,6 @@ void resetClient(redisClient *c) {
     if (!(c->flags & REDIS_MULTI)) c->flags &= (~REDIS_ASKING);
 }
 
-void closeTimedoutClients(void) {
-    redisClient *c;
-    listNode *ln;
-    time_t now = time(NULL);
-    listIter li;
-
-    listRewind(server.clients,&li);
-    while ((ln = listNext(&li)) != NULL) {
-        c = listNodeValue(ln);
-        if (server.maxidletime &&
-            !(c->flags & REDIS_SLAVE) &&    /* no timeout for slaves */
-            !(c->flags & REDIS_MASTER) &&   /* no timeout for masters */
-            !(c->flags & REDIS_BLOCKED) &&  /* no timeout for BLPOP */
-            dictSize(c->pubsub_channels) == 0 && /* no timeout for pubsub */
-            listLength(c->pubsub_patterns) == 0 &&
-            (now - c->lastinteraction > server.maxidletime))
-        {
-            redisLog(REDIS_VERBOSE,"Closing idle client");
-            freeClient(c);
-        } else if (c->flags & REDIS_BLOCKED) {
-            if (c->bpop.timeout != 0 && c->bpop.timeout < now) {
-                addReply(c,shared.nullmultibulk);
-                unblockClientWaitingData(c);
-            }
-        }
-    }
-}
-
 int processInlineBuffer(redisClient *c) {
     char *newline = strstr(c->querybuf,"\r\n");
     int argc, j;
@@ -1026,6 +1000,7 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
     }
 
     qblen = sdslen(c->querybuf);
+    if (c->querybuf_peak < qblen) c->querybuf_peak = qblen;
     c->querybuf = sdsMakeRoomFor(c->querybuf, readlen);
     nread = read(fd, c->querybuf+qblen, readlen);
     if (nread == -1) {
@@ -1043,7 +1018,7 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
     }
     if (nread) {
         sdsIncrLen(c->querybuf,nread);
-        c->lastinteraction = time(NULL);
+        c->lastinteraction = server.unixtime;
     } else {
         server.current_client = NULL;
         return;
@@ -1084,7 +1059,6 @@ void getClientsMaxBuffers(unsigned long *longest_output_list,
 sds getClientInfoString(redisClient *client) {
     char ip[32], flags[16], events[3], *p;
     int port;
-    time_t now = time(NULL);
     int emask;
 
     anetPeerToString(client->fd,ip,&port);
@@ -1111,14 +1085,17 @@ sds getClientInfoString(redisClient *client) {
     if (emask & AE_WRITABLE) *p++ = 'w';
     *p = '\0';
     return sdscatprintf(sdsempty(),
-        "addr=%s:%d fd=%d idle=%ld flags=%s db=%d sub=%d psub=%d qbuf=%lu obl=%lu oll=%lu omem=%lu events=%s cmd=%s",
+        "addr=%s:%d fd=%d age=%ld idle=%ld flags=%s db=%d sub=%d psub=%d multi=%d qbuf=%lu qbuf-free=%lu obl=%lu oll=%lu omem=%lu events=%s cmd=%s",
         ip,port,client->fd,
-        (long)(now - client->lastinteraction),
+        (long)(server.unixtime - client->ctime),
+        (long)(server.unixtime - client->lastinteraction),
         flags,
         client->db->id,
         (int) dictSize(client->pubsub_channels),
         (int) listLength(client->pubsub_patterns),
+        (client->flags & REDIS_MULTI) ? client->mstate.count : -1,
         (unsigned long) sdslen(client->querybuf),
+        (unsigned long) sdsavail(client->querybuf),
         (unsigned long) client->bufpos,
         (unsigned long) listLength(client->reply),
         getClientOutputBufferMemoryUsage(client),

src/object.c

@@ -9,18 +9,8 @@ robj *createObject(int type, void *ptr) {
     o->ptr = ptr;
     o->refcount = 1;
 
-    /* Set the LRU to the current lruclock (minutes resolution).
-     * We do this regardless of the fact VM is active as LRU is also
-     * used for the maxmemory directive when Redis is used as cache.
-     *
-     * Note that this code may run in the context of an I/O thread
-     * and accessing server.lruclock in theory is an error
-     * (no locks). But in practice this is safe, and even if we read
-     * garbage Redis will not fail. */
+    /* Set the LRU to the current lruclock (minutes resolution). */
     o->lru = server.lruclock;
-    /* The following is only needed if VM is active, but since the conditional
-     * is probably more costly than initializing the field it's better to
-     * have every field properly initialized anyway. */
     return o;
 }
 
@@ -56,7 +46,16 @@ robj *createStringObjectFromLongDouble(long double value) {
      * that is "non surprising" for the user (that is, most small decimal
      * numbers will be represented in a way that when converted back into
      * a string are exactly the same as what the user typed.) */
-    len = snprintf(buf,sizeof(buf),"%.17Lg", value);
+    len = snprintf(buf,sizeof(buf),"%.17Lf", value);
+    /* Now remove trailing zeroes after the '.' */
+    if (strchr(buf,'.') != NULL) {
+        char *p = buf+len-1;
+        while(*p == '0') {
+            p--;
+            len--;
+        }
+        if (*p == '.') len--;
+    }
     return createStringObject(buf,len);
 }
 
@@ -261,9 +260,7 @@ robj *tryObjectEncoding(robj *o) {
 
     /* Ok, this object can be encoded...
      *
-     * Can I use a shared object? Only if the object is inside a given
-     * range and if the back end in use is in-memory. For disk store every
-     * object in memory used as value should be independent.
+     * Can I use a shared object? Only if the object is inside a given range
      *
      * Note that we also avoid using shared integers when maxmemory is used
      * because every object needs to have a private LRU field for the LRU

src/pubsub.c

@@ -263,6 +263,5 @@ void punsubscribeCommand(redisClient *c) {
 
 void publishCommand(redisClient *c) {
     int receivers = pubsubPublishMessage(c->argv[1],c->argv[2]);
-    if (server.cluster_enabled) clusterPropagatePublish(c->argv[1],c->argv[2]);
     addReplyLongLong(c,receivers);
 }

src/rdb.c

@@ -1,6 +1,7 @@
 #include "redis.h"
 #include "lzf.h"    /* LZF compression library */
 #include "zipmap.h"
+#include "endianconv.h"
 
 #include <math.h>
 #include <sys/types.h>
@@ -26,11 +27,6 @@ int rdbLoadType(rio *rdb) {
     return type;
 }
 
-int rdbSaveTime(rio *rdb, time_t t) {
-    int32_t t32 = (int32_t) t;
-    return rdbWriteRaw(rdb,&t32,4);
-}
-
 time_t rdbLoadTime(rio *rdb) {
     int32_t t32;
     if (rioRead(rdb,&t32,4) == 0) return -1;
@@ -602,10 +598,12 @@ int rdbSave(char *filename) {
     dictIterator *di = NULL;
     dictEntry *de;
     char tmpfile[256];
+    char magic[10];
     int j;
     long long now = mstime();
     FILE *fp;
     rio rdb;
+    uint64_t cksum;
 
     snprintf(tmpfile,256,"temp-%d.rdb", (int) getpid());
     fp = fopen(tmpfile,"w");
@@ -616,7 +614,10 @@ int rdbSave(char *filename) {
     }
 
     rioInitWithFile(&rdb,fp);
-    if (rdbWriteRaw(&rdb,"REDIS0003",9) == -1) goto werr;
+    if (server.rdb_checksum)
+        rdb.update_cksum = rioGenericUpdateChecksum;
+    snprintf(magic,sizeof(magic),"REDIS%04d",REDIS_RDB_VERSION);
+    if (rdbWriteRaw(&rdb,magic,9) == -1) goto werr;
 
     for (j = 0; j < server.dbnum; j++) {
         redisDb *db = server.db+j;
@@ -644,9 +645,17 @@ int rdbSave(char *filename) {
         }
         dictReleaseIterator(di);
     }
+    di = NULL; /* So that we don't release it again on error. */
+
     /* EOF opcode */
     if (rdbSaveType(&rdb,REDIS_RDB_OPCODE_EOF) == -1) goto werr;
 
+    /* CRC64 checksum. It will be zero if checksum computation is disabled, the
+     * loading code skips the check in this case. */
+    cksum = rdb.cksum;
+    memrev64ifbe(&cksum);
+    rioWrite(&rdb,&cksum,8);
+
     /* Make sure data will not remain on the OS's output buffers */
     fflush(fp);
     fsync(fileno(fp));
@@ -689,7 +698,7 @@ int rdbSaveBackground(char *filename) {
         if (server.ipfd > 0) close(server.ipfd);
         if (server.sofd > 0) close(server.sofd);
         retval = rdbSave(filename);
-        _exit((retval == REDIS_OK) ? 0 : 1);
+        exitFromChild((retval == REDIS_OK) ? 0 : 1);
     } else {
         /* Parent */
         server.stat_fork_time = ustime()-start;
@@ -720,7 +729,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
     size_t len;
     unsigned int i;
 
-    redisLog(REDIS_DEBUG,"LOADING OBJECT %d (at %d)\n",rdbtype,rdb->tell(rdb));
+    redisLog(REDIS_DEBUG,"LOADING OBJECT %d (at %d)\n",rdbtype,rioTell(rdb));
     if (rdbtype == REDIS_RDB_TYPE_STRING) {
         /* Read string value */
         if ((o = rdbLoadEncodedStringObject(rdb)) == NULL) return NULL;
@@ -844,9 +853,10 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
             hashTypeConvert(o, REDIS_ENCODING_HT);
 
         /* Load every field and value into the ziplist */
-        while (o->encoding == REDIS_ENCODING_ZIPLIST && len-- > 0) {
+        while (o->encoding == REDIS_ENCODING_ZIPLIST && len > 0) {
             robj *field, *value;
 
+            len--;
             /* Load raw strings */
             field = rdbLoadStringObject(rdb);
             if (field == NULL) return NULL;
@@ -855,23 +865,27 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
             if (value == NULL) return NULL;
             redisAssert(field->encoding == REDIS_ENCODING_RAW);
 
+            /* Add pair to ziplist */
+            o->ptr = ziplistPush(o->ptr, field->ptr, sdslen(field->ptr), ZIPLIST_TAIL);
+            o->ptr = ziplistPush(o->ptr, value->ptr, sdslen(value->ptr), ZIPLIST_TAIL);
             /* Convert to hash table if size threshold is exceeded */
             if (sdslen(field->ptr) > server.hash_max_ziplist_value ||
                 sdslen(value->ptr) > server.hash_max_ziplist_value)
             {
+                decrRefCount(field);
+                decrRefCount(value);
                 hashTypeConvert(o, REDIS_ENCODING_HT);
                 break;
             }
-
-            /* Add pair to ziplist */
-            o->ptr = ziplistPush(o->ptr, field->ptr, sdslen(field->ptr), ZIPLIST_TAIL);
-            o->ptr = ziplistPush(o->ptr, value->ptr, sdslen(value->ptr), ZIPLIST_TAIL);
+            decrRefCount(field);
+            decrRefCount(value);
         }
 
         /* Load remaining fields and values into the hash table */
-        while (o->encoding == REDIS_ENCODING_HT && len-- > 0) {
+        while (o->encoding == REDIS_ENCODING_HT && len > 0) {
             robj *field, *value;
 
+            len--;
             /* Load encoded strings */
             field = rdbLoadEncodedStringObject(rdb);
             if (field == NULL) return NULL;
@@ -1014,6 +1028,8 @@ int rdbLoad(char *filename) {
         return REDIS_ERR;
     }
     rioInitWithFile(&rdb,fp);
+    if (server.rdb_checksum)
+        rdb.update_cksum = rioGenericUpdateChecksum;
     if (rioRead(&rdb,buf,9) == 0) goto eoferr;
     buf[9] = '\0';
     if (memcmp(buf,"REDIS",5) != 0) {
@@ -1023,7 +1039,7 @@ int rdbLoad(char *filename) {
         return REDIS_ERR;
     }
     rdbver = atoi(buf+5);
-    if (rdbver < 1 || rdbver > 3) {
+    if (rdbver < 1 || rdbver > REDIS_RDB_VERSION) {
         fclose(fp);
         redisLog(REDIS_WARNING,"Can't handle RDB format version %d",rdbver);
         errno = EINVAL;
@@ -1037,7 +1053,7 @@ int rdbLoad(char *filename) {
 
         /* Serve the clients from time to time */
         if (!(loops++ % 1000)) {
-            loadingProgress(rdb.tell(&rdb));
+            loadingProgress(rioTell(&rdb));
             aeProcessEvents(server.el, AE_FILE_EVENTS|AE_DONT_WAIT);
         }
 
@@ -1094,6 +1110,20 @@ int rdbLoad(char *filename) {
 
         decrRefCount(key);
     }
+    /* Verify the checksum if RDB version is >= 5 */
+    if (rdbver >= 5 && server.rdb_checksum) {
+        uint64_t cksum, expected = rdb.cksum;
+
+        if (rioRead(&rdb,&cksum,8) == 0) goto eoferr;
+        memrev64ifbe(&cksum);
+        if (cksum == 0) {
+            redisLog(REDIS_WARNING,"RDB file was saved with checksum disabled: no check performed.");
+        } else if (cksum != expected) {
+            redisLog(REDIS_WARNING,"Wrong RDB checksum. Aborting now.");
+            exit(1);
+        }
+    }
+
     fclose(fp);
     stopLoading();
     return REDIS_OK;

src/rdb.h

@@ -7,6 +7,10 @@
 /* TBD: include only necessary headers. */
 #include "redis.h"
 
+/* The current RDB version. When the format changes in a way that is no longer
+ * backward compatible this number gets incremented. */
+#define REDIS_RDB_VERSION 6
+
 /* Defines related to the dump file format. To store 32 bits lengths for short
  * keys requires a lot of space, so we check the most significant 2 bits of
  * the first byte to interpreter the length:

src/redis-check-dump.c

@@ -20,6 +20,7 @@
 #define REDIS_LIST_ZIPLIST 10
 #define REDIS_SET_INTSET 11
 #define REDIS_ZSET_ZIPLIST 12
+#define REDIS_HASH_ZIPLIST 13
 
 /* Objects encoding. Some kind of objects like Strings and Hashes can be
  * internally represented in multiple ways. The 'encoding' field of the object
@@ -107,6 +108,9 @@ static double R_Zero, R_PosInf, R_NegInf, R_Nan;
 /* store string types for output */
 static char types[256][16];
 
+/* Prototypes */
+uint64_t crc64(uint64_t crc, const unsigned char *s, uint64_t l);
+
 /* when number of bytes to read is negative, do a peek */
 int readBytes(void *target, long num) {
     char peek = (num < 0) ? 1 : 0;
@@ -136,10 +140,10 @@ int processHeader() {
     }
 
     dump_version = (int)strtol(buf + 5, NULL, 10);
-    if (dump_version < 1 || dump_version > 2) {
+    if (dump_version < 1 || dump_version > 6) {
         ERROR("Unknown RDB format version: %d\n", dump_version);
     }
-    return 1;
+    return dump_version;
 }
 
 int loadType(entry *e) {
@@ -384,6 +388,7 @@ int loadPair(entry *e) {
     case REDIS_LIST_ZIPLIST:
     case REDIS_SET_INTSET:
     case REDIS_ZSET_ZIPLIST:
+    case REDIS_HASH_ZIPLIST:
         if (!processStringObject(NULL)) {
             SHIFT_ERROR(offset, "Error reading entry value");
             return 0;
@@ -555,7 +560,16 @@ void printErrorStack(entry *e) {
 void process() {
     uint64_t num_errors = 0, num_valid_ops = 0, num_valid_bytes = 0;
     entry entry;
-    processHeader();
+    int dump_version = processHeader();
+
+    /* Exclude the final checksum for RDB >= 5. Will be checked at the end. */
+    if (dump_version >= 5) {
+        if (positions[0].size < 8) {
+            printf("RDB version >= 5 but no room for checksum.\n");
+            exit(1);
+        }
+        positions[0].size -= 8;;
+    }
 
     level = 1;
     while(positions[0].offset < positions[0].size) {
@@ -620,6 +634,26 @@ void process() {
         num_errors++;
     }
 
+    /* Verify checksum */
+    if (dump_version >= 5) {
+        uint64_t crc = crc64(0,positions[0].data,positions[0].size);
+        uint64_t crc2;
+        unsigned char *p = (unsigned char*)positions[0].data+positions[0].size;
+        crc2 = ((uint64_t)p[0] << 0) |
+               ((uint64_t)p[1] << 8) |
+               ((uint64_t)p[2] << 16) |
+               ((uint64_t)p[3] << 24) |
+               ((uint64_t)p[4] << 32) |
+               ((uint64_t)p[5] << 40) |
+               ((uint64_t)p[6] << 48) |
+               ((uint64_t)p[7] << 56);
+        if (crc != crc2) {
+            SHIFT_ERROR(positions[0].offset, "RDB CRC64 does not match.");
+        } else {
+            printf("CRC64 checksum is OK\n");
+        }
+    }
+
     /* print summary on errors */
     if (num_errors) {
         printf("\n");

src/redis-cli.c

@@ -69,6 +69,7 @@ static struct config {
     int cluster_mode;
     int cluster_reissue_command;
     int slave_mode;
+    int bigkeys;
     int stdinarg; /* get last arg from stdin. (-x option) */
     char *auth;
     int output; /* output mode, see OUTPUT_* defines */
@@ -478,7 +479,7 @@ static sds cliFormatReplyCSV(redisReply *r) {
 static int cliReadReply(int output_raw_strings) {
     void *_reply;
     redisReply *reply;
-    sds out;
+    sds out = NULL;
     int output = 1;
 
     if (redisGetReply(context,&_reply) != REDIS_OK) {
@@ -498,7 +499,8 @@ static int cliReadReply(int output_raw_strings) {
 
     reply = (redisReply*)_reply;
 
-    /* Check if we need to connect to a different node and reissue the request. */
+    /* Check if we need to connect to a different node and reissue the
+     * request. */
     if (config.cluster_mode && reply->type == REDIS_REPLY_ERROR &&
         (!strncmp(reply->str,"MOVED",5) || !strcmp(reply->str,"ASK")))
     {
@@ -654,6 +656,8 @@ static int parseOptions(int argc, char **argv) {
             config.latency_mode = 1;
         } else if (!strcmp(argv[i],"--slave")) {
             config.slave_mode = 1;
+        } else if (!strcmp(argv[i],"--bigkeys")) {
+            config.bigkeys = 1;
         } else if (!strcmp(argv[i],"--eval") && !lastarg) {
             config.eval = argv[++i];
         } else if (!strcmp(argv[i],"-c")) {
@@ -710,6 +714,7 @@ static void usage() {
 "  --raw            Use raw formatting for replies (default when STDOUT is not a tty)\n"
 "  --latency        Enter a special mode continuously sampling latency.\n"
 "  --slave          Simulate a slave showing commands received from the master.\n"
+"  --bigkeys        Sample Redis keys looking for big keys.\n"
 "  --eval <file>    Send an EVAL command using the Lua script at <file>.\n"
 "  --help           Output this help and exit\n"
 "  --version        Output version and exit\n"
@@ -885,7 +890,7 @@ static int evalMode(int argc, char **argv) {
 
 static void latencyMode(void) {
     redisReply *reply;
-    long long start, latency, min, max, tot, count = 0;
+    long long start, latency, min = 0, max = 0, tot = 0, count = 0;
     double avg;
 
     if (!context) exit(1);
@@ -926,7 +931,10 @@ static void slaveMode(void) {
     unsigned long long payload;
 
     /* Send the SYNC command. */
-    write(fd,"SYNC\r\n",6);
+    if (write(fd,"SYNC\r\n",6) != 6) {
+        fprintf(stderr,"Error writing to master\n");
+        exit(1);
+    }
 
     /* Read $<payload>\r\n, making sure to read just up to "\n" */
     p = buf;
@@ -960,6 +968,87 @@ static void slaveMode(void) {
     while (cliReadReply(0) == REDIS_OK);
 }
 
+#define TYPE_STRING 0
+#define TYPE_LIST   1
+#define TYPE_SET    2
+#define TYPE_HASH   3
+#define TYPE_ZSET   4
+
+static void findBigKeys(void) {
+    unsigned long long biggest[5] = {0,0,0,0,0};
+    unsigned long long samples = 0;
+    redisReply *reply1, *reply2, *reply3 = NULL;
+    char *sizecmd, *typename[] = {"string","list","set","hash","zset"};
+    int type;
+
+    printf("\n# Press ctrl+c when you have had enough of it... :)\n");
+    printf("# You can use -i 0.1 to sleep 0.1 sec every 100 sampled keys\n");
+    printf("# in order to reduce server load (usually not needed).\n\n");
+    while(1) {
+        /* Sample with RANDOMKEY */
+        reply1 = redisCommand(context,"RANDOMKEY");
+        if (reply1 == NULL) {
+            fprintf(stderr,"\nI/O error\n");
+            exit(1);
+        } else if (reply1->type == REDIS_REPLY_ERROR) {
+            fprintf(stderr, "RANDOMKEY error: %s\n",
+                reply1->str);
+            exit(1);
+        }
+        /* Get the key type */
+        reply2 = redisCommand(context,"TYPE %s",reply1->str);
+        assert(reply2 && reply2->type == REDIS_REPLY_STATUS);
+        samples++;
+
+        /* Get the key "size" */
+        if (!strcmp(reply2->str,"string")) {
+            sizecmd = "STRLEN";
+            type = TYPE_STRING;
+        } else if (!strcmp(reply2->str,"list")) {
+            sizecmd = "LLEN";
+            type = TYPE_LIST;
+        } else if (!strcmp(reply2->str,"set")) {
+            sizecmd = "SCARD";
+            type = TYPE_SET;
+        } else if (!strcmp(reply2->str,"hash")) {
+            sizecmd = "HLEN";
+            type = TYPE_HASH;
+        } else if (!strcmp(reply2->str,"zset")) {
+            sizecmd = "ZCARD";
+            type = TYPE_ZSET;
+        } else if (!strcmp(reply2->str,"none")) {
+            freeReplyObject(reply1);
+            freeReplyObject(reply2);
+            freeReplyObject(reply3);
+            continue;
+        } else {
+            fprintf(stderr, "Unknown key type '%s' for key '%s'\n",
+                reply2->str, reply1->str);
+            exit(1);
+        }
+
+        reply3 = redisCommand(context,"%s %s", sizecmd, reply1->str);
+        if (reply3 && reply3->type == REDIS_REPLY_INTEGER) {
+            if (biggest[type] < reply3->integer) {
+                printf("[%6s] %s | biggest so far with size %llu\n",
+                    typename[type], reply1->str,
+                    (unsigned long long) reply3->integer);
+                biggest[type] = reply3->integer;
+            }
+        }
+
+        if ((samples % 1000000) == 0)
+            printf("(%llu keys sampled)\n", samples);
+
+        if ((samples % 100) == 0 && config.interval)
+            usleep(config.interval);
+
+        freeReplyObject(reply1);
+        freeReplyObject(reply2);
+        if (reply3) freeReplyObject(reply3);
+    }
+}
+
 int main(int argc, char **argv) {
     int firstarg;
 
@@ -975,6 +1064,8 @@ int main(int argc, char **argv) {
     config.pubsub_mode = 0;
     config.latency_mode = 0;
     config.cluster_mode = 0;
+    config.slave_mode = 0;
+    config.bigkeys = 0;
     config.stdinarg = 0;
     config.auth = NULL;
     config.eval = NULL;
@@ -1001,6 +1092,12 @@ int main(int argc, char **argv) {
         slaveMode();
     }
 
+    /* Find big keys */
+    if (config.bigkeys) {
+        cliConnect(0);
+        findBigKeys();
+    }
+
     /* Start interactive mode when no command is provided */
     if (argc == 0 && !config.eval) {
         /* Note that in repl mode we don't abort on connection error.

src/redis-trib.rb

@@ -1,492 +0,0 @@
-#!/usr/bin/env ruby
-
-# TODO (temporary here, we'll move this into the Github issues once
-#       redis-trib initial implementation is complted).
-#
-# - Make sure that if the rehashing fails in the middle redis-trib will try
-#   to recover.
-# - When redis-trib performs a cluster check, if it detects a slot move in
-#   progress it should prompt the user to continue the move from where it
-#   stopped.
-# - Gracefully handle Ctrl+C in move_slot to prompt the user if really stop
-#   while rehashing, and performing the best cleanup possible if the user
-#   forces the quit.
-# - When doing "fix" set a global Fix to true, and prompt the user to
-#   fix the problem if automatically fixable every time there is something
-#   to fix. For instance:
-#   1) If there is a node that pretend to receive a slot, or to migrate a
-#      slot, but has no entries in that slot, fix it.
-#   2) If there is a node having keys in slots that are not owned by it
-#      fix this condiiton moving the entries in the same node.
-#   3) Perform more possibly slow tests about the state of the cluster.
-#   4) When aborted slot migration is detected, fix it.
-
-require 'rubygems'
-require 'redis'
-
-ClusterHashSlots = 4096
-
-def xputs(s)
-    printf s
-    STDOUT.flush
-end
-
-class ClusterNode
-    def initialize(addr)
-        s = addr.split(":")
-        if s.length != 2
-            puts "Invalid node name #{addr}"
-            exit 1
-        end
-        @r = nil
-        @info = {}
-        @info[:host] = s[0]
-        @info[:port] = s[1]
-        @info[:slots] = {}
-        @dirty = false # True if we need to flush slots info into node.
-        @friends = []
-    end
-
-    def friends
-        @friends
-    end
-
-    def slots 
-        @info[:slots]
-    end
-
-    def to_s
-        "#{@info[:host]}:#{@info[:port]}"
-    end
-
-    def connect(o={})
-        return if @r
-        xputs "Connecting to node #{self}: "
-        begin
-            @r = Redis.new(:host => @info[:host], :port => @info[:port])
-            @r.ping
-        rescue
-            puts "ERROR"
-            puts "Sorry, can't connect to node #{self}"
-            exit 1 if o[:abort]
-            @r = nil
-        end
-        puts "OK"
-    end
-
-    def assert_cluster
-        info = @r.info
-        if !info["cluster_enabled"] || info["cluster_enabled"].to_i == 0
-            puts "Error: Node #{self} is not configured as a cluster node."
-            exit 1
-        end
-    end
-
-    def assert_empty
-        if !(@r.cluster("info").split("\r\n").index("cluster_known_nodes:1")) ||
-            (@r.info['db0'])
-            puts "Error: Node #{self} is not empty. Either the node already knows other nodes (check with nodes-info) or contains some key in database 0."
-            exit 1
-        end
-    end
-
-    def load_info(o={})
-        self.connect
-        nodes = @r.cluster("nodes").split("\n")
-        nodes.each{|n|
-            # name addr flags role ping_sent ping_recv link_status slots
-            split = n.split
-            name,addr,flags,role,ping_sent,ping_recv,link_status = split[0..6]
-            slots = split[7..-1]
-            info = {
-                :name => name,
-                :addr => addr,
-                :flags => flags.split(","),
-                :role => role,
-                :ping_sent => ping_sent.to_i,
-                :ping_recv => ping_recv.to_i,
-                :link_status => link_status
-            }
-            if info[:flags].index("myself")
-                @info = @info.merge(info)
-                @info[:slots] = {}
-                slots.each{|s|
-                    if s[0..0] == '['
-                        # Fixme: for now skipping migration entries
-                    elsif s.index("-")
-                        start,stop = s.split("-")
-                        self.add_slots((start.to_i)..(stop.to_i))
-                    else
-                        self.add_slots((s.to_i)..(s.to_i))
-                    end
-                } if slots
-                @dirty = false
-                @r.cluster("info").split("\n").each{|e|    
-                    k,v=e.split(":")
-                    k = k.to_sym
-                    v.chop!
-                    if k != :cluster_state
-                        @info[k] = v.to_i
-                    else
-                        @info[k] = v
-                    end
-                }
-            elsif o[:getfriends]
-                @friends << info
-            end
-        }
-    end
-
-    def add_slots(slots)
-        slots.each{|s|
-            @info[:slots][s] = :new
-        }
-        @dirty = true
-    end
-
-    def flush_node_config
-        return if !@dirty
-        new = []
-        @info[:slots].each{|s,val|
-            if val == :new
-                new << s
-                @info[:slots][s] = true
-            end
-        }
-        @r.cluster("addslots",*new)
-        @dirty = false
-    end
-
-    def info_string
-        # We want to display the hash slots assigned to this node
-        # as ranges, like in: "1-5,8-9,20-25,30"
-        #
-        # Note: this could be easily written without side effects,
-        # we use 'slots' just to split the computation into steps.
-        
-        # First step: we want an increasing array of integers
-        # for instance: [1,2,3,4,5,8,9,20,21,22,23,24,25,30]
-        slots = @info[:slots].keys.sort
-
-        # As we want to aggregate adiacent slots we convert all the
-        # slot integers into ranges (with just one element)
-        # So we have something like [1..1,2..2, ... and so forth.
-        slots.map!{|x| x..x}
-
-        # Finally we group ranges with adiacent elements.
-        slots = slots.reduce([]) {|a,b|
-            if !a.empty? && b.first == (a[-1].last)+1
-                a[0..-2] + [(a[-1].first)..(b.last)]
-            else
-                a + [b]
-            end
-        }
-
-        # Now our task is easy, we just convert ranges with just one
-        # element into a number, and a real range into a start-end format.
-        # Finally we join the array using the comma as separator.
-        slots = slots.map{|x|
-            x.count == 1 ? x.first.to_s : "#{x.first}-#{x.last}"
-        }.join(",")
-
-        "[#{@info[:cluster_state].upcase}] #{self.info[:name]} #{self.to_s} slots:#{slots} (#{self.slots.length} slots)"
-    end
-
-    def info
-        @info
-    end
-    
-    def is_dirty?
-        @dirty
-    end
-
-    def r
-        @r
-    end
-end
-
-class RedisTrib
-    def initialize
-        @nodes = []
-    end
-
-    def check_arity(req_args, num_args)
-        if ((req_args > 0 and num_args != req_args) ||
-           (req_args < 0 and num_args < req_args.abs))
-           puts "Wrong number of arguments for specified sub command"
-           exit 1
-        end
-    end
-
-    def add_node(node)
-        @nodes << node
-    end
-
-    def get_node_by_name(name)
-        @nodes.each{|n|
-            return n if n.info[:name] == name.downcase
-        }
-        return nil
-    end
-
-    def check_cluster
-        puts "Performing Cluster Check (using node #{@nodes[0]})"
-        errors = []
-        show_nodes
-        # Check if all the slots are covered
-        slots = {}
-        @nodes.each{|n|
-            slots = slots.merge(n.slots)
-        }
-        if slots.length == 4096
-            puts "[OK] All 4096 slots covered."
-        else
-            errors << "[ERR] Not all 4096 slots are covered by nodes."
-            puts errors[-1]
-        end
-        return errors
-    end
-
-    def alloc_slots
-        slots_per_node = ClusterHashSlots/@nodes.length
-        i = 0
-        @nodes.each{|n|
-            first = i*slots_per_node
-            last = first+slots_per_node-1
-            last = ClusterHashSlots-1 if i == @nodes.length-1
-            n.add_slots first..last
-            i += 1
-        }
-    end
-
-    def flush_nodes_config
-        @nodes.each{|n|
-            n.flush_node_config
-        }
-    end
-
-    def show_nodes
-        @nodes.each{|n|
-            puts n.info_string
-        }
-    end
-
-    def join_cluster
-        # We use a brute force approach to make sure the node will meet
-        # each other, that is, sending CLUSTER MEET messages to all the nodes
-        # about the very same node.
-        # Thanks to gossip this information should propagate across all the
-        # cluster in a matter of seconds.
-        first = false
-        @nodes.each{|n|
-            if !first then first = n.info; next; end # Skip the first node
-            n.r.cluster("meet",first[:host],first[:port])
-        }
-    end
-
-    def yes_or_die(msg)
-        print "#{msg} (type 'yes' to accept): "
-        STDOUT.flush
-        if !(STDIN.gets.chomp.downcase == "yes")
-            puts "Aborting..."
-            exit 1
-        end
-    end
-
-    def load_cluster_info_from_node(nodeaddr)
-        node = ClusterNode.new(ARGV[1])
-        node.connect(:abort => true)
-        node.assert_cluster
-        node.load_info(:getfriends => true)
-        add_node(node)
-        node.friends.each{|f|
-            fnode = ClusterNode.new(f[:addr])
-            fnode.connect()
-            fnode.load_info()
-            add_node(fnode)
-        }
-    end
-
-    # Given a list of source nodes return a "resharding plan"
-    # with what slots to move in order to move "numslots" slots to another
-    # instance.
-    def compute_reshard_table(sources,numslots)
-        moved = []
-        # Sort from bigger to smaller instance, for two reasons:
-        # 1) If we take less slots than instanes it is better to start getting from
-        #    the biggest instances.
-        # 2) We take one slot more from the first instance in the case of not perfect
-        #    divisibility. Like we have 3 nodes and need to get 10 slots, we take
-        #    4 from the first, and 3 from the rest. So the biggest is always the first.
-        sources = sources.sort{|a,b| b.slots.length <=> a.slots.length}
-        source_tot_slots = sources.inject(0) {|sum,source| sum+source.slots.length}
-        sources.each_with_index{|s,i|
-            # Every node will provide a number of slots proportional to the
-            # slots it has assigned.
-            n = (numslots.to_f/source_tot_slots*s.slots.length)
-            if i == 0
-                n = n.ceil
-            else
-                n = n.floor
-            end
-            s.slots.keys.sort[(0...n)].each{|slot|
-                if moved.length < numslots
-                    moved << {:source => s, :slot => slot}
-                end
-            }
-        }
-        return moved
-    end
-
-    def show_reshard_table(table)
-        table.each{|e|
-            puts "    Moving slot #{e[:slot]} from #{e[:source].info[:name]}"
-        }
-    end
-
-    def move_slot(source,target,slot,o={})
-        # We start marking the slot as importing in the destination node,
-        # and the slot as migrating in the target host. Note that the order of
-        # the operations is important, as otherwise a client may be redirected to
-        # the target node that does not yet know it is importing this slot.
-        print "Moving slot #{slot} from #{source.info_string}: "; STDOUT.flush
-        target.r.cluster("setslot",slot,"importing",source.info[:name])
-        source.r.cluster("setslot",slot,"migrating",source.info[:name])
-        # Migrate all the keys from source to target using the MIGRATE command
-        while true
-            keys = source.r.cluster("getkeysinslot",slot,10)
-            break if keys.length == 0
-            keys.each{|key|
-                source.r.migrate(target.info[:host],target.info[:port],key,0,1)
-                print "." if o[:verbose]
-                STDOUT.flush
-            }
-        end
-        puts
-        # Set the new node as the owner of the slot in all the known nodes.
-        @nodes.each{|n|
-            n.r.cluster("setslot",slot,"node",target.info[:name])
-        }
-    end
-
-    # redis-trib subcommands implementations
-
-    def check_cluster_cmd   
-        load_cluster_info_from_node(ARGV[1])
-        check_cluster
-    end
-
-    def reshard_cluster_cmd
-        load_cluster_info_from_node(ARGV[1])
-        errors = check_cluster
-        if errors.length != 0
-            puts "Please fix your cluster problems before resharding."
-            exit 1
-        end
-        numslots = 0
-        while numslots <= 0 or numslots > 4096
-            print "How many slots do you want to move (from 1 to 4096)? "
-            numslots = STDIN.gets.to_i
-        end
-        target = nil
-        while not target
-            print "What is the receiving node ID? "
-            target = get_node_by_name(STDIN.gets.chop)
-            if not target
-                puts "The specified node is not known, please retry."
-            end
-        end
-        sources = []
-        puts "Please enter all the source node IDs."
-        puts "  Type 'all' to use all the nodes as source nodes for the hash slots."
-        puts "  Type 'done' once you entered all the source nodes IDs."
-        while true
-            print "Source node ##{sources.length+1}:"
-            line = STDIN.gets.chop
-            src = get_node_by_name(line)
-            if line == "done"
-                if sources.length == 0
-                    puts "No source nodes given, operation aborted"
-                    exit 1
-                else
-                    break
-                end
-            elsif line == "all"
-                @nodes.each{|n|
-                    next if n.info[:name] == target.info[:name]
-                    sources << n
-                }
-                break
-            elsif not src
-                puts "The specified node is not known, please retry."
-            elsif src.info[:name] == target.info[:name]
-                puts "It is not possible to use the target node as source node."
-            else
-                sources << src
-            end
-        end
-        puts "\nReady to move #{numslots} slots."
-        puts "  Source nodes:"
-        sources.each{|s| puts "    "+s.info_string}
-        puts "  Destination node:"
-        puts "    #{target.info_string}"
-        reshard_table = compute_reshard_table(sources,numslots)
-        puts "  Resharding plan:"
-        show_reshard_table(reshard_table)
-        print "Do you want to proceed with the proposed reshard plan (yes/no)? "
-        yesno = STDIN.gets.chop
-        exit(1) if (yesno != "yes")
-        reshard_table.each{|e|
-            move_slot(e[:source],target,e[:slot],:verbose=>true)
-        }
-    end
-
-    def create_cluster_cmd
-        puts "Creating cluster"
-        ARGV[1..-1].each{|n|
-            node = ClusterNode.new(n)
-            node.connect(:abort => true)
-            node.assert_cluster
-            node.load_info
-            node.assert_empty
-            add_node(node)
-        }
-        puts "Performing hash slots allocation on #{@nodes.length} nodes..."
-        alloc_slots
-        show_nodes
-        yes_or_die "Can I set the above configuration?"
-        flush_nodes_config
-        puts "** Nodes configuration updated"
-        puts "** Sending CLUSTER MEET messages to join the cluster"
-        join_cluster
-        check_cluster
-    end
-end
-
-COMMANDS={
-    "create" => ["create_cluster_cmd", -2, "host1:port host2:port ... hostN:port"],
-    "check" =>  ["check_cluster_cmd", 2, "host:port"],
-    "reshard" =>  ["reshard_cluster_cmd", 2, "host:port"]
-}
-
-# Sanity check
-if ARGV.length == 0
-    puts "Usage: redis-trib <command> <arguments ...>"
-    puts
-    COMMANDS.each{|k,v|
-        puts "  #{k.ljust(20)} #{v[2]}"
-    }
-    puts
-    exit 1
-end
-
-rt = RedisTrib.new
-cmd_spec = COMMANDS[ARGV[0].downcase]
-if !cmd_spec
-    puts "Unknown redis-trib subcommand '#{ARGV[0]}'"
-    exit 1
-end
-rt.check_arity(cmd_spec[1],ARGV.length)
-
-# Dispatch
-rt.send(cmd_spec[0])

src/redis.c

@@ -48,6 +48,7 @@
 #include <float.h>
 #include <math.h>
 #include <sys/resource.h>
+#include <sys/utsname.h>
 
 /* Our shared "common" objects */
 
@@ -211,7 +212,7 @@ struct redisCommand redisCommandTable[] = {
     {"lastsave",lastsaveCommand,1,"r",0,NULL,0,0,0,0,0},
     {"type",typeCommand,2,"r",0,NULL,1,1,1,0,0},
     {"multi",multiCommand,1,"rs",0,NULL,0,0,0,0,0},
-    {"exec",execCommand,1,"wms",0,NULL,0,0,0,0,0},
+    {"exec",execCommand,1,"s",0,NULL,0,0,0,0,0},
     {"discard",discardCommand,1,"rs",0,NULL,0,0,0,0,0},
     {"sync",syncCommand,1,"ars",0,NULL,0,0,0,0,0},
     {"flushdb",flushdbCommand,1,"w",0,NULL,0,0,0,0,0},
@@ -222,25 +223,23 @@ struct redisCommand redisCommandTable[] = {
     {"ttl",ttlCommand,2,"r",0,NULL,1,1,1,0,0},
     {"pttl",pttlCommand,2,"r",0,NULL,1,1,1,0,0},
     {"persist",persistCommand,2,"w",0,NULL,1,1,1,0,0},
-    {"slaveof",slaveofCommand,3,"aws",0,NULL,0,0,0,0,0},
-    {"debug",debugCommand,-2,"aws",0,NULL,0,0,0,0,0},
+    {"slaveof",slaveofCommand,3,"as",0,NULL,0,0,0,0,0},
+    {"debug",debugCommand,-2,"as",0,NULL,0,0,0,0,0},
     {"config",configCommand,-2,"ar",0,NULL,0,0,0,0,0},
     {"subscribe",subscribeCommand,-2,"rps",0,NULL,0,0,0,0,0},
     {"unsubscribe",unsubscribeCommand,-1,"rps",0,NULL,0,0,0,0,0},
     {"psubscribe",psubscribeCommand,-2,"rps",0,NULL,0,0,0,0,0},
     {"punsubscribe",punsubscribeCommand,-1,"rps",0,NULL,0,0,0,0,0},
-    {"publish",publishCommand,3,"rpf",0,NULL,0,0,0,0,0},
+    {"publish",publishCommand,3,"pf",0,NULL,0,0,0,0,0},
     {"watch",watchCommand,-2,"rs",0,noPreloadGetKeys,1,-1,1,0,0},
     {"unwatch",unwatchCommand,1,"rs",0,NULL,0,0,0,0,0},
-    {"cluster",clusterCommand,-2,"ar",0,NULL,0,0,0,0,0},
     {"restore",restoreCommand,4,"awm",0,NULL,1,1,1,0,0},
     {"migrate",migrateCommand,6,"aw",0,NULL,0,0,0,0,0},
-    {"asking",askingCommand,1,"r",0,NULL,0,0,0,0,0},
     {"dump",dumpCommand,2,"ar",0,NULL,1,1,1,0,0},
     {"object",objectCommand,-2,"r",0,NULL,2,2,2,0,0},
     {"client",clientCommand,-2,"ar",0,NULL,0,0,0,0,0},
-    {"eval",evalCommand,-3,"wms",0,zunionInterGetKeys,0,0,0,0,0},
-    {"evalsha",evalShaCommand,-3,"wms",0,zunionInterGetKeys,0,0,0,0,0},
+    {"eval",evalCommand,-3,"s",0,zunionInterGetKeys,0,0,0,0,0},
+    {"evalsha",evalShaCommand,-3,"s",0,zunionInterGetKeys,0,0,0,0,0},
     {"slowlog",slowlogCommand,-2,"r",0,NULL,0,0,0,0,0},
     {"script",scriptCommand,-2,"ras",0,NULL,0,0,0,0,0},
     {"time",timeCommand,1,"rR",0,NULL,0,0,0,0,0}
@@ -253,7 +252,6 @@ struct redisCommand redisCommandTable[] = {
 void redisLogRaw(int level, const char *msg) {
     const int syslogLevelMap[] = { LOG_DEBUG, LOG_INFO, LOG_NOTICE, LOG_WARNING };
     const char *c = ".-*#";
-    time_t now = time(NULL);
     FILE *fp;
     char buf[64];
     int rawmode = (level & REDIS_LOG_RAW);
@@ -267,7 +265,12 @@ void redisLogRaw(int level, const char *msg) {
     if (rawmode) {
         fprintf(fp,"%s",msg);
     } else {
-        strftime(buf,sizeof(buf),"%d %b %H:%M:%S",localtime(&now));
+        int off;
+        struct timeval tv;
+
+        gettimeofday(&tv,NULL);
+        off = strftime(buf,sizeof(buf),"%d %b %H:%M:%S.",localtime(&tv.tv_sec));
+        snprintf(buf+off,sizeof(buf)-off,"%03d",(int)tv.tv_usec/1000);
         fprintf(fp,"[%d] %s %c %s\n",(int)getpid(),buf,c[level],msg);
     }
     fflush(fp);
@@ -293,6 +296,35 @@ void redisLog(int level, const char *fmt, ...) {
     redisLogRaw(level,msg);
 }
 
+/* Log a fixed message without printf-alike capabilities, in a way that is
+ * safe to call from a signal handler.
+ *
+ * We actually use this only for signals that are not fatal from the point
+ * of view of Redis. Signals that are going to kill the server anyway and
+ * where we need printf-alike features are served by redisLog(). */
+void redisLogFromHandler(int level, const char *msg) {
+    int fd;
+    char buf[64];
+
+    if ((level&0xff) < server.verbosity ||
+        (server.logfile == NULL && server.daemonize)) return;
+    fd = server.logfile ?
+        open(server.logfile, O_APPEND|O_CREAT|O_WRONLY, 0644) :
+        STDOUT_FILENO;
+    if (fd == -1) return;
+    ll2string(buf,sizeof(buf),getpid());
+    if (write(fd,"[",1) == -1) goto err;
+    if (write(fd,buf,strlen(buf)) == -1) goto err;
+    if (write(fd," | signal handler] (",20) == -1) goto err;
+    ll2string(buf,sizeof(buf),time(NULL));
+    if (write(fd,buf,strlen(buf)) == -1) goto err;
+    if (write(fd,") ",2) == -1) goto err;
+    if (write(fd,msg,strlen(msg)) == -1) goto err;
+    if (write(fd,"\n",1) == -1) goto err;
+err:
+    if (server.logfile) close(fd);
+}
+
 /* Redis generally does not try to recover from out of memory conditions
  * when allocating objects or strings, it is not clear if it will be possible
  * to report this condition to the client since the networking layer itself
@@ -320,6 +352,18 @@ long long mstime(void) {
     return ustime()/1000;
 }
 
+/* After an RDB dump or AOF rewrite we exit from children using _exit() instead of
+ * exit(), because the latter may interact with the same file objects used by
+ * the parent process. However if we are testing the coverage normal exit() is
+ * used in order to obtain the right coverage information. */
+void exitFromChild(int retcode) {
+#ifdef COVERAGE_TEST
+    exit(retcode);
+#else
+    _exit(retcode);
+#endif
+}
+
 /*====================== Hash table type implementation  ==================== */
 
 /* This is an hash table type that uses the SDS dynamic strings libary as
@@ -507,17 +551,6 @@ dictType keylistDictType = {
     dictListDestructor          /* val destructor */
 };
 
-/* Cluster nodes hash table, mapping nodes addresses 1.2.3.4:6379 to
- * clusterNode structures. */
-dictType clusterNodesDictType = {
-    dictSdsHash,                /* hash function */
-    NULL,                       /* key dup */
-    NULL,                       /* val dup */
-    dictSdsKeyCompare,          /* key compare */
-    dictSdsDestructor,          /* key destructor */
-    NULL                        /* val destructor */
-};
-
 int htNeedsResize(dict *dict) {
     long long size, used;
 
@@ -612,7 +645,7 @@ void activeExpireCycle(void) {
 }
 
 void updateLRUClock(void) {
-    server.lruclock = (time(NULL)/REDIS_LRU_CLOCK_RESOLUTION) &
+    server.lruclock = (server.unixtime/REDIS_LRU_CLOCK_RESOLUTION) &
                                                 REDIS_LRU_CLOCK_MAX;
 }
 
@@ -641,12 +674,95 @@ long long getOperationsPerSecond(void) {
     return sum / REDIS_OPS_SEC_SAMPLES;
 }
 
+/* Check for timeouts. Returns non-zero if the client was terminated */
+int clientsCronHandleTimeout(redisClient *c) {
+    time_t now = server.unixtime;
+
+    if (server.maxidletime &&
+        !(c->flags & REDIS_SLAVE) &&    /* no timeout for slaves */
+        !(c->flags & REDIS_MASTER) &&   /* no timeout for masters */
+        !(c->flags & REDIS_BLOCKED) &&  /* no timeout for BLPOP */
+        dictSize(c->pubsub_channels) == 0 && /* no timeout for pubsub */
+        listLength(c->pubsub_patterns) == 0 &&
+        (now - c->lastinteraction > server.maxidletime))
+    {
+        redisLog(REDIS_VERBOSE,"Closing idle client");
+        freeClient(c);
+        return 1;
+    } else if (c->flags & REDIS_BLOCKED) {
+        if (c->bpop.timeout != 0 && c->bpop.timeout < now) {
+            addReply(c,shared.nullmultibulk);
+            unblockClientWaitingData(c);
+        }
+    }
+    return 0;
+}
+
+/* The client query buffer is an sds.c string that can end with a lot of
+ * free space not used, this function reclaims space if needed.
+ *
+ * The funciton always returns 0 as it never terminates the client. */
+int clientsCronResizeQueryBuffer(redisClient *c) {
+    size_t querybuf_size = sdsAllocSize(c->querybuf);
+    time_t idletime = server.unixtime - c->lastinteraction;
+
+    /* There are two conditions to resize the query buffer:
+     * 1) Query buffer is > BIG_ARG and too big for latest peak.
+     * 2) Client is inactive and the buffer is bigger than 1k. */
+    if (((querybuf_size > REDIS_MBULK_BIG_ARG) &&
+         (querybuf_size/(c->querybuf_peak+1)) > 2) ||
+         (querybuf_size > 1024 && idletime > 2))
+    {
+        /* Only resize the query buffer if it is actually wasting space. */
+        if (sdsavail(c->querybuf) > 1024) {
+            c->querybuf = sdsRemoveFreeSpace(c->querybuf);
+        }
+    }
+    /* Reset the peak again to capture the peak memory usage in the next
+     * cycle. */
+    c->querybuf_peak = 0;
+    return 0;
+}
+
+void clientsCron(void) {
+    /* Make sure to process at least 1/100 of clients per call.
+     * Since this function is called 10 times per second we are sure that
+     * in the worst case we process all the clients in 10 seconds.
+     * In normal conditions (a reasonable number of clients) we process
+     * all the clients in a shorter time. */
+    int numclients = listLength(server.clients);
+    int iterations = numclients/100;
+
+    if (iterations < 50)
+        iterations = (numclients < 50) ? numclients : 50;
+    while(listLength(server.clients) && iterations--) {
+        redisClient *c;
+        listNode *head;
+
+        /* Rotate the list, take the current head, process.
+         * This way if the client must be removed from the list it's the
+         * first element and we don't incur into O(N) computation. */
+        listRotate(server.clients);
+        head = listFirst(server.clients);
+        c = listNodeValue(head);
+        /* The following functions do different service checks on the client.
+         * The protocol is that they return non-zero if the client was
+         * terminated. */
+        if (clientsCronHandleTimeout(c)) continue;
+        if (clientsCronResizeQueryBuffer(c)) continue;
+    }
+}
+
 int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
     int j, loops = server.cronloops;
     REDIS_NOTUSED(eventLoop);
     REDIS_NOTUSED(id);
     REDIS_NOTUSED(clientData);
 
+    /* Software watchdog: deliver the SIGALRM that will reach the signal
+     * handler if we don't return here fast enough. */
+    if (server.watchdog_period) watchdogScheduleSignal(server.watchdog_period);
+
     /* We take a cached value of the unix time in the global state because
      * with virtual memory and aging there is to store the current time
      * in objects at every object access, and accuracy is not needed.
@@ -712,9 +828,8 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
             zmalloc_used_memory());
     }
 
-    /* Close connections of timedout clients */
-    if ((server.maxidletime && !(loops % 100)) || server.bpop_blocked_clients)
-        closeTimedoutClients();
+    /* We need to do a few operations on clients asynchronously. */
+    clientsCron();
 
     /* Start a scheduled AOF rewrite if this was requested by the user while
      * a BGSAVE was in progress. */
@@ -743,15 +858,13 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
             updateDictResizePolicy();
         }
     } else {
-         time_t now = time(NULL);
-
         /* If there is not a background saving/rewrite in progress check if
          * we have to save/rewrite now */
          for (j = 0; j < server.saveparamslen; j++) {
             struct saveparam *sp = server.saveparams+j;
 
             if (server.dirty >= sp->changes &&
-                now-server.lastsave > sp->seconds) {
+                server.unixtime-server.lastsave > sp->seconds) {
                 redisLog(REDIS_NOTICE,"%d changes in %d seconds. Saving...",
                     sp->changes, sp->seconds);
                 rdbSaveBackground(server.rdb_filename);
@@ -792,9 +905,6 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
      * to detect transfer failures. */
     if (!(loops % 10)) replicationCron();
 
-    /* Run other sub-systems specific cron jobs */
-    if (server.cluster_enabled && !(loops % 10)) clusterCron();
-
     server.cronloops++;
     return 100;
 }
@@ -861,20 +971,19 @@ void createSharedObjects(void) {
     shared.slowscripterr = createObject(REDIS_STRING,sdsnew(
         "-BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.\r\n"));
     shared.bgsaveerr = createObject(REDIS_STRING,sdsnew(
-        "-MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Write commands are disabled. Please check Redis logs for details about the error.\r\n"));
+        "-MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.\r\n"));
+    shared.roslaveerr = createObject(REDIS_STRING,sdsnew(
+        "-READONLY You can't write against a read only slave.\r\n"));
+    shared.oomerr = createObject(REDIS_STRING,sdsnew(
+        "-OOM command not allowed when used memory > 'maxmemory'.\r\n"));
     shared.space = createObject(REDIS_STRING,sdsnew(" "));
     shared.colon = createObject(REDIS_STRING,sdsnew(":"));
     shared.plus = createObject(REDIS_STRING,sdsnew("+"));
-    shared.select0 = createStringObject("select 0\r\n",10);
-    shared.select1 = createStringObject("select 1\r\n",10);
-    shared.select2 = createStringObject("select 2\r\n",10);
-    shared.select3 = createStringObject("select 3\r\n",10);
-    shared.select4 = createStringObject("select 4\r\n",10);
-    shared.select5 = createStringObject("select 5\r\n",10);
-    shared.select6 = createStringObject("select 6\r\n",10);
-    shared.select7 = createStringObject("select 7\r\n",10);
-    shared.select8 = createStringObject("select 8\r\n",10);
-    shared.select9 = createStringObject("select 9\r\n",10);
+
+    for (j = 0; j < REDIS_SHARED_SELECT_CMDS; j++) {
+        shared.select[j] = createObject(REDIS_STRING,
+            sdscatprintf(sdsempty(),"select %d\r\n", j));
+    }
     shared.messagebulk = createStringObject("$7\r\nmessage\r\n",13);
     shared.pmessagebulk = createStringObject("$8\r\npmessage\r\n",14);
     shared.subscribebulk = createStringObject("$9\r\nsubscribe\r\n",15);
@@ -925,6 +1034,7 @@ void initServerConfig() {
     server.aof_rewrite_base_size = 0;
     server.aof_rewrite_scheduled = 0;
     server.aof_last_fsync = time(NULL);
+    server.aof_delayed_fsync = 0;
     server.aof_fd = -1;
     server.aof_selected_db = -1; /* Make sure the first time will not match */
     server.aof_flush_postponed_start = 0;
@@ -933,6 +1043,7 @@ void initServerConfig() {
     server.aof_filename = zstrdup("appendonly.aof");
     server.requirepass = NULL;
     server.rdb_compression = 1;
+    server.rdb_checksum = 1;
     server.activerehashing = 1;
     server.maxclients = REDIS_MAX_CLIENTS;
     server.bpop_blocked_clients = 0;
@@ -949,8 +1060,6 @@ void initServerConfig() {
     server.shutdown_asap = 0;
     server.repl_ping_slave_period = REDIS_REPL_PING_SLAVE_PERIOD;
     server.repl_timeout = REDIS_REPL_TIMEOUT;
-    server.cluster_enabled = 0;
-    server.cluster.configfile = zstrdup("nodes.conf");
     server.lua_caller = NULL;
     server.lua_time_limit = REDIS_LUA_TIME_LIMIT;
     server.lua_client = NULL;
@@ -970,7 +1079,8 @@ void initServerConfig() {
     server.repl_state = REDIS_REPL_NONE;
     server.repl_syncio_timeout = REDIS_REPL_SYNCIO_TIMEOUT;
     server.repl_serve_stale_data = 1;
-    server.repl_down_since = -1;
+    server.repl_slave_ro = 1;
+    server.repl_down_since = time(NULL);
 
     /* Client output buffer limits */
     server.client_obuf_limits[REDIS_CLIENT_LIMIT_CLASS_NORMAL].hard_limit_bytes = 0;
@@ -1002,11 +1112,12 @@ void initServerConfig() {
     server.slowlog_log_slower_than = REDIS_SLOWLOG_LOG_SLOWER_THAN;
     server.slowlog_max_len = REDIS_SLOWLOG_MAX_LEN;
 
-    /* Assert */
+    /* Debugging */
     server.assert_failed = "<no assertion failed>";
     server.assert_file = "<no file>";
     server.assert_line = 0;
     server.bug_report_start = 0;
+    server.watchdog_period = 0;
 }
 
 /* This function will try to raise the max number of open files accordingly to
@@ -1021,7 +1132,6 @@ void adjustOpenFilesLimit(void) {
     rlim_t maxfiles = server.maxclients+32;
     struct rlimit limit;
 
-    if (maxfiles < 1024) maxfiles = 1024;
     if (getrlimit(RLIMIT_NOFILE,&limit) == -1) {
         redisLog(REDIS_WARNING,"Unable to obtain the current NOFILE limit (%s), assuming 1024 and setting the max clients configuration accordingly.",
             strerror(errno));
@@ -1032,10 +1142,18 @@ void adjustOpenFilesLimit(void) {
         /* Set the max number of files if the current limit is not enough
          * for our needs. */
         if (oldlimit < maxfiles) {
-            limit.rlim_cur = maxfiles;
-            limit.rlim_max = maxfiles;
-            if (setrlimit(RLIMIT_NOFILE,&limit) == -1) {
-                server.maxclients = oldlimit-32;
+            rlim_t f;
+            
+            f = maxfiles;
+            while(f > oldlimit) {
+                limit.rlim_cur = f;
+                limit.rlim_max = f;
+                if (setrlimit(RLIMIT_NOFILE,&limit) != -1) break;
+                f -= 128;
+            }
+            if (f < oldlimit) f = oldlimit;
+            if (f != maxfiles) {
+                server.maxclients = f-32;
                 redisLog(REDIS_WARNING,"Unable to set the max number of files limit to %d (%s), setting the max clients configuration to %d.",
                     (int) maxfiles, strerror(errno), (int) server.maxclients);
             } else {
@@ -1151,7 +1269,6 @@ void initServer() {
         server.maxmemory_policy = REDIS_MAXMEMORY_NO_EVICTION;
     }
 
-    if (server.cluster_enabled) clusterInit();
     scriptingInit();
     slowlogInit();
     bioInit();
@@ -1374,29 +1491,6 @@ int processCommand(redisClient *c) {
         return REDIS_OK;
     }
 
-    /* If cluster is enabled, redirect here */
-    if (server.cluster_enabled &&
-                !(c->cmd->getkeys_proc == NULL && c->cmd->firstkey == 0)) {
-        int hashslot;
-
-        if (server.cluster.state != REDIS_CLUSTER_OK) {
-            addReplyError(c,"The cluster is down. Check with CLUSTER INFO for more information");
-            return REDIS_OK;
-        } else {
-            int ask;
-            clusterNode *n = getNodeByQuery(c,c->cmd,c->argv,c->argc,&hashslot,&ask);
-            if (n == NULL) {
-                addReplyError(c,"Multi keys request invalid in cluster");
-                return REDIS_OK;
-            } else if (n != server.cluster.myself) {
-                addReplySds(c,sdscatprintf(sdsempty(),
-                    "-%s %d %s:%d\r\n", ask ? "ASK" : "MOVED",
-                    hashslot,n->ip,n->port));
-                return REDIS_OK;
-            }
-        }
-    }
-
     /* Handle the maxmemory directive.
      *
      * First we try to free some memory if possible (if there are volatile
@@ -1405,8 +1499,7 @@ int processCommand(redisClient *c) {
     if (server.maxmemory) {
         int retval = freeMemoryIfNeeded();
         if ((c->cmd->flags & REDIS_CMD_DENYOOM) && retval == REDIS_ERR) {
-            addReplyError(c,
-                "command not allowed when used memory > 'maxmemory'");
+            addReply(c, shared.oomerr);
             return REDIS_OK;
         }
     }
@@ -1421,6 +1514,16 @@ int processCommand(redisClient *c) {
         return REDIS_OK;
     }
 
+    /* Don't accept wirte commands if this is a read only slave. But
+     * accept write commands if this is our master. */
+    if (server.masterhost && server.repl_slave_ro &&
+        !(c->flags & REDIS_MASTER) &&
+        c->cmd->flags & REDIS_CMD_WRITE)
+    {
+        addReply(c, shared.roslaveerr);
+        return REDIS_OK;
+    }
+
     /* Only allow SUBSCRIBE and UNSUBSCRIBE in the context of Pub/Sub */
     if ((dictSize(c->pubsub_channels) > 0 || listLength(c->pubsub_patterns) > 0)
         &&
@@ -1451,7 +1554,7 @@ int processCommand(redisClient *c) {
 
     /* Lua script too slow? Only allow SHUTDOWN NOSAVE and SCRIPT KILL. */
     if (server.lua_timedout &&
-        !(c->cmd->proc != shutdownCommand &&
+        !(c->cmd->proc == shutdownCommand &&
           c->argc == 2 &&
           tolower(((char*)c->argv[1]->ptr)[0]) == 'n') &&
         !(c->cmd->proc == scriptCommand &&
@@ -1590,7 +1693,7 @@ void bytesToHuman(char *s, unsigned long long n) {
  * on memory corruption problems. */
 sds genRedisInfoString(char *section) {
     sds info = sdsempty();
-    time_t uptime = time(NULL)-server.stat_starttime;
+    time_t uptime = server.unixtime-server.stat_starttime;
     int j, numcommands;
     struct rusage self_ru, c_ru;
     unsigned long lol, bib;
@@ -1608,12 +1711,16 @@ sds genRedisInfoString(char *section) {
 
     /* Server */
     if (allsections || defsections || !strcasecmp(section,"server")) {
+        struct utsname name;
+
         if (sections++) info = sdscat(info,"\r\n");
+        uname(&name);
         info = sdscatprintf(info,
             "# Server\r\n"
             "redis_version:%s\r\n"
             "redis_git_sha1:%s\r\n"
             "redis_git_dirty:%d\r\n"
+            "os:%s %s %s\r\n"
             "arch_bits:%d\r\n"
             "multiplexing_api:%s\r\n"
             "gcc_version:%d.%d.%d\r\n"
@@ -1626,6 +1733,7 @@ sds genRedisInfoString(char *section) {
             REDIS_VERSION,
             redisGitSHA1(),
             strtol(redisGitDirty(),NULL,10) > 0,
+            name.sysname, name.release, name.machine,
             server.arch_bits,
             aeGetApiName(),
 #ifdef __GNUC__
@@ -1695,14 +1803,16 @@ sds genRedisInfoString(char *section) {
             "bgsave_in_progress:%d\r\n"
             "last_save_time:%ld\r\n"
             "last_bgsave_status:%s\r\n"
-            "bgrewriteaof_in_progress:%d\r\n",
+            "bgrewriteaof_in_progress:%d\r\n"
+            "bgrewriteaof_scheduled:%d\r\n",
             server.loading,
             server.aof_state != REDIS_AOF_OFF,
             server.dirty,
             server.rdb_child_pid != -1,
             server.lastsave,
             server.lastbgsave_status == REDIS_OK ? "ok" : "err",
-            server.aof_child_pid != -1);
+            server.aof_child_pid != -1,
+            server.aof_rewrite_scheduled);
 
         if (server.aof_state != REDIS_AOF_OFF) {
             info = sdscatprintf(info,
@@ -1710,12 +1820,14 @@ sds genRedisInfoString(char *section) {
                 "aof_base_size:%lld\r\n"
                 "aof_pending_rewrite:%d\r\n"
                 "aof_buffer_length:%zu\r\n"
-                "aof_pending_bio_fsync:%llu\r\n",
+                "aof_pending_bio_fsync:%llu\r\n"
+                "aof_delayed_fsync:%lu\r\n",
                 (long long) server.aof_current_size,
                 (long long) server.aof_rewrite_base_size,
                 server.aof_rewrite_scheduled,
                 sdslen(server.aof_buf),
-                bioPendingJobsOfType(REDIS_BIO_AOF_FSYNC));
+                bioPendingJobsOfType(REDIS_BIO_AOF_FSYNC),
+                server.aof_delayed_fsync);
         }
 
         if (server.loading) {
@@ -1727,7 +1839,7 @@ sds genRedisInfoString(char *section) {
             perc = ((double)server.loading_loaded_bytes /
                    server.loading_total_bytes) * 100;
 
-            elapsed = time(NULL)-server.loading_start_time;
+            elapsed = server.unixtime-server.loading_start_time;
             if (elapsed == 0) {
                 eta = 1; /* A fake 1 second figure if we don't have
                             enough info */
@@ -1798,7 +1910,7 @@ sds genRedisInfoString(char *section) {
                 (server.repl_state == REDIS_REPL_CONNECTED) ?
                     "up" : "down",
                 server.master ?
-                ((int)(time(NULL)-server.master->lastinteraction)) : -1,
+                ((int)(server.unixtime-server.master->lastinteraction)) : -1,
                 server.repl_state == REDIS_REPL_TRANSFER
             );
 
@@ -1807,14 +1919,14 @@ sds genRedisInfoString(char *section) {
                     "master_sync_left_bytes:%ld\r\n"
                     "master_sync_last_io_seconds_ago:%d\r\n"
                     ,(long)server.repl_transfer_left,
-                    (int)(time(NULL)-server.repl_transfer_lastio)
+                    (int)(server.unixtime-server.repl_transfer_lastio)
                 );
             }
 
             if (server.repl_state != REDIS_REPL_CONNECTED) {
                 info = sdscatprintf(info,
                     "master_link_down_since_seconds:%ld\r\n",
-                    (long)time(NULL)-server.repl_down_since);
+                    (long)server.unixtime-server.repl_down_since);
             }
         }
         info = sdscatprintf(info,
@@ -1884,15 +1996,6 @@ sds genRedisInfoString(char *section) {
         }
     }
 
-    /* Clusetr */
-    if (allsections || defsections || !strcasecmp(section,"cluster")) {
-        if (sections++) info = sdscat(info,"\r\n");
-        info = sdscatprintf(info,
-        "# Cluster\r\n"
-        "cluster_enabled:%d\r\n",
-        server.cluster_enabled);
-    }
-
     /* Key space */
     if (allsections || defsections || !strcasecmp(section,"keyspace")) {
         if (sections++) info = sdscat(info,"\r\n");
@@ -2144,8 +2247,12 @@ void daemonize(void) {
 }
 
 void version() {
-    printf("Redis server version %s (%s:%d)\n", REDIS_VERSION,
-        redisGitSHA1(), atoi(redisGitDirty()) > 0);
+    printf("Redis server v=%s sha=%s:%d malloc=%s bits=%d\n",
+        REDIS_VERSION,
+        redisGitSHA1(),
+        atoi(redisGitDirty()) > 0,
+        ZMALLOC_LIB,
+        sizeof(long) == 4 ? 32 : 64);
     exit(0);
 }
 
@@ -2153,7 +2260,8 @@ void usage() {
     fprintf(stderr,"Usage: ./redis-server [/path/to/redis.conf] [options]\n");
     fprintf(stderr,"       ./redis-server - (read config from stdin)\n");
     fprintf(stderr,"       ./redis-server -v or --version\n");
-    fprintf(stderr,"       ./redis-server -h or --help\n\n");
+    fprintf(stderr,"       ./redis-server -h or --help\n");
+    fprintf(stderr,"       ./redis-server --test-memory <megabytes>\n\n");
     fprintf(stderr,"Examples:\n");
     fprintf(stderr,"       ./redis-server (run the server with default conf)\n");
     fprintf(stderr,"       ./redis-server /etc/redis/6379.conf\n");
@@ -2172,7 +2280,7 @@ void redisAsciiArt(void) {
         redisGitSHA1(),
         strtol(redisGitDirty(),NULL,10) > 0,
         (sizeof(long) == 8) ? "64" : "32",
-        server.cluster_enabled ? "cluster" : "stand alone",
+        "stand alone",
         server.port,
         (long) getpid()
     );
@@ -2183,7 +2291,7 @@ void redisAsciiArt(void) {
 static void sigtermHandler(int sig) {
     REDIS_NOTUSED(sig);
 
-    redisLog(REDIS_WARNING,"Received SIGTERM, scheduling shutdown...");
+    redisLogFromHandler(REDIS_WARNING,"Received SIGTERM, scheduling shutdown...");
     server.shutdown_asap = 1;
 }
 
@@ -2193,13 +2301,13 @@ void setupSignalHandlers(void) {
     /* When the SA_SIGINFO flag is set in sa_flags then sa_sigaction is used.
      * Otherwise, sa_handler is used. */
     sigemptyset(&act.sa_mask);
-    act.sa_flags = SA_NODEFER | SA_ONSTACK | SA_RESETHAND;
+    act.sa_flags = 0;
     act.sa_handler = sigtermHandler;
     sigaction(SIGTERM, &act, NULL);
 
 #ifdef HAVE_BACKTRACE
     sigemptyset(&act.sa_mask);
-    act.sa_flags = SA_NODEFER | SA_ONSTACK | SA_RESETHAND | SA_SIGINFO;
+    act.sa_flags = SA_NODEFER | SA_RESETHAND | SA_SIGINFO;
     act.sa_sigaction = sigsegvHandler;
     sigaction(SIGSEGV, &act, NULL);
     sigaction(SIGBUS, &act, NULL);
@@ -2209,6 +2317,8 @@ void setupSignalHandlers(void) {
     return;
 }
 
+void memtest(size_t megabytes, int passes);
+
 int main(int argc, char **argv) {
     long long start;
     struct timeval tv;
@@ -2230,6 +2340,17 @@ int main(int argc, char **argv) {
             strcmp(argv[1], "--version") == 0) version();
         if (strcmp(argv[1], "--help") == 0 ||
             strcmp(argv[1], "-h") == 0) usage();
+        if (strcmp(argv[1], "--test-memory") == 0) {
+            if (argc == 3) {
+                memtest(atoi(argv[2]),50);
+                exit(0);
+            } else {
+                fprintf(stderr,"Please specify the amount of memory to test in megabytes.\n");
+                fprintf(stderr,"Example: ./redis-server --test-memory 4096\n\n");
+                exit(1);
+            }
+        }
+
         /* First argument is the config file name? */
         if (argv[j][0] != '-' || argv[j][1] != '-')
             configfile = argv[j++];

src/redis.h

@@ -44,6 +44,7 @@
 #define REDIS_CONFIGLINE_MAX    1024
 #define REDIS_EXPIRELOOKUPS_PER_CRON    10 /* lookup 10 expires per loop */
 #define REDIS_MAX_WRITE_PER_EVENT (1024*64)
+#define REDIS_SHARED_SELECT_CMDS 10
 #define REDIS_SHARED_INTEGERS 10000
 #define REDIS_SHARED_BULKHDR_LEN 32
 #define REDIS_MAX_LOGMSG_LEN    1024 /* Default maximum length of syslog messages */
@@ -51,7 +52,7 @@
 #define REDIS_AOF_REWRITE_MIN_SIZE (1024*1024)
 #define REDIS_AOF_REWRITE_ITEMS_PER_CMD 64
 #define REDIS_SLOWLOG_LOG_SLOWER_THAN 10000
-#define REDIS_SLOWLOG_MAX_LEN 64
+#define REDIS_SLOWLOG_MAX_LEN 128
 #define REDIS_MAX_CLIENTS 10000
 
 #define REDIS_REPL_TIMEOUT 60
@@ -88,7 +89,6 @@
 #define REDIS_SET 2
 #define REDIS_ZSET 3
 #define REDIS_HASH 4
-#define REDIS_VMPOINTER 8
 
 /* Objects encoding. Some kind of objects like Strings and Hashes can be
  * internally represented in multiple ways. The 'encoding' field of the object
@@ -323,6 +323,7 @@ typedef struct redisClient {
     redisDb *db;
     int dictid;
     sds querybuf;
+    size_t querybuf_peak;   /* Recent (100ms or more) peak of querybuf size */
     int argc;
     robj **argv;
     struct redisCommand *cmd, *lastcmd;
@@ -332,6 +333,7 @@ typedef struct redisClient {
     list *reply;
     unsigned long reply_bytes; /* Tot bytes of objects in reply list */
     int sentlen;
+    time_t ctime;           /* Client creation time */
     time_t lastinteraction; /* time of the last interaction, used for timeout */
     time_t obuf_soft_limit_reached_time;
     int flags;              /* REDIS_SLAVE | REDIS_MONITOR | REDIS_MULTI ... */
@@ -364,10 +366,9 @@ struct sharedObjectsStruct {
     *colon, *nullbulk, *nullmultibulk, *queued,
     *emptymultibulk, *wrongtypeerr, *nokeyerr, *syntaxerr, *sameobjecterr,
     *outofrangeerr, *noscripterr, *loadingerr, *slowscripterr, *bgsaveerr,
-    *plus, *select0, *select1, *select2, *select3, *select4,
-    *select5, *select6, *select7, *select8, *select9,
-    *messagebulk, *pmessagebulk, *subscribebulk, *unsubscribebulk,
-    *psubscribebulk, *punsubscribebulk, *del, *rpop, *lpop,
+    *roslaveerr, *oomerr, *plus, *messagebulk, *pmessagebulk, *subscribebulk,
+    *unsubscribebulk, *psubscribebulk, *punsubscribebulk, *del, *rpop, *lpop,
+    *select[REDIS_SHARED_SELECT_CMDS],
     *integers[REDIS_SHARED_INTEGERS],
     *mbulkhdr[REDIS_SHARED_BULKHDR_LEN], /* "*<value>\r\n" */
     *bulkhdr[REDIS_SHARED_BULKHDR_LEN];  /* "$<value>\r\n" */
@@ -425,134 +426,6 @@ typedef struct redisOpArray {
     int numops;
 } redisOpArray;
 
-/*-----------------------------------------------------------------------------
- * Redis cluster data structures
- *----------------------------------------------------------------------------*/
-
-#define REDIS_CLUSTER_SLOTS 4096
-#define REDIS_CLUSTER_OK 0          /* Everything looks ok */
-#define REDIS_CLUSTER_FAIL 1        /* The cluster can't work */
-#define REDIS_CLUSTER_NEEDHELP 2    /* The cluster works, but needs some help */
-#define REDIS_CLUSTER_NAMELEN 40    /* sha1 hex length */
-#define REDIS_CLUSTER_PORT_INCR 10000 /* Cluster port = baseport + PORT_INCR */
-
-struct clusterNode;
-
-/* clusterLink encapsulates everything needed to talk with a remote node. */
-typedef struct clusterLink {
-    int fd;                     /* TCP socket file descriptor */
-    sds sndbuf;                 /* Packet send buffer */
-    sds rcvbuf;                 /* Packet reception buffer */
-    struct clusterNode *node;   /* Node related to this link if any, or NULL */
-} clusterLink;
-
-/* Node flags */
-#define REDIS_NODE_MASTER 1     /* The node is a master */
-#define REDIS_NODE_SLAVE 2      /* The node is a slave */
-#define REDIS_NODE_PFAIL 4      /* Failure? Need acknowledge */
-#define REDIS_NODE_FAIL 8       /* The node is believed to be malfunctioning */
-#define REDIS_NODE_MYSELF 16    /* This node is myself */
-#define REDIS_NODE_HANDSHAKE 32 /* We have still to exchange the first ping */
-#define REDIS_NODE_NOADDR   64  /* We don't know the address of this node */
-#define REDIS_NODE_MEET 128     /* Send a MEET message to this node */
-#define REDIS_NODE_NULL_NAME "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"
-
-struct clusterNode {
-    char name[REDIS_CLUSTER_NAMELEN]; /* Node name, hex string, sha1-size */
-    int flags;      /* REDIS_NODE_... */
-    unsigned char slots[REDIS_CLUSTER_SLOTS/8]; /* slots handled by this node */
-    int numslaves;  /* Number of slave nodes, if this is a master */
-    struct clusterNode **slaves; /* pointers to slave nodes */
-    struct clusterNode *slaveof; /* pointer to the master node */
-    time_t ping_sent;       /* Unix time we sent latest ping */
-    time_t pong_received;   /* Unix time we received the pong */
-    char *configdigest;         /* Configuration digest of this node */
-    time_t configdigest_ts;     /* Configuration digest timestamp */
-    char ip[16];                /* Latest known IP address of this node */
-    int port;                   /* Latest known port of this node */
-    clusterLink *link;          /* TCP/IP link with this node */
-};
-typedef struct clusterNode clusterNode;
-
-typedef struct {
-    char *configfile;
-    clusterNode *myself;  /* This node */
-    int state;            /* REDIS_CLUSTER_OK, REDIS_CLUSTER_FAIL, ... */
-    int node_timeout;
-    dict *nodes;          /* Hash table of name -> clusterNode structures */
-    clusterNode *migrating_slots_to[REDIS_CLUSTER_SLOTS];
-    clusterNode *importing_slots_from[REDIS_CLUSTER_SLOTS];
-    clusterNode *slots[REDIS_CLUSTER_SLOTS];
-    zskiplist *slots_to_keys;
-} clusterState;
-
-/* Redis cluster messages header */
-
-/* Note that the PING, PONG and MEET messages are actually the same exact
- * kind of packet. PONG is the reply to ping, in the extact format as a PING,
- * while MEET is a special PING that forces the receiver to add the sender
- * as a node (if it is not already in the list). */
-#define CLUSTERMSG_TYPE_PING 0          /* Ping */
-#define CLUSTERMSG_TYPE_PONG 1          /* Pong (reply to Ping) */
-#define CLUSTERMSG_TYPE_MEET 2          /* Meet "let's join" message */
-#define CLUSTERMSG_TYPE_FAIL 3          /* Mark node xxx as failing */
-#define CLUSTERMSG_TYPE_PUBLISH 4       /* Pub/Sub Publish propatagion */
-
-/* Initially we don't know our "name", but we'll find it once we connect
- * to the first node, using the getsockname() function. Then we'll use this
- * address for all the next messages. */
-typedef struct {
-    char nodename[REDIS_CLUSTER_NAMELEN];
-    uint32_t ping_sent;
-    uint32_t pong_received;
-    char ip[16];    /* IP address last time it was seen */
-    uint16_t port;  /* port last time it was seen */
-    uint16_t flags;
-    uint32_t notused; /* for 64 bit alignment */
-} clusterMsgDataGossip;
-
-typedef struct {
-    char nodename[REDIS_CLUSTER_NAMELEN];
-} clusterMsgDataFail;
-
-typedef struct {
-    uint32_t channel_len;
-    uint32_t message_len;
-    unsigned char bulk_data[8]; /* defined as 8 just for alignment concerns. */
-} clusterMsgDataPublish;
-
-union clusterMsgData {
-    /* PING, MEET and PONG */
-    struct {
-        /* Array of N clusterMsgDataGossip structures */
-        clusterMsgDataGossip gossip[1];
-    } ping;
-
-    /* FAIL */
-    struct {
-        clusterMsgDataFail about;
-    } fail;
-
-    /* PUBLISH */
-    struct {
-        clusterMsgDataPublish msg;
-    } publish;
-};
-
-typedef struct {
-    uint32_t totlen;    /* Total length of this message */
-    uint16_t type;      /* Message type */
-    uint16_t count;     /* Only used for some kind of messages. */
-    char sender[REDIS_CLUSTER_NAMELEN]; /* Name of the sender node */
-    unsigned char myslots[REDIS_CLUSTER_SLOTS/8];
-    char slaveof[REDIS_CLUSTER_NAMELEN];
-    char configdigest[32];
-    uint16_t port;      /* Sender TCP base port */
-    unsigned char state; /* Cluster state from the POV of the sender */
-    unsigned char notused[5]; /* Reserved for future use. For alignment. */
-    union clusterMsgData data;
-} clusterMsg;
-
 /*-----------------------------------------------------------------------------
  * Global server state
  *----------------------------------------------------------------------------*/
@@ -578,7 +451,6 @@ struct redisServer {
     mode_t unixsocketperm;      /* UNIX socket permission */
     int ipfd;                   /* TCP socket file descriptor */
     int sofd;                   /* Unix socket file descriptor */
-    int cfd;                    /* Cluster bus lisetning socket */
     list *clients;              /* List of active clients */
     list *clients_to_close;     /* Clients to close asynchronously */
     list *slaves, *monitors;    /* List of slaves and MONITORs */
@@ -636,6 +508,7 @@ struct redisServer {
     int aof_selected_db; /* Currently selected DB in AOF */
     time_t aof_flush_postponed_start; /* UNIX time of postponed AOF flush */
     time_t aof_last_fsync;            /* UNIX time of last fsync() */
+    unsigned long aof_delayed_fsync;  /* delayed AOF fsync() counter */
     /* RDB persistence */
     long long dirty;                /* Changes to DB from the last save */
     long long dirty_before_bgsave;  /* Used to restore dirty on failed BGSAVE */
@@ -644,6 +517,7 @@ struct redisServer {
     int saveparamslen;              /* Number of saving points */
     char *rdb_filename;             /* Name of RDB file */
     int rdb_compression;            /* Use compression in RDB? */
+    int rdb_checksum;               /* Use RDB checksum? */
     time_t lastsave;                /* Unix time of last save succeeede */
     int lastbgsave_status;          /* REDIS_OK or REDIS_ERR */
     int stop_writes_on_bgsave_err;  /* Don't allow writes if can't BGSAVE */
@@ -669,6 +543,7 @@ struct redisServer {
     char *repl_transfer_tmpfile; /* Slave-> master SYNC temp file name */
     time_t repl_transfer_lastio; /* Unix time of the latest read, for timeout */
     int repl_serve_stale_data; /* Serve stale data when link is down? */
+    int repl_slave_ro;          /* Slave is read only? */
     time_t repl_down_since; /* Unix time at which link with master went down */
     /* Limits */
     unsigned int maxclients;        /* Max number of simultaneous clients */
@@ -696,9 +571,6 @@ struct redisServer {
     /* Pubsub */
     dict *pubsub_channels;  /* Map channels to list of subscribed clients */
     list *pubsub_patterns;  /* A list of pubsub_patterns */
-    /* Cluster */
-    int cluster_enabled;    /* Is cluster enabled? */
-    clusterState cluster;   /* State of the cluster */
     /* Scripting */
     lua_State *lua; /* The Lua interpreter. We use just one for all clients */
     redisClient *lua_client;   /* The "fake client" to query Redis from Lua */
@@ -718,6 +590,7 @@ struct redisServer {
     char *assert_file;
     int assert_line;
     int bug_report_start; /* True if bug report header was already logged. */
+    int watchdog_period;  /* Software watchdog period in ms. 0 = off */
 };
 
 typedef struct pubsubPattern {
@@ -733,8 +606,7 @@ struct redisCommand {
     int arity;
     char *sflags; /* Flags as string represenation, one char per flag. */
     int flags;    /* The actual flags, obtained from the 'sflags' field. */
-    /* Use a function to determine keys arguments in a command line.
-     * Used for Redis Cluster redirect. */
+    /* Use a function to determine keys arguments in a command line. */
     redisGetKeysProc *getkeys_proc;
     /* What keys should be loaded in background when calling this command? */
     int firstkey; /* The first argument that's a key (0 = no keys) */
@@ -810,10 +682,9 @@ extern struct redisServer server;
 extern struct sharedObjectsStruct shared;
 extern dictType setDictType;
 extern dictType zsetDictType;
-extern dictType clusterNodesDictType;
 extern dictType dbDictType;
 extern double R_Zero, R_PosInf, R_NegInf, R_Nan;
-dictType hashDictType;
+extern dictType hashDictType;
 
 /*-----------------------------------------------------------------------------
  * Functions prototypes
@@ -823,6 +694,8 @@ dictType hashDictType;
 long long ustime(void);
 long long mstime(void);
 void getRandomHexChars(char *p, unsigned int len);
+uint64_t crc64(uint64_t crc, const unsigned char *s, uint64_t l);
+void exitFromChild(int retcode);
 
 /* networking.c -- Networking and Client related operations */
 redisClient *createClient(int fd);
@@ -864,6 +737,7 @@ void asyncCloseClientOnOutputBufferLimitReached(redisClient *c);
 int getClientLimitClassByName(char *name);
 char *getClientLimitClassName(int class);
 void flushSlavesOutputBuffers(void);
+void disconnectSlaves(void);
 
 #ifdef __GNUC__
 void addReplyErrorFormat(redisClient *c, const char *fmt, ...)
@@ -899,6 +773,7 @@ void freeClientMultiState(redisClient *c);
 void queueMultiCommand(redisClient *c);
 void touchWatchedKey(redisDb *db, robj *key);
 void touchWatchedKeysOnFlush(int dbid);
+void discardTransaction(redisClient *c);
 
 /* Redis object implementation */
 void decrRefCount(void *o);
@@ -938,9 +813,9 @@ int equalStringObjects(robj *a, robj *b);
 unsigned long estimateObjectIdleTime(robj *o);
 
 /* Synchronous I/O with timeout */
-int syncWrite(int fd, char *ptr, ssize_t size, int timeout);
-int syncRead(int fd, char *ptr, ssize_t size, int timeout);
-int syncReadLine(int fd, char *ptr, ssize_t size, int timeout);
+ssize_t syncWrite(int fd, char *ptr, ssize_t size, long long timeout);
+ssize_t syncRead(int fd, char *ptr, ssize_t size, long long timeout);
+ssize_t syncReadLine(int fd, char *ptr, ssize_t size, long long timeout);
 
 /* Replication */
 void replicationFeedSlaves(list *slaves, int dictid, robj **argv, int argc);
@@ -998,6 +873,7 @@ void alsoPropagate(struct redisCommand *cmd, int dbid, robj **argv, int argc, in
 int prepareForShutdown();
 void redisLog(int level, const char *fmt, ...);
 void redisLogRaw(int level, const char *msg);
+void redisLogFromHandler(int level, const char *msg);
 void usage();
 void updateDictResizePolicy(void);
 int htNeedsResize(dict *dict);
@@ -1082,16 +958,6 @@ int *noPreloadGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *numke
 int *renameGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *numkeys, int flags);
 int *zunionInterGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *numkeys, int flags);
 
-/* Cluster */
-void clusterInit(void);
-unsigned short crc16(const char *buf, int len);
-unsigned int keyHashSlot(char *key, int keylen);
-clusterNode *createClusterNode(char *nodename, int flags);
-int clusterAddNode(clusterNode *node);
-void clusterCron(void);
-clusterNode *getNodeByQuery(redisClient *c, struct redisCommand *cmd, robj **argv, int argc, int *hashslot, int *ask);
-void clusterPropagatePublish(robj *channel, robj *message);
-
 /* Scripting */
 void scriptingInit(void);
 
@@ -1223,10 +1089,8 @@ void punsubscribeCommand(redisClient *c);
 void publishCommand(redisClient *c);
 void watchCommand(redisClient *c);
 void unwatchCommand(redisClient *c);
-void clusterCommand(redisClient *c);
 void restoreCommand(redisClient *c);
 void migrateCommand(redisClient *c);
-void askingCommand(redisClient *c);
 void dumpCommand(redisClient *c);
 void objectCommand(redisClient *c);
 void clientCommand(redisClient *c);
@@ -1250,4 +1114,7 @@ void bugReportStart(void);
 void redisLogObjectDebugInfo(robj *o);
 void sigsegvHandler(int sig, siginfo_t *info, void *secret);
 sds genRedisInfoString(char *section);
+void enableWatchdog(int period);
+void disableWatchdog(void);
+void watchdogScheduleSignal(int period);
 #endif

src/replication.c

@@ -25,24 +25,15 @@ void replicationFeedSlaves(list *slaves, int dictid, robj **argv, int argc) {
         if (slave->slaveseldb != dictid) {
             robj *selectcmd;
 
-            switch(dictid) {
-            case 0: selectcmd = shared.select0; break;
-            case 1: selectcmd = shared.select1; break;
-            case 2: selectcmd = shared.select2; break;
-            case 3: selectcmd = shared.select3; break;
-            case 4: selectcmd = shared.select4; break;
-            case 5: selectcmd = shared.select5; break;
-            case 6: selectcmd = shared.select6; break;
-            case 7: selectcmd = shared.select7; break;
-            case 8: selectcmd = shared.select8; break;
-            case 9: selectcmd = shared.select9; break;
-            default:
+            if (dictid >= 0 && dictid < REDIS_SHARED_SELECT_CMDS) {
+                selectcmd = shared.select[dictid];
+                incrRefCount(selectcmd);
+            } else {
                 selectcmd = createObject(REDIS_STRING,
                     sdscatprintf(sdsempty(),"select %d\r\n",dictid));
-                selectcmd->refcount = 0;
-                break;
             }
             addReply(slave,selectcmd);
+            decrRefCount(selectcmd);
             slave->slaveseldb = dictid;
         }
         addReplyMultiBulkLen(slave,argc);
@@ -291,7 +282,7 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
     /* If repl_transfer_left == -1 we still have to read the bulk length
      * from the master reply. */
     if (server.repl_transfer_left == -1) {
-        if (syncReadLine(fd,buf,1024,server.repl_syncio_timeout) == -1) {
+        if (syncReadLine(fd,buf,1024,server.repl_syncio_timeout*1000) == -1) {
             redisLog(REDIS_WARNING,
                 "I/O error reading bulk count from MASTER: %s",
                 strerror(errno));
@@ -307,7 +298,7 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
             /* At this stage just a newline works as a PING in order to take
              * the connection live. So we refresh our last interaction
              * timestamp. */
-            server.repl_transfer_lastio = time(NULL);
+            server.repl_transfer_lastio = server.unixtime;
             return;
         } else if (buf[0] != '$') {
             redisLog(REDIS_WARNING,"Bad protocol from MASTER, the first byte is not '$', are you sure the host and port are right?");
@@ -330,9 +321,9 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
         replicationAbortSyncTransfer();
         return;
     }
-    server.repl_transfer_lastio = time(NULL);
+    server.repl_transfer_lastio = server.unixtime;
     if (write(server.repl_transfer_fd,buf,nread) != nread) {
-        redisLog(REDIS_WARNING,"Write error or short write writing to the DB dump file needed for MASTER <-> SLAVE synchrnonization: %s", strerror(errno));
+        redisLog(REDIS_WARNING,"Write error or short write writing to the DB dump file needed for MASTER <-> SLAVE synchronization: %s", strerror(errno));
         goto error;
     }
     server.repl_transfer_left -= nread;
@@ -414,13 +405,13 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
         size_t authlen;
 
         authlen = snprintf(authcmd,sizeof(authcmd),"AUTH %s\r\n",server.masterauth);
-        if (syncWrite(fd,authcmd,authlen,server.repl_syncio_timeout) == -1) {
+        if (syncWrite(fd,authcmd,authlen,server.repl_syncio_timeout*1000) == -1) {
             redisLog(REDIS_WARNING,"Unable to AUTH to MASTER: %s",
                 strerror(errno));
             goto error;
         }
         /* Read the AUTH result.  */
-        if (syncReadLine(fd,buf,1024,server.repl_syncio_timeout) == -1) {
+        if (syncReadLine(fd,buf,1024,server.repl_syncio_timeout*1000) == -1) {
             redisLog(REDIS_WARNING,"I/O error reading auth result from MASTER: %s",
                 strerror(errno));
             goto error;
@@ -432,7 +423,7 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
     }
 
     /* Issue the SYNC command */
-    if (syncWrite(fd,"SYNC \r\n",7,server.repl_syncio_timeout) == -1) {
+    if (syncWrite(fd,"SYNC \r\n",7,server.repl_syncio_timeout*1000) == -1) {
         redisLog(REDIS_WARNING,"I/O error writing to MASTER: %s",
             strerror(errno));
         goto error;
@@ -441,7 +432,7 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
     /* Prepare a suitable temp file for bulk transfer */
     while(maxtries--) {
         snprintf(tmpfile,256,
-            "temp-%d.%ld.rdb",(int)time(NULL),(long int)getpid());
+            "temp-%d.%ld.rdb",(int)server.unixtime,(long int)getpid());
         dfd = open(tmpfile,O_CREAT|O_WRONLY|O_EXCL,0644);
         if (dfd != -1) break;
         sleep(1);
@@ -462,7 +453,7 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
     server.repl_state = REDIS_REPL_TRANSFER;
     server.repl_transfer_left = -1;
     server.repl_transfer_fd = dfd;
-    server.repl_transfer_lastio = time(NULL);
+    server.repl_transfer_lastio = server.unixtime;
     server.repl_transfer_tmpfile = zstrdup(tmpfile);
     return;
 
@@ -490,7 +481,7 @@ int connectWithMaster(void) {
         return REDIS_ERR;
     }
 
-    server.repl_transfer_lastio = time(NULL);
+    server.repl_transfer_lastio = server.unixtime;
     server.repl_transfer_s = fd;
     server.repl_state = REDIS_REPL_CONNECTING;
     return REDIS_OK;
@@ -541,6 +532,7 @@ void slaveofCommand(redisClient *c) {
         server.masterhost = sdsdup(c->argv[1]->ptr);
         server.masterport = port;
         if (server.master) freeClient(server.master);
+        disconnectSlaves(); /* Force our slaves to resync with us as well. */
         if (server.repl_state == REDIS_REPL_TRANSFER)
             replicationAbortSyncTransfer();
         server.repl_state = REDIS_REPL_CONNECT;

src/rio.c

@@ -1,9 +1,25 @@
+/* rio.c is a simple stream-oriented I/O abstraction that provides an interface
+ * to write code that can consume/produce data using different concrete input
+ * and output devices. For instance the same rdb.c code using the rio abstraction
+ * can be used to read and write the RDB format using in-memory buffers or files.
+ *
+ * A rio object provides the following methods:
+ *  read: read from stream.
+ *  write: write to stream.
+ *  tell: get the current offset.
+ *
+ * It is also possible to set a 'checksum' method that is used by rio.c in order
+ * to compute a checksum of the data written or read, or to query the rio object
+ * for the current checksum. */
+
 #include "fmacros.h"
 #include <string.h>
 #include <stdio.h>
 #include "rio.h"
 #include "util.h"
 
+uint64_t crc64(uint64_t crc, const unsigned char *s, uint64_t l);
+
 /* Returns 1 or 0 for success/failure. */
 static size_t rioBufferWrite(rio *r, const void *buf, size_t len) {
     r->io.buffer.ptr = sdscatlen(r->io.buffer.ptr,(char*)buf,len);
@@ -44,6 +60,8 @@ static const rio rioBufferIO = {
     rioBufferRead,
     rioBufferWrite,
     rioBufferTell,
+    NULL,           /* update_checksum */
+    0,              /* current checksum */
     { { NULL, 0 } } /* union for io-specific vars */
 };
 
@@ -51,6 +69,8 @@ static const rio rioFileIO = {
     rioFileRead,
     rioFileWrite,
     rioFileTell,
+    NULL,           /* update_checksum */
+    0,              /* current checksum */
     { { NULL, 0 } } /* union for io-specific vars */
 };
 
@@ -65,6 +85,16 @@ void rioInitWithBuffer(rio *r, sds s) {
     r->io.buffer.pos = 0;
 }
 
+/* This function can be installed both in memory and file streams when checksum
+ * computation is needed. */
+void rioGenericUpdateChecksum(rio *r, const void *buf, size_t len) {
+    r->cksum = crc64(r->cksum,buf,len);
+}
+
+/* ------------------------------ Higher level interface ---------------------------
+ * The following higher level functions use lower level rio.c functions to help
+ * generating the Redis protocol for the Append Only File. */
+
 /* Write multi bulk count in the format: "*<count>\r\n". */
 size_t rioWriteBulkCount(rio *r, char prefix, int count) {
     char cbuf[128];

src/rio.h

@@ -2,6 +2,7 @@
 #define __REDIS_RIO_H
 
 #include <stdio.h>
+#include <stdint.h>
 #include "sds.h"
 
 struct _rio {
@@ -11,6 +12,14 @@ struct _rio {
     size_t (*read)(struct _rio *, void *buf, size_t len);
     size_t (*write)(struct _rio *, const void *buf, size_t len);
     off_t (*tell)(struct _rio *);
+    /* The update_cksum method if not NULL is used to compute the checksum of all the
+     * data that was read or written so far. The method should be designed so that
+     * can be called with the current checksum, and the buf and len fields pointing
+     * to the new block of data to add to the checksum computation. */
+    void (*update_cksum)(struct _rio *, const void *buf, size_t len);
+
+    /* The current checksum */
+    uint64_t cksum;
 
     /* Backend-specific vars. */
     union {
@@ -26,8 +35,26 @@ struct _rio {
 
 typedef struct _rio rio;
 
-#define rioWrite(rio,buf,len) ((rio)->write((rio),(buf),(len)))
-#define rioRead(rio,buf,len) ((rio)->read((rio),(buf),(len)))
+/* The following functions are our interface with the stream. They'll call the
+ * actual implementation of read / write / tell, and will update the checksum
+ * if needed. */
+
+static inline size_t rioWrite(rio *r, const void *buf, size_t len) {
+    if (r->update_cksum) r->update_cksum(r,buf,len);
+    return r->write(r,buf,len);
+}
+
+static inline size_t rioRead(rio *r, void *buf, size_t len) {
+    if (r->read(r,buf,len) == 1) {
+        if (r->update_cksum) r->update_cksum(r,buf,len);
+        return 1;
+    }
+    return 0;
+}
+
+static inline off_t rioTell(rio *r) {
+    return r->tell(r);
+}
 
 void rioInitWithFile(rio *r, FILE *fp);
 void rioInitWithBuffer(rio *r, sds s);
@@ -37,4 +64,6 @@ size_t rioWriteBulkString(rio *r, const char *buf, size_t len);
 size_t rioWriteBulkLongLong(rio *r, long long l);
 size_t rioWriteBulkDouble(rio *r, double d);
 
+void rioGenericUpdateChecksum(rio *r, const void *buf, size_t len);
+
 #endif

src/scripting.c

@@ -15,6 +15,7 @@ char *redisProtocolToLuaType_Error(lua_State *lua, char *reply);
 char *redisProtocolToLuaType_MultiBulk(lua_State *lua, char *reply);
 int redis_math_random (lua_State *L);
 int redis_math_randomseed (lua_State *L);
+void sha1hex(char *digest, char *script, size_t len);
 
 /* Take a Redis reply in the Redis protocol format and convert it into a
  * Lua type. Thanks to this function, and the introduction of not connected
@@ -206,15 +207,45 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
         goto cleanup;
     }
 
+    /* There are commands that are not allowed inside scripts. */
     if (cmd->flags & REDIS_CMD_NOSCRIPT) {
         luaPushError(lua, "This Redis command is not allowed from scripts");
         goto cleanup;
     }
 
-    if (cmd->flags & REDIS_CMD_WRITE && server.lua_random_dirty) {
-        luaPushError(lua,
-            "Write commands not allowed after non deterministic commands");
-        goto cleanup;
+    /* Write commands are forbidden against read-only slaves, or if a
+     * command marked as non-deterministic was already called in the context
+     * of this script. */
+    if (cmd->flags & REDIS_CMD_WRITE) {
+        if (server.lua_random_dirty) {
+            luaPushError(lua,
+                "Write commands not allowed after non deterministic commands");
+            goto cleanup;
+        } else if (server.masterhost && server.repl_slave_ro &&
+                   !(server.lua_caller->flags & REDIS_MASTER))
+        {
+            luaPushError(lua, shared.roslaveerr->ptr);
+            goto cleanup;
+        } else if (server.stop_writes_on_bgsave_err &&
+                   server.saveparamslen > 0 &&
+                   server.lastbgsave_status == REDIS_ERR)
+        {
+            luaPushError(lua, shared.bgsaveerr->ptr);
+            goto cleanup;
+        }
+    }
+
+    /* If we reached the memory limit configured via maxmemory, commands that
+     * could enlarge the memory usage are not allowed, but only if this is the
+     * first write in the context of this script, otherwise we can't stop
+     * in the middle. */
+    if (server.maxmemory && server.lua_write_dirty == 0 &&
+        (cmd->flags & REDIS_CMD_DENYOOM))
+    {
+        if (freeMemoryIfNeeded() == REDIS_ERR) {
+            luaPushError(lua, shared.oomerr->ptr);
+            goto cleanup;
+        }
     }
 
     if (cmd->flags & REDIS_CMD_RANDOM) server.lua_random_dirty = 1;
@@ -276,6 +307,25 @@ int luaRedisPCallCommand(lua_State *lua) {
     return luaRedisGenericCommand(lua,0);
 }
 
+/* This adds redis.sha1hex(string) to Lua scripts using the same hashing
+ * function used for sha1ing lua scripts. */
+int luaRedisSha1hexCommand(lua_State *lua) {
+    int argc = lua_gettop(lua);
+    char digest[41];
+    size_t len;
+    char *s;
+
+    if (argc != 1) {
+        luaPushError(lua, "wrong number of arguments");
+        return 1;
+    }
+
+    s = (char*)lua_tolstring(lua,1,&len);
+    sha1hex(digest,s,len);
+    lua_pushstring(lua,digest);
+    return 1;
+}
+
 int luaLogCommand(lua_State *lua) {
     int j, argc = lua_gettop(lua);
     int level;
@@ -362,13 +412,59 @@ void luaLoadLibraries(lua_State *lua) {
 #endif
 }
 
+/* Remove a functions that we don't want to expose to the Redis scripting
+ * environment. */
+void luaRemoveUnsupportedFunctions(lua_State *lua) {
+    lua_pushnil(lua);
+    lua_setglobal(lua,"loadfile");
+}
+
+/* This function installs metamethods in the global table _G that prevent
+ * the creation of globals accidentally.
+ *
+ * It should be the last to be called in the scripting engine initialization
+ * sequence, because it may interact with creation of globals. */
+void scriptingEnableGlobalsProtection(lua_State *lua) {
+    char *s[32];
+    sds code = sdsempty();
+    int j = 0;
+
+    /* strict.lua from: http://metalua.luaforge.net/src/lib/strict.lua.html.
+     * Modified to be adapted to Redis. */
+    s[j++]="local mt = {}\n";
+    s[j++]="setmetatable(_G, mt)\n";
+    s[j++]="mt.__newindex = function (t, n, v)\n";
+    s[j++]="  if debug.getinfo(2) then\n";
+    s[j++]="    local w = debug.getinfo(2, \"S\").what\n";
+    s[j++]="    if w ~= \"main\" and w ~= \"C\" then\n";
+    s[j++]="      error(\"Script attempted to create global variable '\"..tostring(n)..\"'\", 2)\n";
+    s[j++]="    end\n";
+    s[j++]="  end\n";
+    s[j++]="  rawset(t, n, v)\n";
+    s[j++]="end\n";
+    s[j++]="mt.__index = function (t, n)\n";
+    s[j++]="  if debug.getinfo(2) and debug.getinfo(2, \"S\").what ~= \"C\" then\n";
+    s[j++]="    error(\"Script attempted to access unexisting global variable '\"..tostring(n)..\"'\", 2)\n";
+    s[j++]="  end\n";
+    s[j++]="  return rawget(t, n)\n";
+    s[j++]="end\n";
+    s[j++]=NULL;
+
+    for (j = 0; s[j] != NULL; j++) code = sdscatlen(code,s[j],strlen(s[j]));
+    luaL_loadbuffer(lua,code,sdslen(code),"@enable_strict_lua");
+    lua_pcall(lua,0,0,0);
+    sdsfree(code);
+}
+
 /* Initialize the scripting environment.
  * It is possible to call this function to reset the scripting environment
  * assuming that we call scriptingRelease() before.
  * See scriptingReset() for more information. */
 void scriptingInit(void) {
     lua_State *lua = lua_open();
+
     luaLoadLibraries(lua);
+    luaRemoveUnsupportedFunctions(lua);
 
     /* Initialize a dictionary we use to map SHAs to scripts.
      * This is useful for replication, as we need to replicate EVALSHA
@@ -409,6 +505,11 @@ void scriptingInit(void) {
     lua_pushnumber(lua,REDIS_WARNING);
     lua_settable(lua,-3);
 
+    /* redis.sha1hex */
+    lua_pushstring(lua, "sha1hex");
+    lua_pushcfunction(lua, luaRedisSha1hexCommand);
+    lua_settable(lua, -3);
+
     /* Finally set the table as 'redis' global var. */
     lua_setglobal(lua,"redis");
 
@@ -433,7 +534,7 @@ void scriptingInit(void) {
                                 "  if b == false then b = '' end\n"
                                 "  return a<b\n"
                                 "end\n";
-        luaL_loadbuffer(lua,compare_func,strlen(compare_func),"cmp_func_def");
+        luaL_loadbuffer(lua,compare_func,strlen(compare_func),"@cmp_func_def");
         lua_pcall(lua,0,0,0);
     }
 
@@ -446,6 +547,11 @@ void scriptingInit(void) {
         server.lua_client->flags |= REDIS_LUA_CLIENT;
     }
 
+    /* Lua beginners ofter don't use "local", this is likely to introduce
+     * subtle bugs in their code. To prevent problems we protect accesses
+     * to global variables. */
+    scriptingEnableGlobalsProtection(lua);
+
     server.lua = lua;
 }
 
@@ -461,10 +567,13 @@ void scriptingReset(void) {
     scriptingInit();
 }
 
-/* Hash the scripit into a SHA1 digest. We use this as Lua function name.
- * Digest should point to a 41 bytes buffer: 40 for SHA1 converted into an
+/* Perform the SHA1 of the input string. We use this both for hasing script
+ * bodies in order to obtain the Lua function name, and in the implementation
+ * of redis.sha1().
+ *
+ * 'digest' should point to a 41 bytes buffer: 40 for SHA1 converted into an
  * hexadecimal number, plus 1 byte for null term. */
-void hashScript(char *digest, char *script, size_t len) {
+void sha1hex(char *digest, char *script, size_t len) {
     SHA1_CTX ctx;
     unsigned char hash[20];
     char *cset = "0123456789abcdef";
@@ -576,7 +685,7 @@ int luaCreateFunction(redisClient *c, lua_State *lua, char *funcname, robj *body
     funcdef = sdscatlen(funcdef,body->ptr,sdslen(body->ptr));
     funcdef = sdscatlen(funcdef," end",4);
 
-    if (luaL_loadbuffer(lua,funcdef,sdslen(funcdef),"func definition")) {
+    if (luaL_loadbuffer(lua,funcdef,sdslen(funcdef),"@user_script")) {
         addReplyErrorFormat(c,"Error compiling script (new function): %s\n",
             lua_tostring(lua,-1));
         lua_pop(lua,1);
@@ -607,6 +716,7 @@ void evalGenericCommand(redisClient *c, int evalsha) {
     lua_State *lua = server.lua;
     char funcname[43];
     long long numkeys;
+    int delhook = 0;
 
     /* We want the same PRNG sequence at every call so that our PRNG is
      * not affected by external state. */
@@ -637,7 +747,7 @@ void evalGenericCommand(redisClient *c, int evalsha) {
     funcname[1] = '_';
     if (!evalsha) {
         /* Hash the code if this is an EVAL call */
-        hashScript(funcname+2,c->argv[1]->ptr,sdslen(c->argv[1]->ptr));
+        sha1hex(funcname+2,c->argv[1]->ptr,sdslen(c->argv[1]->ptr));
     } else {
         /* We already have the SHA if it is a EVALSHA */
         int j;
@@ -677,19 +787,19 @@ void evalGenericCommand(redisClient *c, int evalsha) {
      * is running for too much time.
      * We set the hook only if the time limit is enabled as the hook will
      * make the Lua script execution slower. */
+    server.lua_caller = c;
+    server.lua_time_start = ustime()/1000;
+    server.lua_kill = 0;
     if (server.lua_time_limit > 0 && server.masterhost == NULL) {
         lua_sethook(lua,luaMaskCountHook,LUA_MASKCOUNT,100000);
-    } else {
-        lua_sethook(lua,luaMaskCountHook,0,0);
+        delhook = 1;
     }
 
     /* At this point whatever this script was never seen before or if it was
      * already defined, we can call it. We have zero arguments and expect
      * a single return value. */
-    server.lua_caller = c;
-    server.lua_time_start = ustime()/1000;
-    server.lua_kill = 0;
     if (lua_pcall(lua,0,1,0)) {
+        if (delhook) lua_sethook(lua,luaMaskCountHook,0,0); /* Disable hook */
         if (server.lua_timedout) {
             server.lua_timedout = 0;
             /* Restore the readable handler that was unregistered when the
@@ -705,6 +815,7 @@ void evalGenericCommand(redisClient *c, int evalsha) {
         lua_gc(lua,LUA_GCCOLLECT,0);
         return;
     }
+    if (delhook) lua_sethook(lua,luaMaskCountHook,0,0); /* Disable hook */
     server.lua_timedout = 0;
     server.lua_caller = NULL;
     selectDb(c,server.lua_client->db->id); /* set DB ID from Lua client */
@@ -811,7 +922,7 @@ void scriptCommand(redisClient *c) {
 
         funcname[0] = 'f';
         funcname[1] = '_';
-        hashScript(funcname+2,c->argv[2]->ptr,sdslen(c->argv[2]->ptr));
+        sha1hex(funcname+2,c->argv[2]->ptr,sdslen(c->argv[2]->ptr));
         sha = sdsnewlen(funcname+2,40);
         if (dictFind(server.lua_scripts,sha) == NULL) {
             if (luaCreateFunction(c,server.lua,funcname,c->argv[2])

src/sds.c

@@ -111,6 +111,24 @@ sds sdsMakeRoomFor(sds s, size_t addlen) {
     return newsh->buf;
 }
 
+/* Reallocate the sds string so that it has no free space at the end. The
+ * contained string remains not altered, but next concatenation operations
+ * will require a reallocation. */
+sds sdsRemoveFreeSpace(sds s) {
+    struct sdshdr *sh;
+
+    sh = (void*) (s-(sizeof(struct sdshdr)));
+    sh = zrealloc(sh, sizeof(struct sdshdr)+sh->len+1);
+    sh->free = 0;
+    return sh->buf;
+}
+
+size_t sdsAllocSize(sds s) {
+    struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
+
+    return sizeof(*sh)+sh->len+sh->free+1;
+}
+
 /* Increment the sds length and decrements the left free space at the
  * end of the string accordingly to 'incr'. Also set the null term
  * in the new end of the string.

src/sds.h

@@ -94,5 +94,7 @@ sds sdsmapchars(sds s, char *from, char *to, size_t setlen);
 /* Low level functions exposed to the user API */
 sds sdsMakeRoomFor(sds s, size_t addlen);
 void sdsIncrLen(sds s, int incr);
+sds sdsRemoveFreeSpace(sds s);
+size_t sdsAllocSize(sds s);
 
 #endif

src/slowlog.c

@@ -16,13 +16,36 @@
  * this function. */
 slowlogEntry *slowlogCreateEntry(robj **argv, int argc, long long duration) {
     slowlogEntry *se = zmalloc(sizeof(*se));
-    int j;
+    int j, slargc = argc;
 
-    se->argc = argc;
-    se->argv = zmalloc(sizeof(robj*)*argc);
-    for (j = 0; j < argc; j++) {
-        se->argv[j] = argv[j];
-        incrRefCount(argv[j]);
+    if (slargc > SLOWLOG_ENTRY_MAX_ARGC) slargc = SLOWLOG_ENTRY_MAX_ARGC;
+    se->argc = slargc;
+    se->argv = zmalloc(sizeof(robj*)*slargc);
+    for (j = 0; j < slargc; j++) {
+        /* Logging too many arguments is a useless memory waste, so we stop
+         * at SLOWLOG_ENTRY_MAX_ARGC, but use the last argument to specify
+         * how many remaining arguments there were in the original command. */
+        if (slargc != argc && j == slargc-1) {
+            se->argv[j] = createObject(REDIS_STRING,
+                sdscatprintf(sdsempty(),"... (%d more arguments)",
+                argc-slargc+1));
+        } else {
+            /* Trim too long strings as well... */
+            if (argv[j]->type == REDIS_STRING &&
+                argv[j]->encoding == REDIS_ENCODING_RAW &&
+                sdslen(argv[j]->ptr) > SLOWLOG_ENTRY_MAX_STRING)
+            {
+                sds s = sdsnewlen(argv[j]->ptr, SLOWLOG_ENTRY_MAX_STRING);
+
+                s = sdscatprintf(s,"... (%lu more bytes)",
+                    (unsigned long)
+                    sdslen(argv[j]->ptr) - SLOWLOG_ENTRY_MAX_STRING);
+                se->argv[j] = createObject(REDIS_STRING,s);
+            } else {
+                se->argv[j] = argv[j];
+                incrRefCount(argv[j]);
+            }
+        }
     }
     se->time = time(NULL);
     se->duration = duration;

src/slowlog.h

@@ -1,3 +1,6 @@
+#define SLOWLOG_ENTRY_MAX_ARGC 32
+#define SLOWLOG_ENTRY_MAX_STRING 128
+
 /* This structure defines an entry inside the slow log list */
 typedef struct slowlogEntry {
     robj **argv;

src/sort.c

@@ -9,21 +9,27 @@ redisSortOperation *createSortOperation(int type, robj *pattern) {
     return so;
 }
 
-/* Return the value associated to the key with a name obtained
- * substituting the first occurence of '*' in 'pattern' with 'subst'.
+/* Return the value associated to the key with a name obtained using
+ * the following rules:
+ *
+ * 1) The first occurence of '*' in 'pattern' is substituted with 'subst'.
+ *
+ * 2) If 'pattern' matches the "->" string, everything on the left of
+ *    the arrow is treated as the name of an hash field, and the part on the
+ *    left as the key name containing an hash. The value of the specified
+ *    field is returned.
+ *
+ * 3) If 'pattern' equals "#", the function simply returns 'subst' itself so
+ *    that the SORT command can be used like: SORT key GET # to retrieve
+ *    the Set/List elements directly.
+ *
  * The returned object will always have its refcount increased by 1
  * when it is non-NULL. */
 robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst) {
-    char *p, *f;
+    char *p, *f, *k;
     sds spat, ssub;
-    robj keyobj, fieldobj, *o;
+    robj *keyobj, *fieldobj = NULL, *o;
     int prefixlen, sublen, postfixlen, fieldlen;
-    /* Expoit the internal sds representation to create a sds string allocated on the stack in order to make this function faster */
-    struct {
-        int len;
-        int free;
-        char buf[REDIS_SORTKEY_MAX+1];
-    } keyname, fieldname;
 
     /* If the pattern is "#" return the substitution object itself in order
      * to implement the "SORT ... GET #" feature. */
@@ -37,9 +43,10 @@ robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst) {
      * a decoded object on the fly. Otherwise getDecodedObject will just
      * increment the ref count, that we'll decrement later. */
     subst = getDecodedObject(subst);
-
     ssub = subst->ptr;
-    if (sdslen(spat)+sdslen(ssub)-1 > REDIS_SORTKEY_MAX) return NULL;
+
+    /* If we can't find '*' in the pattern we return NULL as to GET a
+     * fixed key does not make sense. */
     p = strchr(spat,'*');
     if (!p) {
         decrRefCount(subst);
@@ -47,46 +54,49 @@ robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst) {
     }
 
     /* Find out if we're dealing with a hash dereference. */
-    if ((f = strstr(p+1, "->")) != NULL) {
-        fieldlen = sdslen(spat)-(f-spat);
-        /* this also copies \0 character */
-        memcpy(fieldname.buf,f+2,fieldlen-1);
-        fieldname.len = fieldlen-2;
+    if ((f = strstr(p+1, "->")) != NULL && *(f+2) != '\0') {
+        fieldlen = sdslen(spat)-(f-spat)-2;
+        fieldobj = createStringObject(f+2,fieldlen);
     } else {
         fieldlen = 0;
     }
 
+    /* Perform the '*' substitution. */
     prefixlen = p-spat;
     sublen = sdslen(ssub);
-    postfixlen = sdslen(spat)-(prefixlen+1)-fieldlen;
-    memcpy(keyname.buf,spat,prefixlen);
-    memcpy(keyname.buf+prefixlen,ssub,sublen);
-    memcpy(keyname.buf+prefixlen+sublen,p+1,postfixlen);
-    keyname.buf[prefixlen+sublen+postfixlen] = '\0';
-    keyname.len = prefixlen+sublen+postfixlen;
-    decrRefCount(subst);
+    postfixlen = sdslen(spat)-(prefixlen+1)-(fieldlen ? fieldlen+2 : 0);
+    keyobj = createStringObject(NULL,prefixlen+sublen+postfixlen);
+    k = keyobj->ptr;
+    memcpy(k,spat,prefixlen);
+    memcpy(k+prefixlen,ssub,sublen);
+    memcpy(k+prefixlen+sublen,p+1,postfixlen);
+    decrRefCount(subst); /* Incremented by decodeObject() */
 
     /* Lookup substituted key */
-    initStaticStringObject(keyobj,((char*)&keyname)+(sizeof(struct sdshdr)));
-    o = lookupKeyRead(db,&keyobj);
-    if (o == NULL) return NULL;
+    o = lookupKeyRead(db,keyobj);
+    if (o == NULL) goto noobj;
 
-    if (fieldlen > 0) {
-        if (o->type != REDIS_HASH || fieldname.len < 1) return NULL;
+    if (fieldobj) {
+        if (o->type != REDIS_HASH) goto noobj;
 
         /* Retrieve value from hash by the field name. This operation
          * already increases the refcount of the returned object. */
-        initStaticStringObject(fieldobj,((char*)&fieldname)+(sizeof(struct sdshdr)));
-        o = hashTypeGetObject(o, &fieldobj);
+        o = hashTypeGetObject(o, fieldobj);
     } else {
-        if (o->type != REDIS_STRING) return NULL;
+        if (o->type != REDIS_STRING) goto noobj;
 
         /* Every object that this function returns needs to have its refcount
          * increased. sortCommand decreases it again. */
         incrRefCount(o);
     }
-
+    decrRefCount(keyobj);
+    if (fieldobj) decrRefCount(fieldobj);
     return o;
+
+noobj:
+    decrRefCount(keyobj);
+    if (fieldlen) decrRefCount(fieldobj);
+    return NULL;
 }
 
 /* sortCompare() is used by qsort in sortCommand(). Given that qsort_r with

src/syncio.c

@@ -36,50 +36,79 @@
  * of the SYNC command where the slave does it in a blocking way, and
  * the MIGRATE command that must be blocking in order to be atomic from the
  * point of view of the two instances (one migrating the key and one receiving
- * the key). This is why need the following blocking I/O functions. */
+ * the key). This is why need the following blocking I/O functions.
+ *
+ * All the functions take the timeout in milliseconds. */
 
-int syncWrite(int fd, char *ptr, ssize_t size, int timeout) {
+#define REDIS_SYNCIO_RESOLUTION 10 /* Resolution in milliseconds */
+
+/* Write the specified payload to 'fd'. If writing the whole payload will be done
+ * within 'timeout' milliseconds the operation succeeds and 'size' is returned.
+ * Otherwise the operation fails, -1 is returned, and an unspecified partial write
+ * could be performed against the file descriptor. */
+ssize_t syncWrite(int fd, char *ptr, ssize_t size, long long timeout) {
     ssize_t nwritten, ret = size;
-    time_t start = time(NULL);
+    long long start = mstime();
+    long long remaining = timeout;
 
-    timeout++;
-    while(size) {
-        if (aeWait(fd,AE_WRITABLE,1000) & AE_WRITABLE) {
+    while(1) {
+        long long wait = (remaining > REDIS_SYNCIO_RESOLUTION) ?
+                          remaining : REDIS_SYNCIO_RESOLUTION;
+        long long elapsed;
+
+        if (aeWait(fd,AE_WRITABLE,wait) & AE_WRITABLE) {
             nwritten = write(fd,ptr,size);
             if (nwritten == -1) return -1;
             ptr += nwritten;
             size -= nwritten;
+            if (size == 0) return ret;
         }
-        if ((time(NULL)-start) > timeout) {
+        elapsed = mstime() - start;
+        if (elapsed >= timeout) {
             errno = ETIMEDOUT;
             return -1;
         }
+        remaining = timeout - elapsed;
     }
-    return ret;
 }
 
-int syncRead(int fd, char *ptr, ssize_t size, int timeout) {
+/* Read the specified amount of bytes from 'fd'. If all the bytes are read within
+ * 'timeout' milliseconds the operation succeed and 'size' is returned.
+ * Otherwise the operation fails, -1 is returned, and an unspecified amount of
+ * data could be read from the file descriptor. */
+ssize_t syncRead(int fd, char *ptr, ssize_t size, long long timeout) {
     ssize_t nread, totread = 0;
-    time_t start = time(NULL);
+    long long start = mstime();
+    long long remaining = timeout;
 
-    timeout++;
-    while(size) {
-        if (aeWait(fd,AE_READABLE,1000) & AE_READABLE) {
+    while(1) {
+        long long wait = (remaining > REDIS_SYNCIO_RESOLUTION) ?
+                          remaining : REDIS_SYNCIO_RESOLUTION;
+        long long elapsed;
+
+        if (aeWait(fd,AE_READABLE,wait) & AE_READABLE) {
             nread = read(fd,ptr,size);
             if (nread <= 0) return -1;
             ptr += nread;
             size -= nread;
             totread += nread;
+            if (size == 0) return totread;
         }
-        if ((time(NULL)-start) > timeout) {
+        elapsed = mstime() - start;
+        if (elapsed >= timeout) {
             errno = ETIMEDOUT;
             return -1;
         }
+        remaining = timeout - elapsed;
     }
-    return totread;
 }
 
-int syncReadLine(int fd, char *ptr, ssize_t size, int timeout) {
+/* Read a line making sure that every char will not require more than 'timeout'
+ * milliseconds to be read.
+ * 
+ * On success the number of bytes read is returned, otherwise -1.
+ * On success the string is always correctly terminated with a 0 byte. */
+ssize_t syncReadLine(int fd, char *ptr, ssize_t size, long long timeout) {
     ssize_t nread = 0;
 
     size--;

src/t_hash.c

@@ -135,7 +135,9 @@ int hashTypeExists(robj *o, robj *field) {
 }
 
 /* Add an element, discard the old if the key already exists.
- * Return 0 on insert and 1 on update. */
+ * Return 0 on insert and 1 on update.
+ * This function will take care of incrementing the reference count of the
+ * retained fields and value objects. */
 int hashTypeSet(robj *o, robj *field, robj *value) {
     int update = 0;
 
@@ -168,30 +170,23 @@ int hashTypeSet(robj *o, robj *field, robj *value) {
             zl = ziplistPush(zl, field->ptr, sdslen(field->ptr), ZIPLIST_TAIL);
             zl = ziplistPush(zl, value->ptr, sdslen(value->ptr), ZIPLIST_TAIL);
         }
-
         o->ptr = zl;
-
         decrRefCount(field);
         decrRefCount(value);
 
         /* Check if the ziplist needs to be converted to a hash table */
-        if (hashTypeLength(o) > server.hash_max_ziplist_entries) {
+        if (hashTypeLength(o) > server.hash_max_ziplist_entries)
             hashTypeConvert(o, REDIS_ENCODING_HT);
-        }
-
     } else if (o->encoding == REDIS_ENCODING_HT) {
         if (dictReplace(o->ptr, field, value)) { /* Insert */
             incrRefCount(field);
         } else { /* Update */
             update = 1;
         }
-
         incrRefCount(value);
-
     } else {
         redisPanic("Unknown hash encoding");
     }
-
     return update;
 }
 
@@ -520,7 +515,7 @@ void hincrbyCommand(redisClient *c) {
 
 void hincrbyfloatCommand(redisClient *c) {
     double long value, incr;
-    robj *o, *current, *new;
+    robj *o, *current, *new, *aux;
 
     if (getLongDoubleFromObjectOrReply(c,c->argv[3],&incr,NULL) != REDIS_OK) return;
     if ((o = hashTypeLookupWriteOrCreate(c,c->argv[1])) == NULL) return;
@@ -540,9 +535,17 @@ void hincrbyfloatCommand(redisClient *c) {
     hashTypeTryObjectEncoding(o,&c->argv[2],NULL);
     hashTypeSet(o,c->argv[2],new);
     addReplyBulk(c,new);
-    decrRefCount(new);
     signalModifiedKey(c->db,c->argv[1]);
     server.dirty++;
+
+    /* Always replicate HINCRBYFLOAT as an HSET command with the final value
+     * in order to make sure that differences in float pricision or formatting
+     * will not create differences in replicas or after an AOF restart. */
+    aux = createStringObject("HSET",4);
+    rewriteClientCommandArgument(c,0,aux);
+    decrRefCount(aux);
+    rewriteClientCommandArgument(c,3,new);
+    decrRefCount(new);
 }
 
 static void addHashFieldToReply(redisClient *c, robj *o, robj *field) {

src/t_list.c

@@ -699,6 +699,8 @@ void rpoplpushCommand(redisClient *c) {
         checkType(c,sobj,REDIS_LIST)) return;
 
     if (listTypeLength(sobj) == 0) {
+        /* This may only happen after loading very old RDB files. Recent
+         * versions of Redis delete keys of empty lists. */
         addReply(c,shared.nullbulk);
     } else {
         robj *dobj = lookupKeyWrite(c->db,c->argv[2]);
@@ -910,7 +912,7 @@ int getTimeoutFromObjectOrReply(redisClient *c, robj *object, time_t *timeout) {
         return REDIS_ERR;
     }
 
-    if (tval > 0) tval += time(NULL);
+    if (tval > 0) tval += server.unixtime;
     *timeout = tval;
 
     return REDIS_OK;

src/t_zset.c

@@ -1259,7 +1259,9 @@ int zuiNext(zsetopsrc *op, zsetopval *val) {
     if (op->type == REDIS_SET) {
         iterset *it = &op->iter.set;
         if (op->encoding == REDIS_ENCODING_INTSET) {
-            if (!intsetGet(it->is.is,it->is.ii,(int64_t*)&val->ell))
+            int64_t ell = val->ell;
+
+            if (!intsetGet(it->is.is,it->is.ii,&ell))
                 return 0;
             val->score = 1.0;
 

src/version.h

@@ -1 +1 @@
-#define REDIS_VERSION "2.9.5"
+#define REDIS_VERSION "2.5.7"

src/ziplist.c

@@ -52,12 +52,23 @@
  *      String value with length less than or equal to 16383 bytes (14 bits).
  * |10______|qqqqqqqq|rrrrrrrr|ssssssss|tttttttt| - 5 bytes
  *      String value with length greater than or equal to 16384 bytes.
- * |1100____| - 1 byte
+ * |11000000| - 1 byte
  *      Integer encoded as int16_t (2 bytes).
- * |1101____| - 1 byte
+ * |11010000| - 1 byte
  *      Integer encoded as int32_t (4 bytes).
- * |1110____| - 1 byte
+ * |11100000| - 1 byte
  *      Integer encoded as int64_t (8 bytes).
+ * |11110000| - 1 byte
+ *      Integer encoded as 24 bit signed (3 bytes).
+ * |11111110| - 1 byte
+ *      Integer encoded as 8 bit signed (1 byte).
+ * |1111xxxx| - (with xxxx between 0000 and 1101) immediate 4 bit integer.
+ *      Unsigned integer from 0 to 12. The encoded value is actually from
+ *      1 to 13 because 0000 and 1111 can not be used, so 1 should be
+ *      subtracted from the encoded 4 bit value to obtain the right value.
+ * |11111111| - End of ziplist.
+ *
+ * All the integers are represented in little endian byte order.
  */
 
 #include <stdio.h>
@@ -75,14 +86,24 @@
 #define ZIP_BIGLEN 254
 
 /* Different encoding/length possibilities */
-#define ZIP_STR_MASK (0xc0)
-#define ZIP_INT_MASK (0x30)
+#define ZIP_STR_MASK 0xc0
+#define ZIP_INT_MASK 0x30
 #define ZIP_STR_06B (0 << 6)
 #define ZIP_STR_14B (1 << 6)
 #define ZIP_STR_32B (2 << 6)
 #define ZIP_INT_16B (0xc0 | 0<<4)
 #define ZIP_INT_32B (0xc0 | 1<<4)
 #define ZIP_INT_64B (0xc0 | 2<<4)
+#define ZIP_INT_24B (0xc0 | 3<<4)
+#define ZIP_INT_8B 0xfe
+/* 4 bit integer immediate encoding */
+#define ZIP_INT_IMM_MASK 0x0f
+#define ZIP_INT_IMM_MIN 0xf1    /* 11110001 */
+#define ZIP_INT_IMM_MAX 0xfd    /* 11111101 */
+#define ZIP_INT_IMM_VAL(v) (v & ZIP_INT_IMM_MASK)
+
+#define INT24_MAX 0x7fffff
+#define INT24_MIN (-INT24_MAX - 1)
 
 /* Macro to determine type */
 #define ZIP_IS_STR(enc) (((enc) & ZIP_STR_MASK) < ZIP_STR_MASK)
@@ -111,20 +132,22 @@ typedef struct zlentry {
     unsigned char *p;
 } zlentry;
 
-#define ZIP_ENTRY_ENCODING(ptr, encoding) do {                                 \
-    (encoding) = (ptr[0]) & (ZIP_STR_MASK | ZIP_INT_MASK);                     \
-    if (((encoding) & ZIP_STR_MASK) < ZIP_STR_MASK) {                          \
-        /* String encoding: 2 MSBs */                                          \
-        (encoding) &= ZIP_STR_MASK;                                            \
-    }                                                                          \
+/* Extract the encoding from the byte pointed by 'ptr' and set it into
+ * 'encoding'. */
+#define ZIP_ENTRY_ENCODING(ptr, encoding) do {  \
+    (encoding) = (ptr[0]); \
+    if ((encoding) < ZIP_STR_MASK) (encoding) &= ZIP_STR_MASK; \
 } while(0)
 
 /* Return bytes needed to store integer encoded by 'encoding' */
 static unsigned int zipIntSize(unsigned char encoding) {
     switch(encoding) {
-    case ZIP_INT_16B: return sizeof(int16_t);
-    case ZIP_INT_32B: return sizeof(int32_t);
-    case ZIP_INT_64B: return sizeof(int64_t);
+    case ZIP_INT_8B:  return 1;
+    case ZIP_INT_16B: return 2;
+    case ZIP_INT_24B: return 3;
+    case ZIP_INT_32B: return 4;
+    case ZIP_INT_64B: return 8;
+    default: return 0; /* 4 bit immediate */
     }
     assert(NULL);
     return 0;
@@ -240,7 +263,7 @@ static void zipPrevEncodeLengthForceLarge(unsigned char *p, unsigned int len) {
     } else if ((prevlensize) == 5) {                                           \
         assert(sizeof((prevlensize)) == 4);                                    \
         memcpy(&(prevlen), ((char*)(ptr)) + 1, 4);                             \
-        memrev32ifbe(&len);                                                    \
+        memrev32ifbe(&prevlen);                                                \
     }                                                                          \
 } while(0);
 
@@ -269,8 +292,14 @@ static int zipTryEncoding(unsigned char *entry, unsigned int entrylen, long long
     if (string2ll((char*)entry,entrylen,&value)) {
         /* Great, the string can be encoded. Check what's the smallest
          * of our encoding types that can hold this value. */
-        if (value >= INT16_MIN && value <= INT16_MAX) {
+        if (value >= 0 && value <= 12) {
+            *encoding = ZIP_INT_IMM_MIN+value;
+        } else if (value >= INT8_MIN && value <= INT8_MAX) {
+            *encoding = ZIP_INT_8B;
+        } else if (value >= INT16_MIN && value <= INT16_MAX) {
             *encoding = ZIP_INT_16B;
+        } else if (value >= INT24_MIN && value <= INT24_MAX) {
+            *encoding = ZIP_INT_24B;
         } else if (value >= INT32_MIN && value <= INT32_MAX) {
             *encoding = ZIP_INT_32B;
         } else {
@@ -287,10 +316,16 @@ static void zipSaveInteger(unsigned char *p, int64_t value, unsigned char encodi
     int16_t i16;
     int32_t i32;
     int64_t i64;
-    if (encoding == ZIP_INT_16B) {
+    if (encoding == ZIP_INT_8B) {
+        ((char*)p)[0] = (char)value;
+    } else if (encoding == ZIP_INT_16B) {
         i16 = value;
         memcpy(p,&i16,sizeof(i16));
         memrev16ifbe(p);
+    } else if (encoding == ZIP_INT_24B) {
+        i32 = value<<8;
+        memrev32ifbe(&i32);
+        memcpy(p,((unsigned char*)&i32)+1,sizeof(i32)-sizeof(int8_t));
     } else if (encoding == ZIP_INT_32B) {
         i32 = value;
         memcpy(p,&i32,sizeof(i32));
@@ -299,6 +334,8 @@ static void zipSaveInteger(unsigned char *p, int64_t value, unsigned char encodi
         i64 = value;
         memcpy(p,&i64,sizeof(i64));
         memrev64ifbe(p);
+    } else if (encoding >= ZIP_INT_IMM_MIN && encoding <= ZIP_INT_IMM_MAX) {
+        /* Nothing to do, the value is stored in the encoding itself. */
     } else {
         assert(NULL);
     }
@@ -309,7 +346,9 @@ static int64_t zipLoadInteger(unsigned char *p, unsigned char encoding) {
     int16_t i16;
     int32_t i32;
     int64_t i64, ret = 0;
-    if (encoding == ZIP_INT_16B) {
+    if (encoding == ZIP_INT_8B) {
+        ret = ((char*)p)[0];
+    } else if (encoding == ZIP_INT_16B) {
         memcpy(&i16,p,sizeof(i16));
         memrev16ifbe(&i16);
         ret = i16;
@@ -317,10 +356,17 @@ static int64_t zipLoadInteger(unsigned char *p, unsigned char encoding) {
         memcpy(&i32,p,sizeof(i32));
         memrev32ifbe(&i32);
         ret = i32;
+    } else if (encoding == ZIP_INT_24B) {
+        i32 = 0;
+        memcpy(((unsigned char*)&i32)+1,p,sizeof(i32)-sizeof(int8_t));
+        memrev32ifbe(&i32);
+        ret = i32>>8;
     } else if (encoding == ZIP_INT_64B) {
         memcpy(&i64,p,sizeof(i64));
         memrev64ifbe(&i64);
         ret = i64;
+    } else if (encoding >= ZIP_INT_IMM_MIN && encoding <= ZIP_INT_IMM_MAX) {
+        ret = (encoding & ZIP_INT_IMM_MASK)-1;
     } else {
         assert(NULL);
     }

src/zmalloc.c

@@ -30,6 +30,15 @@
 
 #include <stdio.h>
 #include <stdlib.h>
+
+/* This function provide us access to the original libc free(). This is useful
+ * for instance to free results obtained by backtrace_symbols(). We need
+ * to define this function before including zmalloc.h that may shadow the
+ * free implementation if we use jemalloc or another non standard allocator. */
+void zlibc_free(void *ptr) {
+    free(ptr);
+}
+
 #include <string.h>
 #include <pthread.h>
 #include "config.h"

src/zmalloc.h

@@ -38,7 +38,7 @@
 #if defined(USE_TCMALLOC)
 #define ZMALLOC_LIB ("tcmalloc-" __xstr(TC_VERSION_MAJOR) "." __xstr(TC_VERSION_MINOR))
 #include <google/tcmalloc.h>
-#if TC_VERSION_MAJOR >= 1 && TC_VERSION_MINOR >= 6
+#if (TC_VERSION_MAJOR == 1 && TC_VERSION_MINOR >= 6) || (TC_VERSION_MAJOR > 1)
 #define HAVE_MALLOC_SIZE 1
 #define zmalloc_size(p) tc_malloc_size(p)
 #else
@@ -49,7 +49,7 @@
 #define ZMALLOC_LIB ("jemalloc-" __xstr(JEMALLOC_VERSION_MAJOR) "." __xstr(JEMALLOC_VERSION_MINOR) "." __xstr(JEMALLOC_VERSION_BUGFIX))
 #define JEMALLOC_MANGLE
 #include <jemalloc/jemalloc.h>
-#if JEMALLOC_VERSION_MAJOR >= 2 && JEMALLOC_VERSION_MINOR >= 1
+#if (JEMALLOC_VERSION_MAJOR == 2 && JEMALLOC_VERSION_MINOR >= 1) || (JEMALLOC_VERSION_MAJOR > 2)
 #define HAVE_MALLOC_SIZE 1
 #define zmalloc_size(p) JEMALLOC_P(malloc_usable_size)(p)
 #else
@@ -75,6 +75,7 @@ size_t zmalloc_used_memory(void);
 void zmalloc_enable_thread_safeness(void);
 float zmalloc_get_fragmentation_ratio(void);
 size_t zmalloc_get_rss(void);
+void zlibc_free(void *ptr);
 
 #ifndef HAVE_MALLOC_SIZE
 size_t zmalloc_size(void *ptr);

tests/helpers/bg_complex_data.tcl

@@ -0,0 +1,10 @@
+source tests/support/redis.tcl
+source tests/support/util.tcl
+
+proc bg_complex_data {host port db ops} {
+    set r [redis $host $port]
+    $r select $db
+    createComplexDataset $r $ops
+}
+
+bg_complex_data [lindex $argv 0] [lindex $argv 1] [lindex $argv 2] [lindex $argv 3]

tests/integration/aof.tcl

@@ -35,7 +35,7 @@ tags {"aof"} {
             set pattern "*Unexpected end of file reading the append only file*"
             set retry 10
             while {$retry} {
-                set result [exec cat [dict get $srv stdout] | tail -n1]
+                set result [exec tail -n1 < [dict get $srv stdout]]
                 if {[string match $pattern $result]} {
                     break
                 }
@@ -59,7 +59,7 @@ tags {"aof"} {
             set pattern "*Bad file format reading the append only file*"
             set retry 10
             while {$retry} {
-                set result [exec cat [dict get $srv stdout] | tail -n1]
+                set result [exec tail -n1 < [dict get $srv stdout]]
                 if {[string match $pattern $result]} {
                     break
                 }
@@ -81,7 +81,7 @@ tags {"aof"} {
     }
 
     test "Short read: Utility should be able to fix the AOF" {
-        set result [exec echo y | src/redis-check-aof --fix $aof_path]
+        set result [exec src/redis-check-aof --fix $aof_path << "y\n"]
         assert_match "*Successfully truncated AOF*" $result
     }
 

tests/integration/convert-zipmap-hash-on-load.tcl

@@ -1,8 +1,7 @@
+# Copy RDB with zipmap encoded hash to server path
 set server_path [tmpdir "server.convert-zipmap-hash-on-load"]
 
-# Copy RDB with zipmap encoded hash to server path
-exec cp tests/assets/hash-zipmap.rdb $server_path
-
+exec cp -f tests/assets/hash-zipmap.rdb $server_path
 start_server [list overrides [list "dir" $server_path "dbfilename" "hash-zipmap.rdb"]] {
   test "RDB load zipmap hash: converts to ziplist" {
     r select 0
@@ -13,6 +12,7 @@ start_server [list overrides [list "dir" $server_path "dbfilename" "hash-zipmap.
   }
 }
 
+exec cp -f tests/assets/hash-zipmap.rdb $server_path
 start_server [list overrides [list "dir" $server_path "dbfilename" "hash-zipmap.rdb" "hash-max-ziplist-entries" 1]] {
   test "RDB load zipmap hash: converts to hash table when hash-max-ziplist-entries is exceeded" {
     r select 0
@@ -23,6 +23,7 @@ start_server [list overrides [list "dir" $server_path "dbfilename" "hash-zipmap.
   }
 }
 
+exec cp -f tests/assets/hash-zipmap.rdb $server_path
 start_server [list overrides [list "dir" $server_path "dbfilename" "hash-zipmap.rdb" "hash-max-ziplist-value" 1]] {
   test "RDB load zipmap hash: converts to hash table when hash-max-ziplist-value is exceeded" {
     r select 0

tests/integration/rdb.tcl

@@ -0,0 +1,25 @@
+set server_path [tmpdir "server.rdb-encoding-test"]
+
+# Copy RDB with different encodings in server path
+exec cp tests/assets/encodings.rdb $server_path
+
+start_server [list overrides [list "dir" $server_path "dbfilename" "encodings.rdb"]] {
+  test "RDB encoding loading test" {
+    r select 0
+    csvdump r
+  } {"compressible","string","aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
+"hash","hash","a","1","aa","10","aaa","100","b","2","bb","20","bbb","200","c","3","cc","30","ccc","300","ddd","400","eee","5000000000",
+"hash_zipped","hash","a","1","b","2","c","3",
+"list","list","1","2","3","a","b","c","100000","6000000000","1","2","3","a","b","c","100000","6000000000","1","2","3","a","b","c","100000","6000000000",
+"list_zipped","list","1","2","3","a","b","c","100000","6000000000",
+"number","string","10"
+"set","set","1","100000","2","3","6000000000","a","b","c",
+"set_zipped_1","set","1","2","3","4",
+"set_zipped_2","set","100000","200000","300000","400000",
+"set_zipped_3","set","1000000000","2000000000","3000000000","4000000000","5000000000","6000000000",
+"string","string","Hello World"
+"zset","zset","a","1","b","2","c","3","aa","10","bb","20","cc","30","aaa","100","bbb","200","ccc","300","aaaa","1000","cccc","123456789","bbbb","5000000000",
+"zset_zipped","zset","a","1","b","2","c","3",
+}
+}
+

tests/integration/replication-4.tcl

@@ -0,0 +1,54 @@
+proc start_bg_complex_data {host port db ops} {
+    exec tclsh8.5 tests/helpers/bg_complex_data.tcl $host $port $db $ops &
+}
+
+proc stop_bg_complex_data {handle} {
+    catch {exec /bin/kill -9 $handle}
+}
+
+start_server {tags {"repl"}} {
+    start_server {} {
+
+        set master [srv 0 client]
+        set master_host [srv 0 host]
+        set master_port [srv 0 port]
+        set load_handle0 [start_bg_complex_data $master_host $master_port 9 100000]
+        set load_handle1 [start_bg_complex_data $master_host $master_port 11 100000]
+        set load_handle2 [start_bg_complex_data $master_host $master_port 12 100000]
+
+        test {First server should have role slave after SLAVEOF} {
+            r -1 slaveof [srv 0 host] [srv 0 port]
+            after 1000
+            s -1 role
+        } {slave}
+
+        test {Test replication with parallel clients writing in differnet DBs} {
+            lappend slave [srv 0 client]
+            after 5000
+            stop_bg_complex_data $load_handle0
+            stop_bg_complex_data $load_handle1
+            stop_bg_complex_data $load_handle2
+            set retry 10
+            while {$retry && ([$master debug digest] ne [$slave debug digest])}\
+            {
+                after 1000
+                incr retry -1
+            }
+            assert {[$master dbsize] > 0}
+
+            if {[r debug digest] ne [r -1 debug digest]} {
+                set csv1 [csvdump r]
+                set csv2 [csvdump {r -1}]
+                set fd [open /tmp/repldump1.txt w]
+                puts -nonewline $fd $csv1
+                close $fd
+                set fd [open /tmp/repldump2.txt w]
+                puts -nonewline $fd $csv2
+                close $fd
+                puts "Master - Slave inconsistency"
+                puts "Run diff -u against /tmp/repldump*.txt for more info"
+            }
+            assert_equal [r debug digest] [r -1 debug digest]
+        }
+    }
+}

tests/integration/replication.tcl

@@ -10,8 +10,11 @@ start_server {tags {"repl"}} {
             set rd [redis_deferring_client]
             $rd brpoplpush a b 5
             r lpush a foo
-            after 1000
-            assert_equal [r debug digest] [r -1 debug digest]
+            wait_for_condition 50 100 {
+                [r debug digest] eq [r -1 debug digest]
+            } else {
+                fail "Master and slave have different digest: [r debug digest] VS [r -1 debug digest]"
+            }
         }
 
         test {BRPOPLPUSH replication, list exists} {
@@ -97,7 +100,7 @@ start_server {tags {"repl"}} {
                     [lindex $slaves 2] slaveof $master_host $master_port
 
                     # Wait for all the three slaves to reach the "online" state
-                    set retry 100
+                    set retry 500
                     while {$retry} {
                         set info [r -3 info]
                         if {[string match {*slave0:*,online*slave1:*,online*slave2:*,online*} $info]} {

tests/support/redis.tcl

@@ -142,9 +142,15 @@ proc ::redis::redis_multi_bulk_read fd {
     set count [redis_read_line $fd]
     if {$count == -1} return {}
     set l {}
+    set err {}
     for {set i 0} {$i < $count} {incr i} {
-        lappend l [redis_read_reply $fd]
+        if {[catch {
+            lappend l [redis_read_reply $fd]
+        } e] && $err eq {}} {
+            set err $e
+        }
     }
+    if {$err ne {}} {return -code error $err}
     return $l
 }
 
@@ -160,7 +166,7 @@ proc ::redis::redis_read_reply fd {
         - {return -code error [redis_read_line $fd]}
         $ {redis_bulk_read $fd}
         * {redis_multi_bulk_read $fd}
-        default {return -code error "Bad protocol, $type as reply type byte"}
+        default {return -code error "Bad protocol, '$type' as reply type byte"}
     }
 }
 

tests/support/server.tcl

@@ -17,7 +17,7 @@ proc check_valgrind_errors stderr {
     set buf [read $fd]
     close $fd
 
-    if {![regexp -- {ERROR SUMMARY: 0 errors} $buf] ||
+    if {[regexp -- { at 0x} $buf] ||
         (![regexp -- {definitely lost: 0 bytes} $buf] &&
          ![regexp -- {no leaks are possible} $buf])} {
         send_data_packet $::test_server_fd err "Valgrind error: $buf\n"
@@ -46,11 +46,16 @@ proc kill_server config {
     }
 
     # kill server and wait for the process to be totally exited
+    catch {exec kill $pid}
     while {[is_alive $config]} {
-        if {[incr wait 10] % 1000 == 0} {
+        incr wait 10
+
+        if {$wait >= 5000} {
+            puts "Forcing process $pid to exit..."
+            catch {exec kill -KILL $pid}
+        } elseif {$wait % 1000 == 0} {
             puts "Waiting for process $pid to exit..."
         }
-        catch {exec kill $pid}
         after 10
     }
 
@@ -176,7 +181,7 @@ proc start_server {options {code undefined}} {
     set stderr [format "%s/%s" [dict get $config "dir"] "stderr"]
 
     if {$::valgrind} {
-        exec valgrind --suppressions=src/valgrind.sup src/redis-server $config_file > $stdout 2> $stderr &
+        exec valgrind --suppressions=src/valgrind.sup --show-reachable=no --show-possibly-lost=no --leak-check=full src/redis-server $config_file > $stdout 2> $stderr &
     } else {
         exec src/redis-server $config_file > $stdout 2> $stderr &
     }
@@ -247,7 +252,7 @@ proc start_server {options {code undefined}} {
 
         while 1 {
             # check that the server actually started and is ready for connections
-            if {[exec cat $stdout | grep "ready to accept" | wc -l] > 0} {
+            if {[exec grep "ready to accept" | wc -l < $stdout] > 0} {
                 break
             }
             after 10

tests/support/test.tcl

@@ -3,6 +3,10 @@ set ::num_passed 0
 set ::num_failed 0
 set ::tests_failed {}
 
+proc fail {msg} {
+    error "assertion:$msg"
+}
+
 proc assert {condition} {
     if {![uplevel 1 [list expr $condition]]} {
         error "assertion:Expected condition '$condition' to be true ([uplevel 1 [list subst -nocommands $condition]])"
@@ -44,6 +48,19 @@ proc assert_type {type key} {
     assert_equal $type [r type $key]
 }
 
+# Wait for the specified condition to be true, with the specified number of
+# max retries and delay between retries. Otherwise the 'elsescript' is
+# executed.
+proc wait_for_condition {maxtries delay e _else_ elsescript} {
+    while {[incr maxtries -1] >= 0} {
+        if {[uplevel 1 [list expr $e]]} break
+        after $delay
+    }
+    if {$maxtries == -1} {
+        uplevel 1 $elsescript
+    }
+}
+
 # Test if TERM looks like to support colors
 proc color_term {} {
     expr {[info exists ::env(TERM)] && [string match *xterm* $::env(TERM)]}

tests/test_helper.tcl

@@ -29,13 +29,18 @@ set ::all_tests {
     integration/replication
     integration/replication-2
     integration/replication-3
+    integration/replication-4
     integration/aof
+    integration/rdb
+    integration/convert-zipmap-hash-on-load
     unit/pubsub
     unit/slowlog
     unit/scripting
     unit/maxmemory
     unit/introspection
+    unit/limits
     unit/obuf-limits
+    unit/dump
 }
 # Index to the next test to run in the ::all_tests list.
 set ::next_test 0
@@ -341,6 +346,7 @@ proc print_help_screen {} {
         "--quiet            Don't show individual tests."
         "--single <unit>    Just execute the specified unit (see next option)."
         "--list-tests       List all the available test units."
+        "--clients <num>    Number of test clients (16)."
         "--force-failure    Force the execution of a test that always fails."
         "--help             Print this help screen."
     } "\n"]
@@ -386,6 +392,9 @@ for {set j 0} {$j < [llength $argv]} {incr j} {
         set ::client 1
         set ::test_server_port $arg
         incr j
+    } elseif {$opt eq {--clients}} {
+        set ::numclients $arg
+        incr j
     } elseif {$opt eq {--help}} {
         print_help_screen
         exit 0

tests/unit/aofrw.tcl

@@ -1,4 +1,15 @@
 start_server {tags {"aofrw"}} {
+
+    test {Turning off AOF kills the background writing child if any} {
+        r config set appendonly yes
+        waitForBgrewriteaof r
+        r multi
+        r bgrewriteaof
+        r config set appendonly no
+        r exec
+        set result [exec tail -n1 < [srv 0 stdout] ]
+    } {*Killing*AOF*child*}
+
     foreach d {string int} {
         foreach e {ziplist linkedlist} {
             test "AOF rewrite of list with $e encoding, $d data" {
@@ -104,4 +115,30 @@ start_server {tags {"aofrw"}} {
             }
         }
     }
+
+    test {BGREWRITEAOF is delayed if BGSAVE is in progress} {
+        r multi
+        r bgsave
+        r bgrewriteaof
+        r info persistence
+        set res [r exec]
+        assert_match {*scheduled*} [lindex $res 1]
+        assert_match {*bgrewriteaof_scheduled:1*} [lindex $res 2]
+        while {[string match {*bgrewriteaof_scheduled:1*} [r info persistence]]} {
+            after 100
+        }
+    }
+
+    test {BGREWRITEAOF is refused if already in progress} {
+        catch {
+            r multi
+            r bgrewriteaof
+            r bgrewriteaof
+            r exec
+        } e
+        assert_match {*ERR*already*} $e
+        while {[string match {*bgrewriteaof_scheduled:1*} [r info persistence]]} {
+            after 100
+        }
+    }
 }

tests/unit/dump.tcl

@@ -0,0 +1,112 @@
+start_server {tags {"dump"}} {
+    test {DUMP / RESTORE are able to serialize / unserialize a simple key} {
+        r set foo bar
+        set encoded [r dump foo]
+        r del foo
+        list [r exists foo] [r restore foo 0 $encoded] [r ttl foo] [r get foo]
+    } {0 OK -1 bar}
+
+    test {RESTORE can set an arbitrary expire to the materialized key} {
+        r set foo bar
+        set encoded [r dump foo]
+        r del foo
+        r restore foo 5000 $encoded
+        set ttl [r pttl foo]
+        assert {$ttl >= 3000 && $ttl <= 5000}
+        r get foo
+    } {bar}
+
+    test {RESTORE returns an error of the key already exists} {
+        r set foo bar
+        set e {}
+        catch {r restore foo 0 "..."} e
+        set e
+    } {*is busy*}
+
+    test {DUMP of non existing key returns nil} {
+        r dump nonexisting_key
+    } {}
+
+    test {MIGRATE is able to migrate a key between two instances} {
+        set first [srv 0 client]
+        r set key "Some Value"
+        start_server {tags {"repl"}} {
+            set second [srv 0 client]
+            set second_host [srv 0 host]
+            set second_port [srv 0 port]
+
+            assert {[$first exists key] == 1}
+            assert {[$second exists key] == 0}
+            set ret [r -1 migrate $second_host $second_port key 9 5000]
+            assert {$ret eq {OK}}
+            assert {[$first exists key] == 0}
+            assert {[$second exists key] == 1}
+            assert {[$second get key] eq {Some Value}}
+            assert {[$second ttl key] == -1}
+        }
+    }
+
+    test {MIGRATE propagates TTL correctly} {
+        set first [srv 0 client]
+        r set key "Some Value"
+        start_server {tags {"repl"}} {
+            set second [srv 0 client]
+            set second_host [srv 0 host]
+            set second_port [srv 0 port]
+
+            assert {[$first exists key] == 1}
+            assert {[$second exists key] == 0}
+            $first expire key 10
+            set ret [r -1 migrate $second_host $second_port key 9 5000]
+            assert {$ret eq {OK}}
+            assert {[$first exists key] == 0}
+            assert {[$second exists key] == 1}
+            assert {[$second get key] eq {Some Value}}
+            assert {[$second ttl key] >= 7 && [$second ttl key] <= 10}
+        }
+    }
+
+    test {MIGRATE can correctly transfer large values} {
+        set first [srv 0 client]
+        r del key
+        for {set j 0} {$j < 5000} {incr j} {
+            r rpush key 1 2 3 4 5 6 7 8 9 10
+            r rpush key "item 1" "item 2" "item 3" "item 4" "item 5" \
+                        "item 6" "item 7" "item 8" "item 9" "item 10"
+        }
+        assert {[string length [r dump key]] > (1024*64)}
+        start_server {tags {"repl"}} {
+            set second [srv 0 client]
+            set second_host [srv 0 host]
+            set second_port [srv 0 port]
+
+            assert {[$first exists key] == 1}
+            assert {[$second exists key] == 0}
+            set ret [r -1 migrate $second_host $second_port key 9 10000]
+            assert {$ret eq {OK}}
+            assert {[$first exists key] == 0}
+            assert {[$second exists key] == 1}
+            assert {[$second ttl key] == -1}
+            assert {[$second llen key] == 5000*20}
+        }
+    }
+
+    test {MIGRATE timeout actually works} {
+        set first [srv 0 client]
+        r set key "Some Value"
+        start_server {tags {"repl"}} {
+            set second [srv 0 client]
+            set second_host [srv 0 host]
+            set second_port [srv 0 port]
+
+            assert {[$first exists key] == 1}
+            assert {[$second exists key] == 0}
+
+            set rd [redis_deferring_client]
+            $rd debug sleep 5.0 ; # Make second server unable to reply.
+            set e {}
+            catch {r -1 migrate $second_host $second_port key 9 1000} e
+            assert_match {IOERR*} $e
+        }
+    }
+}

tests/unit/introspection.tcl

@@ -1,5 +1,22 @@
 start_server {tags {"introspection"}} {
     test {CLIENT LIST} {
         r client list
-    } {*addr=*:* fd=* idle=* flags=N db=9 sub=0 psub=0 qbuf=0 obl=0 oll=0 omem=0 events=r cmd=client*}
+    } {*addr=*:* fd=* age=* idle=* flags=N db=9 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=* obl=0 oll=0 omem=0 events=r cmd=client*}
+
+    test {MONITOR can log executed commands} {
+        set rd [redis_deferring_client]
+        $rd monitor
+        r set foo bar
+        r get foo
+        list [$rd read] [$rd read] [$rd read]
+    } {*OK*"set" "foo"*"get" "foo"*}
+
+    test {MONITOR can log commands issued by the scripting engine} {
+        set rd [redis_deferring_client]
+        $rd monitor
+        r eval {redis.call('set',KEYS[1],ARGV[1])} 1 foo bar
+        $rd read ;# Discard the OK
+        assert_match {*eval*} [$rd read]
+        assert_match {*lua*"set"*"foo"*"bar"*} [$rd read]
+    }
 }

tests/unit/limits.tcl

@@ -0,0 +1,16 @@
+start_server {tags {"limits"} overrides {maxclients 10}} {
+    test {Check if maxclients works refusing connections} {
+        set c 0
+        catch {
+            while {$c < 50} {
+                incr c
+                set rd [redis_deferring_client]
+                $rd ping
+                $rd read
+                after 100
+            }
+        } e
+        assert {$c > 8 && $c <= 10}
+        set e
+    } {*ERR max*reached*}
+}

tests/unit/protocol.tcl

@@ -68,7 +68,7 @@ start_server {tags {"protocol"}} {
             puts -nonewline $s $seq
             set payload [string repeat A 1024]"\n"
             set test_start [clock seconds]
-            set test_time_limit 5
+            set test_time_limit 30
             while 1 {
                 if {[catch {
                     puts -nonewline $s payload

tests/unit/scripting.tcl

@@ -214,6 +214,101 @@ start_server {tags {"scripting"}} {
         r sadd myset a b c
         r eval {return redis.call('sort','myset','by','_','get','#','get','_:*')} 0
     } {{} {} {} a b c}
+
+    test "redis.sha1hex() implementation" {
+        list [r eval {return redis.sha1hex('')} 0] \
+             [r eval {return redis.sha1hex('Pizza & Mandolino')} 0]
+    } {da39a3ee5e6b4b0d3255bfef95601890afd80709 74822d82031af7493c20eefa13bd07ec4fada82f}
+
+    test {Globals protection reading an undeclared global variable} {
+        catch {r eval {return a} 0} e
+        set e
+    } {*ERR*attempted to access unexisting global*}
+
+    test {Globals protection setting an undeclared global*} {
+        catch {r eval {a=10} 0} e
+        set e
+    } {*ERR*attempted to create global*}
+
+    test {Test an example script DECR_IF_GT} {
+        set decr_if_gt {
+            local current
+
+            current = redis.call('get',KEYS[1])
+            if not current then return nil end
+            if current > ARGV[1] then
+                return redis.call('decr',KEYS[1])
+            else
+                return redis.call('get',KEYS[1])
+            end
+        }
+        r set foo 5
+        set res {}
+        lappend res [r eval $decr_if_gt 1 foo 2]
+        lappend res [r eval $decr_if_gt 1 foo 2]
+        lappend res [r eval $decr_if_gt 1 foo 2]
+        lappend res [r eval $decr_if_gt 1 foo 2]
+        lappend res [r eval $decr_if_gt 1 foo 2]
+        set res
+    } {4 3 2 2 2}
+
+    test {Scripting engine resets PRNG at every script execution} {
+        set rand1 [r eval {return tostring(math.random())} 0]
+        set rand2 [r eval {return tostring(math.random())} 0]
+        assert_equal $rand1 $rand2
+    }
+
+    test {Scripting engine PRNG can be seeded correctly} {
+        set rand1 [r eval {
+            math.randomseed(ARGV[1]); return tostring(math.random())
+        } 0 10]
+        set rand2 [r eval {
+            math.randomseed(ARGV[1]); return tostring(math.random())
+        } 0 10]
+        set rand3 [r eval {
+            math.randomseed(ARGV[1]); return tostring(math.random())
+        } 0 20]
+        assert_equal $rand1 $rand2
+        assert {$rand2 ne $rand3}
+    }
+}
+
+# Start a new server since the last test in this stanza will kill the
+# instance at all.
+start_server {tags {"scripting"}} {
+    test {Timedout read-only scripts can be killed by SCRIPT KILL} {
+        set rd [redis_deferring_client]
+        r config set lua-time-limit 10
+        $rd eval {while true do end} 0
+        after 200
+        catch {r ping} e
+        assert_match {BUSY*} $e
+        r script kill
+        assert_equal [r ping] "PONG"
+    }
+
+    test {Timedout scripts that modified data can't be killed by SCRIPT KILL} {
+        set rd [redis_deferring_client]
+        r config set lua-time-limit 10
+        $rd eval {redis.call('set','x','y'); while true do end} 0
+        after 200
+        catch {r ping} e
+        assert_match {BUSY*} $e
+        catch {r script kill} e
+        assert_match {ERR*} $e
+        catch {r ping} e
+        assert_match {BUSY*} $e
+    }
+
+    test {SHUTDOWN NOSAVE can kill a timedout script anyway} {
+        # The server sould be still unresponding to normal commands.
+        catch {r ping} e
+        assert_match {BUSY*} $e
+        catch {r shutdown nosave}
+        # Make sure the server was killed
+        catch {set rd [redis_deferring_client]} e
+        assert_match {*connection refused*} $e
+    }
 }
 
 start_server {tags {"scripting repl"}} {
@@ -232,10 +327,12 @@ start_server {tags {"scripting repl"}} {
             r evalsha ae3477e27be955de7e1bc9adfdca626b478d3cb2 0
         } {2}
 
-        if {$::valgrind} {after 2000} else {after 100}
-
         test {If EVALSHA was replicated as EVAL the slave should be ok} {
-            r -1 get x
-        } {2}
+            wait_for_condition 50 100 {
+                [r -1 get x] eq {2}
+            } else {
+                fail "Expected 2 in x, but value is '[r -1 get x]'"
+            }
+        }
     }
 }

tests/unit/slowlog.tcl

@@ -38,4 +38,21 @@ start_server {tags {"slowlog"} overrides {slowlog-log-slower-than 1000000}} {
         assert_equal [expr {[lindex $e 2] > 100000}] 1
         assert_equal [lindex $e 3] {debug sleep 0.2}
     }
+
+    test {SLOWLOG - commands with too many arguments are trimmed} {
+        r config set slowlog-log-slower-than 0
+        r slowlog reset
+        r sadd set 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
+        set e [lindex [r slowlog get] 0]
+        lindex $e 3
+    } {sadd set 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 {... (2 more arguments)}}
+
+    test {SLOWLOG - too long arguments are trimmed} {
+        r config set slowlog-log-slower-than 0
+        r slowlog reset
+        set arg [string repeat A 129]
+        r sadd set foo $arg
+        set e [lindex [r slowlog get] 0]
+        lindex $e 3
+    } {sadd set foo {AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA... (1 more bytes)}}
 }

tests/unit/sort.tcl

@@ -190,6 +190,13 @@ start_server {
         r sort myset by score:*
     } {a aa aaa azz b c d e f g h i l m n o p q r s t u v z}
 
+    test "SORT GET with pattern ending with just -> does not get hash field" {
+        r del mylist
+        r lpush mylist a
+        r set x:a-> 100
+        r sort mylist by num get x:*->
+    } {100}
+
     tags {"slow"} {
         set num 100
         set res [create_random_dataset $num lpush]

tests/unit/type/hash.tcl

@@ -395,4 +395,28 @@ start_server {tags {"hash"}} {
         r hset hash kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk b
         r hget hash kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
     } {b}
+
+    foreach size {10 512} {
+        test "Hash fuzzing - $size fields" {
+            for {set times 0} {$times < 10} {incr times} {
+                catch {unset hash}
+                array set hash {}
+                r del hash
+
+                # Create
+                for {set j 0} {$j < $size} {incr j} {
+                    set field [randomValue]
+                    set value [randomValue]
+                    r hset hash $field $value
+                    set hash($field) $value
+                }
+
+                # Verify
+                foreach {k v} [array get hash] {
+                    assert_equal $v [r hget hash $k]
+                }
+                assert_equal [array size hash] [r hlen hash]
+            }
+        }
+    }
 }

tests/unit/type/list-3.tcl

@@ -29,6 +29,15 @@ start_server {
                         set data [randomInt 4294967296]
                     } {
                         set data [randomInt 18446744073709551616]
+                    } {
+                        set data -[randomInt 65536]
+                        if {$data eq {-0}} {set data 0}
+                    } {
+                        set data -[randomInt 4294967296]
+                        if {$data eq {-0}} {set data 0}
+                    } {
+                        set data -[randomInt 18446744073709551616]
+                        if {$data eq {-0}} {set data 0}
                     }
                     lappend l $data
                     r rpush l $data

tests/unit/type/list.tcl

@@ -7,7 +7,7 @@ start_server {
 } {
     source "tests/unit/type/list-common.tcl"
 
-    test {LPUSH, RPUSH, LLENGTH, LINDEX - ziplist} {
+    test {LPUSH, RPUSH, LLENGTH, LINDEX, LPOP - ziplist} {
         # first lpush then rpush
         assert_equal 1 [r lpush myziplist1 a]
         assert_equal 2 [r rpush myziplist1 b]
@@ -16,6 +16,9 @@ start_server {
         assert_equal a [r lindex myziplist1 0]
         assert_equal b [r lindex myziplist1 1]
         assert_equal c [r lindex myziplist1 2]
+        assert_equal {} [r lindex myziplist2 3]
+        assert_equal c [r rpop myziplist1]
+        assert_equal a [r lpop myziplist1]
         assert_encoding ziplist myziplist1
 
         # first rpush then lpush
@@ -26,10 +29,13 @@ start_server {
         assert_equal c [r lindex myziplist2 0]
         assert_equal b [r lindex myziplist2 1]
         assert_equal a [r lindex myziplist2 2]
+        assert_equal {} [r lindex myziplist2 3]
+        assert_equal a [r rpop myziplist2]
+        assert_equal c [r lpop myziplist2]
         assert_encoding ziplist myziplist2
     }
 
-    test {LPUSH, RPUSH, LLENGTH, LINDEX - regular list} {
+    test {LPUSH, RPUSH, LLENGTH, LINDEX, LPOP - regular list} {
         # first lpush then rpush
         assert_equal 1 [r lpush mylist1 $largevalue(linkedlist)]
         assert_encoding linkedlist mylist1
@@ -39,6 +45,9 @@ start_server {
         assert_equal $largevalue(linkedlist) [r lindex mylist1 0]
         assert_equal b [r lindex mylist1 1]
         assert_equal c [r lindex mylist1 2]
+        assert_equal {} [r lindex mylist1 3]
+        assert_equal c [r rpop mylist1]
+        assert_equal $largevalue(linkedlist) [r lpop mylist1]
 
         # first rpush then lpush
         assert_equal 1 [r rpush mylist2 $largevalue(linkedlist)]
@@ -49,8 +58,15 @@ start_server {
         assert_equal c [r lindex mylist2 0]
         assert_equal b [r lindex mylist2 1]
         assert_equal $largevalue(linkedlist) [r lindex mylist2 2]
+        assert_equal {} [r lindex mylist2 3]
+        assert_equal $largevalue(linkedlist) [r rpop mylist2]
+        assert_equal c [r lpop mylist2]
     }
 
+    test {R/LPOP against empty list} {
+        r lpop non-existing-list
+    } {}
+
     test {Variadic RPUSH/LPUSH} {
         r del mylist
         assert_equal 4 [r lpush mylist a b c d]
@@ -396,6 +412,11 @@ start_server {
         }
     }
 
+    test {LINSERT raise error on bad syntax} {
+        catch {[r linsert xlist aft3r aa 42]} e
+        set e
+    } {*ERR*syntax*error*}
+
     test {LPUSHX, RPUSHX convert from ziplist to list} {
         set large $largevalue(linkedlist)
 

tests/unit/type/set.tcl

@@ -206,6 +206,13 @@ start_server {
         }
     }
 
+    test "SDIFF with first set empty" {
+        r del set1 set2 set3
+        r sadd set2 1 2 3 4
+        r sadd set3 a b c d
+        r sdiff set1 set2 set3
+    } {}
+
     test "SINTER against non-set should throw error" {
         r set key1 x
         assert_error "ERR*wrong kind*" {r sinter key1 noset}
@@ -216,6 +223,23 @@ start_server {
         assert_error "ERR*wrong kind*" {r sunion key1 noset}
     }
 
+    test "SINTER should handle non existing key as empty" {
+        r del set1 set2 set3
+        r sadd set1 a b c
+        r sadd set2 b c d
+        r sinter set1 set2 set3
+    } {}
+
+    test "SINTER with same integer elements but different encoding" {
+        r del set1 set2
+        r sadd set1 1 2 3
+        r sadd set2 1 2 3 a
+        r srem set2 a
+        assert_encoding intset set1
+        assert_encoding hashtable set2
+        lsort [r sinter set1 set2]
+    } {1 2 3}
+
     test "SINTERSTORE against non existing keys should delete dstkey" {
         r set setres xxx
         assert_equal 0 [r sinterstore setres foo111 bar222]
@@ -317,6 +341,13 @@ start_server {
         assert_error "ERR*wrong kind*" {r smove myset2 x foo}
     }
 
+    test "SMOVE with identical source and destination" {
+        r del set
+        r sadd set a b c
+        r smove set set b
+        lsort [r smembers set]
+    } {a b c}
+
     tags {slow} {
         test {intsets implementation stress testing} {
             for {set j 0} {$j < 20} {incr j} {

utils/generate-command-help.rb

@@ -10,7 +10,8 @@ GROUPS = [
   "pubsub",
   "transactions",
   "connection",
-  "server"
+  "server",
+  "scripting"
 ].freeze
 
 GROUPS_BY_NAME = Hash[*
@@ -48,7 +49,7 @@ def commands
   require "json"
   require "uri"
 
-  url = URI.parse "https://github.com/antirez/redis-doc/raw/master/commands.json"
+  url = URI.parse "https://raw.github.com/antirez/redis-doc/master/commands.json"
   client = Net::HTTP.new url.host, url.port
   client.use_ssl = true
   response = client.get url.path

utils/install_server.sh

@@ -1,4 +1,4 @@
-#! /bin/sh
+#!/bin/bash
 
 # Copyright 2011 Dvir Volk <dvirsk at gmail dot com>. All rights reserved.
 #
@@ -166,7 +166,7 @@ if [[ ! `which chkconfig` ]] ; then
 else
 	# we're chkconfig, so lets add to chkconfig and put in runlevel 345
 	chkconfig --add redis_$REDIS_PORT && echo "Successfully added to chkconfig!"
-	chkconfig--level 345 redis_$REDIS_PORT on && echo "Successfully added to runlevels 345!"
+	chkconfig --level 345 redis_$REDIS_PORT on && echo "Successfully added to runlevels 345!"
 fi
 	
 /etc/init.d/redis_$REDIS_PORT start || die "Failed starting service..."

utils/redis.conf.tpl

@@ -1,6 +1,6 @@
 # Redis configuration file example
 
-# Note on units: when memory size is needed, it is possible to specifiy
+# Note on units: when memory size is needed, it is possible to specify
 # it in the usual form of 1k 5GB 4M and so forth:
 #
 # 1k => 1000 bytes
@@ -34,9 +34,10 @@ port $REDIS_PORT
 # on a unix socket when not specified.
 #
 # unixsocket /tmp/redis.sock
+# unixsocketperm 755
 
 # Close the connection after a client is idle for N seconds (0 to disable)
-timeout 300
+timeout 0
 
 # Set server verbosity to 'debug'
 # it can be one of:
@@ -44,7 +45,7 @@ timeout 300
 # verbose (many rarely useful info, but not a mess like the debug level)
 # notice (moderately verbose, what you want in production probably)
 # warning (only very important / critical messages are logged)
-loglevel verbose
+loglevel notice
 
 # Specify the log file name. Also 'stdout' can be used to force
 # Redis to log on the standard output. Note that if you use standard
@@ -81,11 +82,32 @@ databases 16
 #   after 60 sec if at least 10000 keys changed
 #
 #   Note: you can disable saving at all commenting all the "save" lines.
+#
+#   It is also possible to remove all the previously configured save
+#   points by adding a save directive with a single empty string argument
+#   like in the following example:
+#
+#   save ""
 
 save 900 1
 save 300 10
 save 60 10000
 
+# By default Redis will stop accepting writes if RDB snapshots are enabled
+# (at least one save point) and the latest background save failed.
+# This will make the user aware (in an hard way) that data is not persisting
+# on disk properly, otherwise chances are that no one will notice and some
+# distater will happen.
+#
+# If the background saving process will start working again Redis will
+# automatically allow writes again.
+#
+# However if you have setup your proper monitoring of the Redis server
+# and persistence, you may want to disable this feature so that Redis will
+# continue to work as usually even if there are problems with disk,
+# permissions, and so forth.
+stop-writes-on-bgsave-error yes
+
 # Compress string objects using LZF when dump .rdb databases?
 # For default that's set to 'yes' as it's almost always a win.
 # If you want to save some CPU in the saving child set it to 'no' but
@@ -125,7 +147,7 @@ dir $REDIS_DATA_DIR
 # is still in progress, the slave can act in two different ways:
 #
 # 1) if slave-serve-stale-data is set to 'yes' (the default) the slave will
-#    still reply to client requests, possibly with out of data data, or the
+#    still reply to client requests, possibly with out of date data, or the
 #    data set may just be empty if this is the first synchronization.
 #
 # 2) if slave-serve-stale data is set to 'no' the slave will reply with
@@ -134,6 +156,21 @@ dir $REDIS_DATA_DIR
 #
 slave-serve-stale-data yes
 
+# Slaves send PINGs to server in a predefined interval. It's possible to change
+# this interval with the repl_ping_slave_period option. The default value is 10
+# seconds.
+#
+# repl-ping-slave-period 10
+
+# The following option sets a timeout for both Bulk transfer I/O timeout and
+# master data or ping response timeout. The default value is 60 seconds.
+#
+# It is important to make sure that this value is greater than the value
+# specified for repl-ping-slave-period otherwise a timeout will be detected
+# every time there is low traffic between the master and the slave.
+#
+# repl-timeout 60
+
 ################################## SECURITY ###################################
 
 # Require clients to issue AUTH <PASSWORD> before processing any other
@@ -151,7 +188,7 @@ slave-serve-stale-data yes
 
 # Command renaming.
 #
-# It is possilbe to change the name of dangerous commands in a shared
+# It is possible to change the name of dangerous commands in a shared
 # environment. For instance the CONFIG command may be renamed into something
 # of hard to guess so that it will be still available for internal-use
 # tools but not available for general clients.
@@ -160,37 +197,46 @@ slave-serve-stale-data yes
 #
 # rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
 #
-# It is also possilbe to completely kill a command renaming it into
+# It is also possible to completely kill a command renaming it into
 # an empty string:
 #
 # rename-command CONFIG ""
 
 ################################### LIMITS ####################################
 
-# Set the max number of connected clients at the same time. By default there
-# is no limit, and it's up to the number of file descriptors the Redis process
-# is able to open. The special value '0' means no limits.
+# Set the max number of connected clients at the same time. By default
+# this limit is set to 10000 clients, however if the Redis server is not
+# able ot configure the process file limit to allow for the specified limit
+# the max number of allowed clients is set to the current file limit
+# minus 32 (as Redis reserves a few file descriptors for internal uses).
+#
 # Once the limit is reached Redis will close all the new connections sending
 # an error 'max number of clients reached'.
 #
-# maxclients 128
+# maxclients 10000
 
 # Don't use more memory than the specified amount of bytes.
-# When the memory limit is reached Redis will try to remove keys with an
-# EXPIRE set. It will try to start freeing keys that are going to expire
-# in little time and preserve keys with a longer time to live.
-# Redis will also try to remove objects from free lists if possible.
+# When the memory limit is reached Redis will try to remove keys
+# accordingly to the eviction policy selected (see maxmemmory-policy).
 #
-# If all this fails, Redis will start to reply with errors to commands
-# that will use more memory, like SET, LPUSH, and so on, and will continue
-# to reply to most read-only commands like GET.
+# If Redis can't remove keys according to the policy, or if the policy is
+# set to 'noeviction', Redis will start to reply with errors to commands
+# that would use more memory, like SET, LPUSH, and so on, and will continue
+# to reply to read-only commands like GET.
 #
-# WARNING: maxmemory can be a good idea mainly if you want to use Redis as a
-# 'state' server or cache, not as a real DB. When Redis is used as a real
-# database the memory usage will grow over the weeks, it will be obvious if
-# it is going to use too much memory in the long run, and you'll have the time
-# to upgrade. With maxmemory after the limit is reached you'll start to get
-# errors for write operations, and this may even lead to DB inconsistency.
+# This option is usually useful when using Redis as an LRU cache, or to set
+# an hard memory limit for an instance (using the 'noeviction' policy).
+#
+# WARNING: If you have slaves attached to an instance with maxmemory on,
+# the size of the output buffers needed to feed the slaves are subtracted
+# from the used memory count, so that network problems / resyncs will
+# not trigger a loop where keys are evicted, and in turn the output
+# buffer of slaves is full with DELs of keys evicted triggering the deletion
+# of more keys, and so forth until the database is completely emptied.
+#
+# In short... if you have slaves attached it is suggested that you set a lower
+# limit for maxmemory so that there is some free RAM on the system for slave
+# output buffers (but this is not needed if the policy is 'noeviction').
 #
 # maxmemory <bytes>
 
@@ -200,7 +246,7 @@ slave-serve-stale-data yes
 # volatile-lru -> remove the key with an expire set using an LRU algorithm
 # allkeys-lru -> remove any key accordingly to the LRU algorithm
 # volatile-random -> remove a random key with an expire set
-# allkeys->random -> remove a random key, any key
+# allkeys-random -> remove a random key, any key
 # volatile-ttl -> remove the key with the nearest expire time (minor TTL)
 # noeviction -> don't expire at all, just return an error on write operations
 # 
@@ -260,7 +306,7 @@ appendonly no
 #
 # The default is "everysec" that's usually the right compromise between
 # speed and data safety. It's up to you to understand if you can relax this to
-# "no" that will will let the operating system flush the output buffer when
+# "no" that will let the operating system flush the output buffer when
 # it wants, for better performances (but if you can live with the idea of
 # some data loss consider the default persistence mode that's snapshotting),
 # or on the contrary, use "always" that's very slow but a bit safer than
@@ -284,7 +330,7 @@ appendfsync everysec
 # BGSAVE or BGREWRITEAOF is in progress.
 #
 # This means that while another child is saving the durability of Redis is
-# the same as "appendfsync none", that in pratical terms means that it is
+# the same as "appendfsync none", that in practical terms means that it is
 # possible to lost up to 30 seconds of log in the worst scenario (with the
 # default Linux settings).
 # 
@@ -306,7 +352,7 @@ no-appendfsync-on-rewrite no
 # is useful to avoid rewriting the AOF file even if the percentage increase
 # is reached but it is still pretty small.
 #
-# Specify a precentage of zero in order to disable the automatic AOF
+# Specify a percentage of zero in order to disable the automatic AOF
 # rewrite feature.
 
 auto-aof-rewrite-percentage 100
@@ -315,9 +361,20 @@ auto-aof-rewrite-min-size 64mb
 ################################ LUA SCRIPTING  ###############################
 
 # Max execution time of a Lua script in milliseconds.
-# This prevents that a programming error generating an infinite loop will block
-# your server forever. Set it to 0 or a negative value for unlimited execution.
-#lua-time-limit 60000
+#
+# If the maximum execution time is reached Redis will log that a script is
+# still in execution after the maximum allowed time and will start to
+# reply to queries with an error.
+#
+# When a long running script exceed the maximum execution time only the
+# SCRIPT KILL and SHUTDOWN NOSAVE commands are available. The first can be
+# used to stop a script that did not yet called write commands. The second
+# is the only way to shut down the server in the case a write commands was
+# already issue by the script but the user don't want to wait for the natural
+# termination of the script.
+#
+# Set it to 0 or a negative value for unlimited execution without warnings.
+lua-time-limit 5000
 
 ################################## SLOW LOG ###################################
 
@@ -345,12 +402,11 @@ slowlog-max-len 1024
 
 ############################### ADVANCED CONFIG ###############################
 
-# Hashes are encoded in a special way (much more memory efficient) when they
-# have at max a given numer of elements, and the biggest element does not
-# exceed a given threshold. You can configure this limits with the following
-# configuration directives.
-hash-max-zipmap-entries 512
-hash-max-zipmap-value 64
+# Hashes are encoded using a memory efficient data structure when they have a
+# small number of entries, and the biggest entry does not exceed a given
+# threshold. These thresholds can be configured using the following directives.
+hash-max-ziplist-entries 512
+hash-max-ziplist-value 64
 
 # Similarly to hashes, small lists are also encoded in a special way in order
 # to save a lot of space. The special representation is only used when
@@ -373,9 +429,9 @@ zset-max-ziplist-value 64
 
 # Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in
 # order to help rehashing the main Redis hash table (the one mapping top-level
-# keys to values). The hash table implementation redis uses (see dict.c)
+# keys to values). The hash table implementation Redis uses (see dict.c)
 # performs a lazy rehashing: the more operation you run into an hash table
-# that is rhashing, the more rehashing "steps" are performed, so if the
+# that is rehashing, the more rehashing "steps" are performed, so if the
 # server is idle the rehashing is never complete and some more memory is used
 # by the hash table.
 # 
@@ -391,10 +447,47 @@ zset-max-ziplist-value 64
 # want to free memory asap when possible.
 activerehashing yes
 
+# The client output buffer limits can be used to force disconnection of clients
+# that are not reading data from the server fast enough for some reason (a
+# common reason is that a Pub/Sub client can't consume messages as fast as the
+# publisher can produce them).
+#
+# The limit can be set differently for the three different classes of clients:
+#
+# normal -> normal clients
+# slave  -> slave clients and MONITOR clients
+# pubsub -> clients subcribed to at least one pubsub channel or pattern
+#
+# The syntax of every client-output-buffer-limit directive is the following:
+#
+# client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>
+#
+# A client is immediately disconnected once the hard limit is reached, or if
+# the soft limit is reached and remains reached for the specified number of
+# seconds (continuously).
+# So for instance if the hard limit is 32 megabytes and the soft limit is
+# 16 megabytes / 10 seconds, the client will get disconnected immediately
+# if the size of the output buffers reach 32 megabytes, but will also get
+# disconnected if the client reaches 16 megabytes and continuously overcomes
+# the limit for 10 seconds.
+#
+# By default normal clients are not limited because they don't receive data
+# without asking (in a push way), but just after a request, so only
+# asynchronous clients may create a scenario where data is requested faster
+# than it can read.
+#
+# Instead there is a default limit for pubsub and slave clients, since
+# subscribers and slaves receive data in a push fashion.
+#
+# Both the hard or the soft limit can be disabled just setting it to zero.
+client-output-buffer-limit normal 0 0 0
+client-output-buffer-limit slave 256mb 64mb 60
+client-output-buffer-limit pubsub 32mb 8mb 60
+
 ################################## INCLUDES ###################################
 
 # Include one or more other config files here.  This is useful if you
-# have a standard template that goes to all redis server but also need
+# have a standard template that goes to all Redis server but also need
 # to customize a few per-server settings.  Include files can include
 # other files, so use this wisely.
 #