Initial changes to issue #2427.

Cluster lock fix

COPYING

@@ -1,4 +1,4 @@
-Copyright (c) 2006-2014, Salvatore Sanfilippo
+Copyright (c) 2006-2015, Salvatore Sanfilippo
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

README.md

@@ -1,18 +1,35 @@
-Where to find complete Redis documentation?
--------------------------------------------
+This README is just a fast *quick start* document. You can find more detailed documentation at http://redis.io.
 
-This README is just a fast "quick start" document. You can find more detailed
-documentation at http://redis.io
+What is Redis?
+--------------
+
+Redis is often referred as a *data structures* server. What this means is that Redis provides access to mutable data structures via a set of commands, which are sent using a *server-client* model with TCP sockets and a simple protocol. So different processes can query and modify the same data structures in a shared way.
+
+Data structures implemented into Redis have a few special properties:
+
+* Redis cares to store them on disk, even if they are always served and modified into the server memory. This means that Redis is fast, but that is also non-volatile.
+* Implementation of data structures stress on memory efficiency, so data structures inside Redis will likely use less memory compared to the same data structure modeled using an high level programming language.
+* Redis offers a number of features that are natural to find in a database, like replication, tunable levels of durability, cluster, high availability.
+
+Another good example is to think of Redis as a more complex version of memcached, where the operations are not just SETs and GETs, but operations to work with complex data types like Lists, Sets, ordered data structures, and so forth.
+
+If you want to know more, this is a list of selected starting points:
+
+* Introduction to Redis data types. http://redis.io/topics/data-types-intro
+* Try Redis directly inside your browser. http://try.redis.io
+* The full list of Redis commands. http://redis.io/commands
+* There is much more inside the Redis official documentation. http://redis.io/documentation
 
 Building Redis
 --------------
 
 Redis can be compiled and used on Linux, OSX, OpenBSD, NetBSD, FreeBSD.
-We support big endian and little endian architectures.
+We support big endian and little endian architectures, and both 32 bit
+and 64 bit systems.
 
 It may compile on Solaris derived systems (for instance SmartOS) but our
-support for this platform is "best effort" and Redis is not guaranteed to
-work as well as in Linux, OSX, and *BSD there.
+support for this platform is *best effort* and Redis is not guaranteed to
+work as well as in Linux, OSX, and \*BSD there.
 
 It is as simple as:
 
@@ -27,9 +44,10 @@ After building Redis is a good idea to test it, using:
     % make test
 
 Fixing build problems with dependencies or cached build options
-—--------
-Redis has some dependencies which are included into the "deps" directory.
-"make" does not rebuild dependencies automatically, even if something in the
+---------
+
+Redis has some dependencies which are included into the `deps` directory.
+`make` does not rebuild dependencies automatically, even if something in the
 source code of dependencies is changes.
 
 When you update the source code with `git pull` or when code inside the
@@ -42,7 +60,7 @@ This will clean: jemalloc, lua, hiredis, linenoise.
 
 Also if you force certain build options like 32bit target, no C compiler
 optimizations (for debugging purposes), and other similar build time options,
-those options are cached indefinitely until you issue a "make distclean"
+those options are cached indefinitely until you issue a `make distclean`
 command.
 
 Fixing problems building 32 bit binaries
@@ -50,15 +68,14 @@ Fixing problems building 32 bit binaries
 
 If after building Redis with a 32 bit target you need to rebuild it
 with a 64 bit target, or the other way around, you need to perform a
-"make distclean" in the root directory of the Redis distribution.
+`make distclean` in the root directory of the Redis distribution.
 
 In case of build errors when trying to build a 32 bit binary of Redis, try
 the following steps:
 
 * Install the packages libc6-dev-i386 (also try g++-multilib).
-* Try using the following command line instead of "make 32bit":
-
-    make CFLAGS="-m32 -march=native" LDFLAGS="-m32"
+* Try using the following command line instead of `make 32bit`:
+  `make CFLAGS="-m32 -march=native" LDFLAGS="-m32"`
 
 Allocator
 ---------
@@ -126,11 +143,9 @@ then in another terminal try the following:
     (integer) 1
     redis> incr mycounter
     (integer) 2
-    redis> 
+    redis>
 
-You can find the list of all the available commands here:
-
-    http://redis.io/commands
+You can find the list of all the available commands at http://redis.io/commands.
 
 Installing Redis
 -----------------
@@ -139,7 +154,7 @@ In order to install Redis binaries into /usr/local/bin just use:
 
     % make install
 
-You can use "make PREFIX=/some/other/directory install" if you wish to use a
+You can use `make PREFIX=/some/other/directory install` if you wish to use a
 different destination.
 
 Make install will just install binaries in your system, but will not configure
@@ -156,7 +171,7 @@ to run Redis properly as a background daemon that will start again on
 system reboots.
 
 You'll be able to stop and start Redis using the script named
-/etc/init.d/redis_<portnumber>, for instance /etc/init.d/redis_6379.
+`/etc/init.d/redis_<portnumber>`, for instance `/etc/init.d/redis_6379`.
 
 Code contributions
 ---
@@ -164,10 +179,13 @@ Code contributions
 Note: by contributing code to the Redis project in any form, including sending
 a pull request via Github, a code fragment or patch via private email or
 public discussion groups, you agree to release your code under the terms
-of the BSD license that you can find in the COPYING file included in the Redis
+of the BSD license that you can find in the [COPYING][1] file included in the Redis
 source distribution.
 
-Please see the CONTRIBUTING file in this source distribution for more
+Please see the [CONTRIBUTING][2] file in this source distribution for more
 information.
 
 Enjoy!
+
+[1]: https://github.com/antirez/redis/blob/unstable/COPYING
+[2]: https://github.com/antirez/redis/blob/unstable/CONTRIBUTING

deps/Makefile

@@ -36,6 +36,7 @@ distclean:
 	-(cd hiredis && $(MAKE) clean) > /dev/null || true
 	-(cd linenoise && $(MAKE) clean) > /dev/null || true
 	-(cd lua && $(MAKE) clean) > /dev/null || true
+	-(cd geohash-int && $(MAKE) clean) > /dev/null || true
 	-(cd jemalloc && [ -f Makefile ] && $(MAKE) distclean) > /dev/null || true
 	-(rm -f .make-*)
 
@@ -81,3 +82,9 @@ jemalloc: .make-prerequisites
 	cd jemalloc && $(MAKE) CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)" lib/libjemalloc.a
 
 .PHONY: jemalloc
+
+geohash-int: .make-prerequisites
+	@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
+	cd geohash-int && $(MAKE)
+
+.PHONY: geohash-int

deps/geohash-int/Makefile

@@ -0,0 +1,23 @@
+STD=
+WARN= -Wall
+OPT= -O2
+
+R_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS)
+R_LDFLAGS= $(LDFLAGS)
+DEBUG= -g
+
+R_CC=$(CC) $(R_CFLAGS)
+R_LD=$(CC) $(R_LDFLAGS)
+
+all: geohash.o geohash_helper.o
+
+.PHONY: all
+
+geohash.o: geohash.h geohash.c
+geohash_helper.o: geohash.h geohash_helper.h geohash_helper.c
+
+.c.o:
+	$(R_CC) -c $<
+
+clean:
+	rm -f *.o

deps/geohash-int/geohash.c

@@ -0,0 +1,295 @@
+/*
+ * Copyright (c) 2013-2014, yinqiwen <yinqiwen@gmail.com>
+ * Copyright (c) 2014, Matt Stancliff <matt@genges.com>.
+ * Copyright (c) 2015, Salvatore Sanfilippo <antirez@gmail.com>.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ *  * Redistributions of source code must retain the above copyright notice,
+ *    this list of conditions and the following disclaimer.
+ *  * Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *  * Neither the name of Redis nor the names of its contributors may be used
+ *    to endorse or promote products derived from this software without
+ *    specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include "geohash.h"
+
+/**
+ * Hashing works like this:
+ * Divide the world into 4 buckets.  Label each one as such:
+ *  -----------------
+ *  |       |       |
+ *  |       |       |
+ *  | 0,1   | 1,1   |
+ *  -----------------
+ *  |       |       |
+ *  |       |       |
+ *  | 0,0   | 1,0   |
+ *  -----------------
+ */
+
+/* Interleave lower bits of x and y, so the bits of x
+ * are in the even positions and bits from y in the odd;
+ * x and y must initially be less than 2**32 (65536).
+ * From:  https://graphics.stanford.edu/~seander/bithacks.html#InterleaveBMN
+ */
+static inline uint64_t interleave64(uint32_t xlo, uint32_t ylo) {
+    static const uint64_t B[] = {0x5555555555555555ULL, 0x3333333333333333ULL,
+                                 0x0F0F0F0F0F0F0F0FULL, 0x00FF00FF00FF00FFULL,
+                                 0x0000FFFF0000FFFFULL};
+    static const unsigned int S[] = {1, 2, 4, 8, 16};
+
+    uint64_t x = xlo;
+    uint64_t y = ylo;
+
+    x = (x | (x << S[4])) & B[4];
+    y = (y | (y << S[4])) & B[4];
+
+    x = (x | (x << S[3])) & B[3];
+    y = (y | (y << S[3])) & B[3];
+
+    x = (x | (x << S[2])) & B[2];
+    y = (y | (y << S[2])) & B[2];
+
+    x = (x | (x << S[1])) & B[1];
+    y = (y | (y << S[1])) & B[1];
+
+    x = (x | (x << S[0])) & B[0];
+    y = (y | (y << S[0])) & B[0];
+
+    return x | (y << 1);
+}
+
+/* reverse the interleave process
+ * derived from http://stackoverflow.com/questions/4909263
+ */
+static inline uint64_t deinterleave64(uint64_t interleaved) {
+    static const uint64_t B[] = {0x5555555555555555ULL, 0x3333333333333333ULL,
+                                 0x0F0F0F0F0F0F0F0FULL, 0x00FF00FF00FF00FFULL,
+                                 0x0000FFFF0000FFFFULL, 0x00000000FFFFFFFFULL};
+    static const unsigned int S[] = {0, 1, 2, 4, 8, 16};
+
+    uint64_t x = interleaved;
+    uint64_t y = interleaved >> 1;
+
+    x = (x | (x >> S[0])) & B[0];
+    y = (y | (y >> S[0])) & B[0];
+
+    x = (x | (x >> S[1])) & B[1];
+    y = (y | (y >> S[1])) & B[1];
+
+    x = (x | (x >> S[2])) & B[2];
+    y = (y | (y >> S[2])) & B[2];
+
+    x = (x | (x >> S[3])) & B[3];
+    y = (y | (y >> S[3])) & B[3];
+
+    x = (x | (x >> S[4])) & B[4];
+    y = (y | (y >> S[4])) & B[4];
+
+    x = (x | (x >> S[5])) & B[5];
+    y = (y | (y >> S[5])) & B[5];
+
+    return x | (y << 32);
+}
+
+void geohashGetCoordRange(GeoHashRange *long_range, GeoHashRange *lat_range) {
+    /* These are constraints from EPSG:900913 / EPSG:3785 / OSGEO:41001 */
+    /* We can't geocode at the north/south pole. */
+    long_range->max = GEO_LONG_MAX;
+    long_range->min = GEO_LONG_MIN;
+    lat_range->max = GEO_LAT_MAX;
+    lat_range->min = GEO_LAT_MIN;
+}
+
+int geohashEncode(GeoHashRange *long_range, GeoHashRange *lat_range,
+                  double longitude, double latitude, uint8_t step,
+                  GeoHashBits *hash) {
+    /* Check basic arguments sanity. */
+    if (hash == NULL || step > 32 || step == 0 ||
+        RANGEPISZERO(lat_range) || RANGEPISZERO(long_range)) return 0;
+
+    /* Return an error when trying to index outside the supported
+     * constraints. */
+    if (longitude > 180 || longitude < -180 ||
+        latitude > 85.05112878 || latitude < -85.05112878) return 0;
+
+    hash->bits = 0;
+    hash->step = step;
+
+    if (latitude < lat_range->min || latitude > lat_range->max ||
+        longitude < long_range->min || longitude > long_range->max) {
+        return 0;
+    }
+
+    double lat_offset =
+        (latitude - lat_range->min) / (lat_range->max - lat_range->min);
+    double long_offset =
+        (longitude - long_range->min) / (long_range->max - long_range->min);
+
+    /* convert to fixed point based on the step size */
+    lat_offset *= (1 << step);
+    long_offset *= (1 << step);
+    hash->bits = interleave64(lat_offset, long_offset);
+    return 1;
+}
+
+int geohashEncodeType(double longitude, double latitude, uint8_t step, GeoHashBits *hash) {
+    GeoHashRange r[2] = { { 0 } };
+    geohashGetCoordRange(&r[0], &r[1]);
+    return geohashEncode(&r[0], &r[1], longitude, latitude, step, hash);
+}
+
+int geohashEncodeWGS84(double longitude, double latitude, uint8_t step,
+                       GeoHashBits *hash) {
+    return geohashEncodeType(longitude, latitude, step, hash);
+}
+
+int geohashDecode(const GeoHashRange long_range, const GeoHashRange lat_range,
+                   const GeoHashBits hash, GeoHashArea *area) {
+    if (HASHISZERO(hash) || NULL == area || RANGEISZERO(lat_range) ||
+        RANGEISZERO(long_range)) {
+        return 0;
+    }
+
+    area->hash = hash;
+    uint8_t step = hash.step;
+    uint64_t hash_sep = deinterleave64(hash.bits); /* hash = [LAT][LONG] */
+
+    double lat_scale = lat_range.max - lat_range.min;
+    double long_scale = long_range.max - long_range.min;
+
+    uint32_t ilato = hash_sep;       /* get lat part of deinterleaved hash */
+    uint32_t ilono = hash_sep >> 32; /* shift over to get long part of hash */
+
+    /* divide by 2**step.
+     * Then, for 0-1 coordinate, multiply times scale and add
+       to the min to get the absolute coordinate. */
+    area->latitude.min =
+        lat_range.min + (ilato * 1.0 / (1ull << step)) * lat_scale;
+    area->latitude.max =
+        lat_range.min + ((ilato + 1) * 1.0 / (1ull << step)) * lat_scale;
+    area->longitude.min =
+        long_range.min + (ilono * 1.0 / (1ull << step)) * long_scale;
+    area->longitude.max =
+        long_range.min + ((ilono + 1) * 1.0 / (1ull << step)) * long_scale;
+
+    return 1;
+}
+
+int geohashDecodeType(const GeoHashBits hash, GeoHashArea *area) {
+    GeoHashRange r[2] = { { 0 } };
+    geohashGetCoordRange(&r[0], &r[1]);
+    return geohashDecode(r[0], r[1], hash, area);
+}
+
+int geohashDecodeWGS84(const GeoHashBits hash, GeoHashArea *area) {
+    return geohashDecodeType(hash, area);
+}
+
+int geohashDecodeAreaToLongLat(const GeoHashArea *area, double *xy) {
+    if (!xy) return 0;
+    xy[0] = (area->longitude.min + area->longitude.max) / 2;
+    xy[1] = (area->latitude.min + area->latitude.max) / 2;
+    return 1;
+}
+
+int geohashDecodeToLongLatType(const GeoHashBits hash, double *xy) {
+    GeoHashArea area = { { 0 } };
+    if (!xy || !geohashDecodeType(hash, &area))
+        return 0;
+    return geohashDecodeAreaToLongLat(&area, xy);
+}
+
+int geohashDecodeToLongLatWGS84(const GeoHashBits hash, double *xy) {
+    return geohashDecodeToLongLatType(hash, xy);
+}
+
+static void geohash_move_x(GeoHashBits *hash, int8_t d) {
+    if (d == 0)
+        return;
+
+    uint64_t x = hash->bits & 0xaaaaaaaaaaaaaaaaULL;
+    uint64_t y = hash->bits & 0x5555555555555555ULL;
+
+    uint64_t zz = 0x5555555555555555ULL >> (64 - hash->step * 2);
+
+    if (d > 0) {
+        x = x + (zz + 1);
+    } else {
+        x = x | zz;
+        x = x - (zz + 1);
+    }
+
+    x &= (0xaaaaaaaaaaaaaaaaULL >> (64 - hash->step * 2));
+    hash->bits = (x | y);
+}
+
+static void geohash_move_y(GeoHashBits *hash, int8_t d) {
+    if (d == 0)
+        return;
+
+    uint64_t x = hash->bits & 0xaaaaaaaaaaaaaaaaULL;
+    uint64_t y = hash->bits & 0x5555555555555555ULL;
+
+    uint64_t zz = 0xaaaaaaaaaaaaaaaaULL >> (64 - hash->step * 2);
+    if (d > 0) {
+        y = y + (zz + 1);
+    } else {
+        y = y | zz;
+        y = y - (zz + 1);
+    }
+    y &= (0x5555555555555555ULL >> (64 - hash->step * 2));
+    hash->bits = (x | y);
+}
+
+void geohashNeighbors(const GeoHashBits *hash, GeoHashNeighbors *neighbors) {
+    neighbors->east = *hash;
+    neighbors->west = *hash;
+    neighbors->north = *hash;
+    neighbors->south = *hash;
+    neighbors->south_east = *hash;
+    neighbors->south_west = *hash;
+    neighbors->north_east = *hash;
+    neighbors->north_west = *hash;
+
+    geohash_move_x(&neighbors->east, 1);
+    geohash_move_y(&neighbors->east, 0);
+
+    geohash_move_x(&neighbors->west, -1);
+    geohash_move_y(&neighbors->west, 0);
+
+    geohash_move_x(&neighbors->south, 0);
+    geohash_move_y(&neighbors->south, -1);
+
+    geohash_move_x(&neighbors->north, 0);
+    geohash_move_y(&neighbors->north, 1);
+
+    geohash_move_x(&neighbors->north_west, -1);
+    geohash_move_y(&neighbors->north_west, 1);
+
+    geohash_move_x(&neighbors->north_east, 1);
+    geohash_move_y(&neighbors->north_east, 1);
+
+    geohash_move_x(&neighbors->south_east, 1);
+    geohash_move_y(&neighbors->south_east, -1);
+
+    geohash_move_x(&neighbors->south_west, -1);
+    geohash_move_y(&neighbors->south_west, -1);
+}

deps/geohash-int/geohash.h

@@ -0,0 +1,118 @@
+/*
+ * Copyright (c) 2013-2014, yinqiwen <yinqiwen@gmail.com>
+ * Copyright (c) 2014, Matt Stancliff <matt@genges.com>.
+ * Copyright (c) 2015, Salvatore Sanfilippo <antirez@gmail.com>.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ *  * Redistributions of source code must retain the above copyright notice,
+ *    this list of conditions and the following disclaimer.
+ *  * Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *  * Neither the name of Redis nor the names of its contributors may be used
+ *    to endorse or promote products derived from this software without
+ *    specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef GEOHASH_H_
+#define GEOHASH_H_
+
+#include <stddef.h>
+#include <stdint.h>
+#include <stdint.h>
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+#define HASHISZERO(r) (!(r).bits && !(r).step)
+#define RANGEISZERO(r) (!(r).max && !(r).min)
+#define RANGEPISZERO(r) (r == NULL || RANGEISZERO(*r))
+
+#define GEO_STEP_MAX 26 /* 26*2 = 52 bits. */
+
+/* Limits from EPSG:900913 / EPSG:3785 / OSGEO:41001 */
+#define GEO_LAT_MIN -85.05112878
+#define GEO_LAT_MAX 85.05112878
+#define GEO_LONG_MIN -180
+#define GEO_LONG_MAX 180
+
+typedef enum {
+    GEOHASH_NORTH = 0,
+    GEOHASH_EAST,
+    GEOHASH_WEST,
+    GEOHASH_SOUTH,
+    GEOHASH_SOUTH_WEST,
+    GEOHASH_SOUTH_EAST,
+    GEOHASH_NORT_WEST,
+    GEOHASH_NORT_EAST
+} GeoDirection;
+
+typedef struct {
+    uint64_t bits;
+    uint8_t step;
+} GeoHashBits;
+
+typedef struct {
+    double min;
+    double max;
+} GeoHashRange;
+
+typedef struct {
+    GeoHashBits hash;
+    GeoHashRange longitude;
+    GeoHashRange latitude;
+} GeoHashArea;
+
+typedef struct {
+    GeoHashBits north;
+    GeoHashBits east;
+    GeoHashBits west;
+    GeoHashBits south;
+    GeoHashBits north_east;
+    GeoHashBits south_east;
+    GeoHashBits north_west;
+    GeoHashBits south_west;
+} GeoHashNeighbors;
+
+/*
+ * 0:success
+ * -1:failed
+ */
+void geohashGetCoordRange(GeoHashRange *long_range, GeoHashRange *lat_range);
+int geohashEncode(GeoHashRange *long_range, GeoHashRange *lat_range,
+                  double longitude, double latitude, uint8_t step,
+                  GeoHashBits *hash);
+int geohashEncodeType(double longitude, double latitude,
+                      uint8_t step, GeoHashBits *hash);
+int geohashEncodeWGS84(double longitude, double latitude, uint8_t step,
+                       GeoHashBits *hash);
+int geohashDecode(const GeoHashRange long_range, const GeoHashRange lat_range,
+                  const GeoHashBits hash, GeoHashArea *area);
+int geohashDecodeType(const GeoHashBits hash, GeoHashArea *area);
+int geohashDecodeWGS84(const GeoHashBits hash, GeoHashArea *area);
+int geohashDecodeAreaToLongLat(const GeoHashArea *area, double *xy);
+int geohashDecodeToLongLatType(const GeoHashBits hash, double *xy);
+int geohashDecodeToLongLatWGS84(const GeoHashBits hash, double *xy);
+int geohashDecodeToLongLatMercator(const GeoHashBits hash, double *xy);
+void geohashNeighbors(const GeoHashBits *hash, GeoHashNeighbors *neighbors);
+
+#if defined(__cplusplus)
+}
+#endif
+#endif /* GEOHASH_H_ */

deps/geohash-int/geohash_helper.c

@@ -0,0 +1,195 @@
+/*
+ * Copyright (c) 2013-2014, yinqiwen <yinqiwen@gmail.com>
+ * Copyright (c) 2014, Matt Stancliff <matt@genges.com>.
+ * Copyright (c) 2015, Salvatore Sanfilippo <antirez@gmail.com>.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ *  * Redistributions of source code must retain the above copyright notice,
+ *    this list of conditions and the following disclaimer.
+ *  * Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *  * Neither the name of Redis nor the names of its contributors may be used
+ *    to endorse or promote products derived from this software without
+ *    specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/* This is a C++ to C conversion from the ardb project.
+ * This file started out as:
+ * https://github.com/yinqiwen/ardb/blob/d42503/src/geo/geohash_helper.cpp
+ */
+
+#include "geohash_helper.h"
+
+#define D_R (M_PI / 180.0)
+#define R_MAJOR 6378137.0
+#define R_MINOR 6356752.3142
+#define RATIO (R_MINOR / R_MAJOR)
+#define ECCENT (sqrt(1.0 - (RATIO *RATIO)))
+#define COM (0.5 * ECCENT)
+
+/// @brief The usual PI/180 constant
+const double DEG_TO_RAD = 0.017453292519943295769236907684886;
+/// @brief Earth's quatratic mean radius for WGS-84
+const double EARTH_RADIUS_IN_METERS = 6372797.560856;
+
+const double MERCATOR_MAX = 20037726.37;
+const double MERCATOR_MIN = -20037726.37;
+
+static inline double deg_rad(double ang) { return ang * D_R; }
+static inline double rad_deg(double ang) { return ang / D_R; }
+
+/* You must *ONLY* estimate steps when you are encoding.
+ * If you are decoding, always decode to GEO_STEP_MAX (26). */
+uint8_t geohashEstimateStepsByRadius(double range_meters, double lat) {
+    if (range_meters == 0) return 26;
+    int step = 1;
+    while (range_meters < MERCATOR_MAX) {
+        range_meters *= 2;
+        step++;
+    }
+    step -= 2; /* Make sure range is included in the worst case. */
+    /* Wider range torwards the poles... Note: it is possible to do better
+     * than this approximation by computing the distance between meridians
+     * at this latitude, but this does the trick for now. */
+    if (lat > 67 || lat < -67) step--;
+    if (lat > 80 || lat < -80) step--;
+
+    /* Frame to valid range. */
+    if (step < 1) step = 1;
+    if (step > 26) step = 25;
+    return step;
+}
+
+int geohashBitsComparator(const GeoHashBits *a, const GeoHashBits *b) {
+    /* If step not equal, compare on step.  Else, compare on bits. */
+    return a->step != b->step ? a->step - b->step : a->bits - b->bits;
+}
+
+int geohashBoundingBox(double longitude, double latitude, double radius_meters,
+                       double *bounds) {
+    if (!bounds) return 0;
+
+    double lonr, latr;
+    lonr = deg_rad(longitude);
+    latr = deg_rad(latitude);
+
+    double distance = radius_meters / EARTH_RADIUS_IN_METERS;
+    double min_latitude = latr - distance;
+    double max_latitude = latr + distance;
+
+    /* Note: we're being lazy and not accounting for coordinates near poles */
+    double min_longitude, max_longitude;
+    double difference_longitude = asin(sin(distance) / cos(latr));
+    min_longitude = lonr - difference_longitude;
+    max_longitude = lonr + difference_longitude;
+
+    bounds[0] = rad_deg(min_longitude);
+    bounds[1] = rad_deg(min_latitude);
+    bounds[2] = rad_deg(max_longitude);
+    bounds[3] = rad_deg(max_latitude);
+    return 1;
+}
+
+GeoHashRadius geohashGetAreasByRadius(double longitude, double latitude, double radius_meters) {
+    GeoHashRange long_range, lat_range;
+    GeoHashRadius radius = { { 0 } };
+    GeoHashBits hash = { 0 };
+    GeoHashNeighbors neighbors = { { 0 } };
+    GeoHashArea area = { { 0 } };
+    double min_lon, max_lon, min_lat, max_lat;
+    double bounds[4];
+    int steps;
+
+    geohashBoundingBox(longitude, latitude, radius_meters, bounds);
+    min_lon = bounds[0];
+    min_lat = bounds[1];
+    max_lon = bounds[2];
+    max_lat = bounds[3];
+
+    steps = geohashEstimateStepsByRadius(radius_meters,latitude);
+
+    geohashGetCoordRange(&long_range, &lat_range);
+    geohashEncode(&long_range, &lat_range, longitude, latitude, steps, &hash);
+    geohashNeighbors(&hash, &neighbors);
+    geohashGetCoordRange(&long_range, &lat_range);
+    geohashDecode(long_range, lat_range, hash, &area);
+
+    if (area.latitude.min < min_lat) {
+        GZERO(neighbors.south);
+        GZERO(neighbors.south_west);
+        GZERO(neighbors.south_east);
+    }
+    if (area.latitude.max > max_lat) {
+        GZERO(neighbors.north);
+        GZERO(neighbors.north_east);
+        GZERO(neighbors.north_west);
+    }
+    if (area.longitude.min < min_lon) {
+        GZERO(neighbors.west);
+        GZERO(neighbors.south_west);
+        GZERO(neighbors.north_west);
+    }
+    if (area.longitude.max > max_lon) {
+        GZERO(neighbors.east);
+        GZERO(neighbors.south_east);
+        GZERO(neighbors.north_east);
+    }
+    radius.hash = hash;
+    radius.neighbors = neighbors;
+    radius.area = area;
+    return radius;
+}
+
+GeoHashRadius geohashGetAreasByRadiusWGS84(double longitude, double latitude,
+                                           double radius_meters) {
+    return geohashGetAreasByRadius(longitude, latitude, radius_meters);
+}
+
+GeoHashFix52Bits geohashAlign52Bits(const GeoHashBits hash) {
+    uint64_t bits = hash.bits;
+    bits <<= (52 - hash.step * 2);
+    return bits;
+}
+
+/* Calculate distance using haversin great circle distance formula. */
+double geohashGetDistance(double lon1d, double lat1d, double lon2d, double lat2d) {
+    double lat1r, lon1r, lat2r, lon2r, u, v;
+    lat1r = deg_rad(lat1d);
+    lon1r = deg_rad(lon1d);
+    lat2r = deg_rad(lat2d);
+    lon2r = deg_rad(lon2d);
+    u = sin((lat2r - lat1r) / 2);
+    v = sin((lon2r - lon1r) / 2);
+    return 2.0 * EARTH_RADIUS_IN_METERS *
+           asin(sqrt(u * u + cos(lat1r) * cos(lat2r) * v * v));
+}
+
+int geohashGetDistanceIfInRadius(double x1, double y1,
+                                 double x2, double y2, double radius,
+                                 double *distance) {
+    *distance = geohashGetDistance(x1, y1, x2, y2);
+    if (*distance > radius) return 0;
+    return 1;
+}
+
+int geohashGetDistanceIfInRadiusWGS84(double x1, double y1, double x2,
+                                      double y2, double radius,
+                                      double *distance) {
+    return geohashGetDistanceIfInRadius(x1, y1, x2, y2, radius, distance);
+}

deps/geohash-int/geohash_helper.h

@@ -0,0 +1,71 @@
+/*
+ * Copyright (c) 2013-2014, yinqiwen <yinqiwen@gmail.com>
+ * Copyright (c) 2014, Matt Stancliff <matt@genges.com>.
+ * Copyright (c) 2015, Salvatore Sanfilippo <antirez@gmail.com>.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ *  * Redistributions of source code must retain the above copyright notice,
+ *    this list of conditions and the following disclaimer.
+ *  * Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *  * Neither the name of Redis nor the names of its contributors may be used
+ *    to endorse or promote products derived from this software without
+ *    specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef GEOHASH_HELPER_HPP_
+#define GEOHASH_HELPER_HPP_
+
+#include <math.h>
+#include "geohash.h"
+
+#define GZERO(s) s.bits = s.step = 0;
+#define GISZERO(s) (!s.bits && !s.step)
+#define GISNOTZERO(s) (s.bits || s.step)
+
+typedef uint64_t GeoHashFix52Bits;
+typedef uint64_t GeoHashVarBits;
+
+typedef struct {
+    GeoHashBits hash;
+    GeoHashArea area;
+    GeoHashNeighbors neighbors;
+} GeoHashRadius;
+
+int GeoHashBitsComparator(const GeoHashBits *a, const GeoHashBits *b);
+uint8_t geohashEstimateStepsByRadius(double range_meters, double lat);
+int geohashBoundingBox(double longitude, double latitude, double radius_meters,
+                        double *bounds);
+GeoHashRadius geohashGetAreasByRadius(double longitude,
+                                      double latitude, double radius_meters);
+GeoHashRadius geohashGetAreasByRadiusWGS84(double longitude, double latitude,
+                                           double radius_meters);
+GeoHashRadius geohashGetAreasByRadiusMercator(double longitude, double latitude,
+                                              double radius_meters);
+GeoHashFix52Bits geohashAlign52Bits(const GeoHashBits hash);
+double geohashGetDistance(double lon1d, double lat1d,
+                          double lon2d, double lat2d);
+int geohashGetDistanceIfInRadius(double x1, double y1,
+                                 double x2, double y2, double radius,
+                                 double *distance);
+int geohashGetDistanceIfInRadiusWGS84(double x1, double y1, double x2,
+                                      double y2, double radius,
+                                      double *distance);
+
+#endif /* GEOHASH_HELPER_HPP_ */

deps/hiredis/async.c

@@ -443,6 +443,7 @@ void redisProcessCallbacks(redisAsyncContext *ac) {
             if (((redisReply*)reply)->type == REDIS_REPLY_ERROR) {
                 c->err = REDIS_ERR_OTHER;
                 snprintf(c->errstr,sizeof(c->errstr),"%s",((redisReply*)reply)->str);
+                c->reader->fn->freeObject(reply);
                 __redisAsyncDisconnect(ac);
                 return;
             }

deps/lua/src/ldo.c

@@ -495,7 +495,7 @@ static void f_parser (lua_State *L, void *ud) {
   struct SParser *p = cast(struct SParser *, ud);
   int c = luaZ_lookahead(p->z);
   luaC_checkGC(L);
-  tf = ((c == LUA_SIGNATURE[0]) ? luaU_undump : luaY_parser)(L, p->z,
+  tf = (luaY_parser)(L, p->z,
                                                              &p->buff, p->name);
   cl = luaF_newLclosure(L, tf->nups, hvalue(gt(L)));
   cl->l.p = tf;

deps/lua/src/lua_cmsgpack.c

@@ -66,7 +66,7 @@
 /* Reverse memory bytes if arch is little endian. Given the conceptual
  * simplicity of the Lua build system we prefer check for endianess at runtime.
  * The performance difference should be acceptable. */
-static void memrevifle(void *ptr, size_t len) {
+void memrevifle(void *ptr, size_t len) {
     unsigned char   *p = (unsigned char *)ptr,
                     *e = (unsigned char *)p+len-1,
                     aux;
@@ -96,7 +96,7 @@ typedef struct mp_buf {
     size_t len, free;
 } mp_buf;
 
-static void *mp_realloc(lua_State *L, void *target, size_t osize,size_t nsize) {
+void *mp_realloc(lua_State *L, void *target, size_t osize,size_t nsize) {
     void *(*local_realloc) (void *, void *, size_t osize, size_t nsize) = NULL;
     void *ud;
 
@@ -105,7 +105,7 @@ static void *mp_realloc(lua_State *L, void *target, size_t osize,size_t nsize) {
     return local_realloc(ud, target, osize, nsize);
 }
 
-static mp_buf *mp_buf_new(lua_State *L) {
+mp_buf *mp_buf_new(lua_State *L) {
     mp_buf *buf = NULL;
 
     /* Old size = 0; new size = sizeof(*buf) */
@@ -117,7 +117,7 @@ static mp_buf *mp_buf_new(lua_State *L) {
     return buf;
 }
 
-static void mp_buf_append(mp_buf *buf, const unsigned char *s, size_t len) {
+void mp_buf_append(mp_buf *buf, const unsigned char *s, size_t len) {
     if (buf->free < len) {
         size_t newlen = buf->len+len;
 
@@ -153,7 +153,7 @@ typedef struct mp_cur {
     int err;
 } mp_cur;
 
-static void mp_cur_init(mp_cur *cursor, const unsigned char *s, size_t len) {
+void mp_cur_init(mp_cur *cursor, const unsigned char *s, size_t len) {
     cursor->p = s;
     cursor->left = len;
     cursor->err = MP_CUR_ERROR_NONE;
@@ -173,13 +173,17 @@ static void mp_cur_init(mp_cur *cursor, const unsigned char *s, size_t len) {
 
 /* ------------------------- Low level MP encoding -------------------------- */
 
-static void mp_encode_bytes(mp_buf *buf, const unsigned char *s, size_t len) {
+void mp_encode_bytes(mp_buf *buf, const unsigned char *s, size_t len) {
     unsigned char hdr[5];
     int hdrlen;
 
     if (len < 32) {
         hdr[0] = 0xa0 | (len&0xff); /* fix raw */
         hdrlen = 1;
+    } else if (len <= 0xff) {
+        hdr[0] = 0xd9;
+        hdr[1] = len;
+        hdrlen = 2;
     } else if (len <= 0xffff) {
         hdr[0] = 0xda;
         hdr[1] = (len&0xff00)>>8;
@@ -198,7 +202,7 @@ static void mp_encode_bytes(mp_buf *buf, const unsigned char *s, size_t len) {
 }
 
 /* we assume IEEE 754 internal format for single and double precision floats. */
-static void mp_encode_double(mp_buf *buf, double d) {
+void mp_encode_double(mp_buf *buf, double d) {
     unsigned char b[9];
     float f = d;
 
@@ -216,7 +220,7 @@ static void mp_encode_double(mp_buf *buf, double d) {
     }
 }
 
-static void mp_encode_int(mp_buf *buf, int64_t n) {
+void mp_encode_int(mp_buf *buf, int64_t n) {
     unsigned char b[9];
     int enclen;
 
@@ -288,7 +292,7 @@ static void mp_encode_int(mp_buf *buf, int64_t n) {
     mp_buf_append(buf,b,enclen);
 }
 
-static void mp_encode_array(mp_buf *buf, int64_t n) {
+void mp_encode_array(mp_buf *buf, int64_t n) {
     unsigned char b[5];
     int enclen;
 
@@ -311,7 +315,7 @@ static void mp_encode_array(mp_buf *buf, int64_t n) {
     mp_buf_append(buf,b,enclen);
 }
 
-static void mp_encode_map(mp_buf *buf, int64_t n) {
+void mp_encode_map(mp_buf *buf, int64_t n) {
     unsigned char b[5];
     int enclen;
 
@@ -336,7 +340,7 @@ static void mp_encode_map(mp_buf *buf, int64_t n) {
 
 /* --------------------------- Lua types encoding --------------------------- */
 
-static void mp_encode_lua_string(lua_State *L, mp_buf *buf) {
+void mp_encode_lua_string(lua_State *L, mp_buf *buf) {
     size_t len;
     const char *s;
 
@@ -344,13 +348,13 @@ static void mp_encode_lua_string(lua_State *L, mp_buf *buf) {
     mp_encode_bytes(buf,(const unsigned char*)s,len);
 }
 
-static void mp_encode_lua_bool(lua_State *L, mp_buf *buf) {
+void mp_encode_lua_bool(lua_State *L, mp_buf *buf) {
     unsigned char b = lua_toboolean(L,-1) ? 0xc3 : 0xc2;
     mp_buf_append(buf,&b,1);
 }
 
 /* Lua 5.3 has a built in 64-bit integer type */
-static void mp_encode_lua_integer(lua_State *L, mp_buf *buf) {
+void mp_encode_lua_integer(lua_State *L, mp_buf *buf) {
 #if (LUA_VERSION_NUM < 503) && BITS_32
     lua_Number i = lua_tonumber(L,-1);
 #else
@@ -362,7 +366,7 @@ static void mp_encode_lua_integer(lua_State *L, mp_buf *buf) {
 /* Lua 5.2 and lower only has 64-bit doubles, so we need to
  * detect if the double may be representable as an int
  * for Lua < 5.3 */
-static void mp_encode_lua_number(lua_State *L, mp_buf *buf) {
+void mp_encode_lua_number(lua_State *L, mp_buf *buf) {
     lua_Number n = lua_tonumber(L,-1);
 
     if (IS_INT64_EQUIVALENT(n)) {
@@ -372,10 +376,10 @@ static void mp_encode_lua_number(lua_State *L, mp_buf *buf) {
     }
 }
 
-static void mp_encode_lua_type(lua_State *L, mp_buf *buf, int level);
+void mp_encode_lua_type(lua_State *L, mp_buf *buf, int level);
 
 /* Convert a lua table into a message pack list. */
-static void mp_encode_lua_table_as_array(lua_State *L, mp_buf *buf, int level) {
+void mp_encode_lua_table_as_array(lua_State *L, mp_buf *buf, int level) {
 #if LUA_VERSION_NUM < 502
     size_t len = lua_objlen(L,-1), j;
 #else
@@ -391,7 +395,7 @@ static void mp_encode_lua_table_as_array(lua_State *L, mp_buf *buf, int level) {
 }
 
 /* Convert a lua table into a message pack key-value map. */
-static void mp_encode_lua_table_as_map(lua_State *L, mp_buf *buf, int level) {
+void mp_encode_lua_table_as_map(lua_State *L, mp_buf *buf, int level) {
     size_t len = 0;
 
     /* First step: count keys into table. No other way to do it with the
@@ -418,7 +422,7 @@ static void mp_encode_lua_table_as_map(lua_State *L, mp_buf *buf, int level) {
 /* Returns true if the Lua table on top of the stack is exclusively composed
  * of keys from numerical keys from 1 up to N, with N being the total number
  * of elements, without any hole in the middle. */
-static int table_is_an_array(lua_State *L) {
+int table_is_an_array(lua_State *L) {
     int count = 0, max = 0;
 #if LUA_VERSION_NUM < 503
     lua_Number n;
@@ -461,14 +465,14 @@ static int table_is_an_array(lua_State *L) {
 /* If the length operator returns non-zero, that is, there is at least
  * an object at key '1', we serialize to message pack list. Otherwise
  * we use a map. */
-static void mp_encode_lua_table(lua_State *L, mp_buf *buf, int level) {
+void mp_encode_lua_table(lua_State *L, mp_buf *buf, int level) {
     if (table_is_an_array(L))
         mp_encode_lua_table_as_array(L,buf,level);
     else
         mp_encode_lua_table_as_map(L,buf,level);
 }
 
-static void mp_encode_lua_null(lua_State *L, mp_buf *buf) {
+void mp_encode_lua_null(lua_State *L, mp_buf *buf) {
     unsigned char b[1];
     (void)L;
 
@@ -476,7 +480,7 @@ static void mp_encode_lua_null(lua_State *L, mp_buf *buf) {
     mp_buf_append(buf,b,1);
 }
 
-static void mp_encode_lua_type(lua_State *L, mp_buf *buf, int level) {
+void mp_encode_lua_type(lua_State *L, mp_buf *buf, int level) {
     int t = lua_type(L,-1);
 
     /* Limit the encoding of nested tables to a specified maximum depth, so that
@@ -506,7 +510,7 @@ static void mp_encode_lua_type(lua_State *L, mp_buf *buf, int level) {
  * Packs all arguments as a stream for multiple upacking later.
  * Returns error if no arguments provided.
  */
-static int mp_pack(lua_State *L) {
+int mp_pack(lua_State *L) {
     int nargs = lua_gettop(L);
     int i;
     mp_buf *buf;
@@ -687,6 +691,15 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
             mp_cur_consume(c,9);
         }
         break;
+    case 0xd9:  /* raw 8 */
+        mp_cur_need(c,2);
+        {
+            size_t l = c->p[1];
+            mp_cur_need(c,2+l);
+            lua_pushlstring(L,(char*)c->p+2,l);
+            mp_cur_consume(c,2+l);
+        }
+        break;
     case 0xda:  /* raw 16 */
         mp_cur_need(c,3);
         {
@@ -773,7 +786,7 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
     }
 }
 
-static int mp_unpack_full(lua_State *L, int limit, int offset) {
+int mp_unpack_full(lua_State *L, int limit, int offset) {
     size_t len;
     const char *s;
     mp_cur c;
@@ -826,18 +839,18 @@ static int mp_unpack_full(lua_State *L, int limit, int offset) {
     return cnt;
 }
 
-static int mp_unpack(lua_State *L) {
+int mp_unpack(lua_State *L) {
     return mp_unpack_full(L, 0, 0);
 }
 
-static int mp_unpack_one(lua_State *L) {
+int mp_unpack_one(lua_State *L) {
     int offset = luaL_optinteger(L, 2, 0);
     /* Variable pop because offset may not exist */
     lua_pop(L, lua_gettop(L)-1);
     return mp_unpack_full(L, 1, offset);
 }
 
-static int mp_unpack_limit(lua_State *L) {
+int mp_unpack_limit(lua_State *L) {
     int limit = luaL_checkinteger(L, 2);
     int offset = luaL_optinteger(L, 3, 0);
     /* Variable pop because offset may not exist */
@@ -846,7 +859,7 @@ static int mp_unpack_limit(lua_State *L) {
     return mp_unpack_full(L, limit, offset);
 }
 
-static int mp_safe(lua_State *L) {
+int mp_safe(lua_State *L) {
     int argc, err, total_results;
 
     argc = lua_gettop(L);
@@ -869,7 +882,7 @@ static int mp_safe(lua_State *L) {
 }
 
 /* -------------------------------------------------------------------------- */
-static const struct luaL_Reg cmds[] = {
+const struct luaL_Reg cmds[] = {
     {"pack", mp_pack},
     {"unpack", mp_unpack},
     {"unpack_one", mp_unpack_one},
@@ -877,7 +890,7 @@ static const struct luaL_Reg cmds[] = {
     {0}
 };
 
-static int luaopen_create(lua_State *L) {
+int luaopen_create(lua_State *L) {
     int i;
     /* Manually construct our module table instead of
      * relying on _register or _newlib */

redis.conf

@@ -30,28 +30,32 @@
 # include /path/to/local.conf
 # include /path/to/other.conf
 
-################################ GENERAL  #####################################
+################################## NETWORK #####################################
 
-# By default Redis does not run as a daemon. Use 'yes' if you need it.
-# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
-daemonize no
+# By default, if no "bind" configuration directive is specified, Redis listens
+# for connections from all the network interfaces available on the server.
+# It is possible to listen to just one or multiple selected interfaces using
+# the "bind" configuration directive, followed by one or more IP addresses.
+#
+# Examples:
+#
+# bind 192.168.1.100 10.0.0.1
+# bind 127.0.0.1 ::1
+#
+# ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the
+# internet, binding to all the interfaces is dangerous and will expose the
+# instance to everybody on the internet. So by default we uncomment the
+# following bind directive, that will force Redis to listen only into
+# the IPv4 lookback interface address (this means Redis will be able to
+# accept connections only from clients running into the same computer it
+# is running).
+#
+# IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES
+# JUST UNCOMMENT THE FOLLOWING LINE.
+# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+bind 127.0.0.1
 
-# If you run Redis from upstart or systemd, Redis can interact with your
-# supervision tree. Options:
-#   supervised no      - no supervision interaction
-#   supervised upstart - signal upstart by putting Redis into SIGSTOP mode
-#   supervised systemd - signal systemd by writing READY=1 to $NOTIFY_SOCKET
-#   supervised auto    - detect upstart or systemd method based on
-#                        UPSTART_JOB or NOTIFY_SOCKET environment variables
-# Note: these supervision methods only signal "process is ready."
-#       They do not enable continuous liveness pings back to your supervisor.
-supervised no
-
-# When running daemonized, Redis writes a pid file in /var/run/redis.pid by
-# default. You can specify a custom pid file location here.
-pidfile /var/run/redis.pid
-
-# Accept connections on the specified port, default is 6379.
+# Accept connections on the specified port, default is 6379 (IANA #815344).
 # If port 0 is specified Redis will not listen on a TCP socket.
 port 6379
 
@@ -64,16 +68,8 @@ port 6379
 # in order to get the desired effect.
 tcp-backlog 511
 
-# By default Redis listens for connections from all the network interfaces
-# available on the server. It is possible to listen to just one or multiple
-# interfaces using the "bind" configuration directive, followed by one or
-# more IP addresses.
+# Unix socket.
 #
-# Examples:
-#
-# bind 192.168.1.100 10.0.0.1
-# bind 127.0.0.1
-
 # Specify the path for the Unix socket that will be used to listen for
 # incoming connections. There is no default, so Redis will not listen
 # on a unix socket when not specified.
@@ -100,6 +96,34 @@ timeout 0
 # A reasonable value for this option is 60 seconds.
 tcp-keepalive 0
 
+################################# GENERAL #####################################
+
+# By default Redis does not run as a daemon. Use 'yes' if you need it.
+# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
+daemonize no
+
+# If you run Redis from upstart or systemd, Redis can interact with your
+# supervision tree. Options:
+#   supervised no      - no supervision interaction
+#   supervised upstart - signal upstart by putting Redis into SIGSTOP mode
+#   supervised systemd - signal systemd by writing READY=1 to $NOTIFY_SOCKET
+#   supervised auto    - detect upstart or systemd method based on
+#                        UPSTART_JOB or NOTIFY_SOCKET environment variables
+# Note: these supervision methods only signal "process is ready."
+#       They do not enable continuous liveness pings back to your supervisor.
+supervised no
+
+# If a pid file is specified, Redis writes it where specified at startup
+# and removes it at exit.
+#
+# When the server runs non daemonized, no pid file is created if none is
+# specified in the configuration. When the server is daemonized, the pid file
+# is used even if not specified, defaulting to "/var/run/redis.pid".
+#
+# Creating a pid file is best effort: if Redis is not able to create it
+# nothing bad happens, the server will start and run normally.
+pidfile /var/run/redis.pid
+
 # Specify the server verbosity level.
 # This can be one of:
 # debug (a lot of information, useful for development/testing)
@@ -294,6 +318,36 @@ repl-diskless-sync no
 # it entirely just set it to 0 seconds and the transfer will start ASAP.
 repl-diskless-sync-delay 5
 
+# Enable diskless replication on slave side.
+#
+# When this option is on, the slave loads the RDB directly from the socket
+# rather than saving it to disk first. However there are data loss risks
+# associated with this feature, so make sure to read the following WARNING
+# section.
+#
+# WARNING: Note that this means that the dataset in the slave gets flushed
+# before the slave is actually sure the RDB transfer is complete, so if the
+# replication link is disconnected after the slave already flushed away its
+# dataset, but before successfully loading the new one, the slave will
+# remain empty (for all the time needed to attempt a new synchornization with
+# the master).
+#
+# This means that you should carefully consider the effects of this feature
+# on slaves that may be promoted to masters:
+#
+# 1) Sentinel checks the disconnection time and the offset of slaves before
+#    promotion. However it is possible that after the check, the slave
+#    attempts to connect with the master again and flushes its dataset.
+#    In order to run Sentinel safely in this setup, make sure to enable
+#    the "slave-protected-restart" option.
+#
+# 2) Redis Cluster slaves will refuse to try to be promoted to masters if
+#    if the dataset was flushed, so this is safe in the context of Redis Cluster.
+#
+# 3) If you are using your own HA setup, make sure to enable slave
+#    "slave-protected-restart".
+repl-diskless-load no
+
 # Slaves send PINGs to server in a predefined interval. It's possible to change
 # this interval with the repl_ping_slave_period option. The default value is 10
 # seconds.
@@ -781,7 +835,7 @@ slowlog-max-len 128
 # "CONFIG SET latency-monitor-threshold <milliseconds>" if needed.
 latency-monitor-threshold 0
 
-############################# Event notification ##############################
+############################# EVENT NOTIFICATION ##############################
 
 # Redis can notify Pub/Sub clients about events happening in the key space.
 # This feature is documented at http://redis.io/topics/notifications

src/Makefile

@@ -15,7 +15,7 @@
 release_hdr := $(shell sh -c './mkreleasehdr.sh')
 uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
 OPTIMIZATION?=-O2
-DEPENDENCY_TARGETS=hiredis linenoise lua
+DEPENDENCY_TARGETS=hiredis linenoise lua geohash-int
 
 # Default settings
 STD=-std=c99 -pedantic -DREDIS_STATIC=''
@@ -53,7 +53,7 @@ endif
 # Override default settings if possible
 -include .make-settings
 
-FINAL_CFLAGS=$(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS)
+FINAL_CFLAGS=$(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS) -I../deps/geohash-int
 FINAL_LDFLAGS=$(LDFLAGS) $(REDIS_LDFLAGS) $(DEBUG)
 FINAL_LIBS=-lm
 DEBUG=-g -ggdb
@@ -117,7 +117,8 @@ endif
 
 REDIS_SERVER_NAME=redis-server
 REDIS_SENTINEL_NAME=redis-sentinel
-REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o
+REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o geo.o
+REDIS_GEOHASH_OBJ=../deps/geohash-int/geohash.o ../deps/geohash-int/geohash_helper.o
 REDIS_CLI_NAME=redis-cli
 REDIS_CLI_OBJ=anet.o sds.o adlist.o redis-cli.o zmalloc.o release.o anet.o ae.o crc64.o
 REDIS_BENCHMARK_NAME=redis-benchmark
@@ -171,7 +172,7 @@ endif
 
 # redis-server
 $(REDIS_SERVER_NAME): $(REDIS_SERVER_OBJ)
-	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/lua/src/liblua.a $(FINAL_LIBS)
+	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/lua/src/liblua.a $(REDIS_GEOHASH_OBJ) $(FINAL_LIBS)
 
 # redis-sentinel
 $(REDIS_SENTINEL_NAME): $(REDIS_SERVER_NAME)

src/anet.c

@@ -193,6 +193,20 @@ int anetSendTimeout(char *err, int fd, long long ms) {
     return ANET_OK;
 }
 
+/* Set the socket receive timeout (SO_RCVTIMEO socket option) to the specified
+ * number of milliseconds, or disable it if the 'ms' argument is zero. */
+int anetRecvTimeout(char *err, int fd, long long ms) {
+    struct timeval tv;
+
+    tv.tv_sec = ms/1000;
+    tv.tv_usec = (ms%1000)*1000;
+    if (setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)) == -1) {
+        anetSetError(err, "setsockopt SO_RCVTIMEO: %s", strerror(errno));
+        return ANET_ERR;
+    }
+    return ANET_OK;
+}
+
 /* anetGenericResolve() is called by anetResolve() and anetResolveIP() to
  * do the actual work. It resolves the hostname "host" and set the string
  * representation of the IP address into the buffer pointed by "ipbuf".
@@ -264,6 +278,7 @@ static int anetCreateSocket(char *err, int domain) {
 
 #define ANET_CONNECT_NONE 0
 #define ANET_CONNECT_NONBLOCK 1
+#define ANET_CONNECT_BE_BINDING 2 /* Best effort binding. */
 static int anetTcpGenericConnect(char *err, char *addr, int port,
                                  char *source_addr, int flags)
 {
@@ -295,7 +310,7 @@ static int anetTcpGenericConnect(char *err, char *addr, int port,
             if ((rv = getaddrinfo(source_addr, NULL, &hints, &bservinfo)) != 0)
             {
                 anetSetError(err, "%s", gai_strerror(rv));
-                goto end;
+                goto error;
             }
             for (b = bservinfo; b != NULL; b = b->ai_next) {
                 if (bind(s,b->ai_addr,b->ai_addrlen) != -1) {
@@ -306,7 +321,7 @@ static int anetTcpGenericConnect(char *err, char *addr, int port,
             freeaddrinfo(bservinfo);
             if (!bound) {
                 anetSetError(err, "bind: %s", strerror(errno));
-                goto end;
+                goto error;
             }
         }
         if (connect(s,p->ai_addr,p->ai_addrlen) == -1) {
@@ -331,9 +346,17 @@ error:
         close(s);
         s = ANET_ERR;
     }
+
 end:
     freeaddrinfo(servinfo);
-    return s;
+
+    /* Handle best effort binding: if a binding address was used, but it is
+     * not possible to create a socket, try again without a binding address. */
+    if (s == ANET_ERR && source_addr && (flags & ANET_CONNECT_BE_BINDING)) {
+        return anetTcpGenericConnect(err,addr,port,NULL,flags);
+    } else {
+        return s;
+    }
 }
 
 int anetTcpConnect(char *err, char *addr, int port)
@@ -346,9 +369,18 @@ int anetTcpNonBlockConnect(char *err, char *addr, int port)
     return anetTcpGenericConnect(err,addr,port,NULL,ANET_CONNECT_NONBLOCK);
 }
 
-int anetTcpNonBlockBindConnect(char *err, char *addr, int port, char *source_addr)
+int anetTcpNonBlockBindConnect(char *err, char *addr, int port,
+                               char *source_addr)
 {
-    return anetTcpGenericConnect(err,addr,port,source_addr,ANET_CONNECT_NONBLOCK);
+    return anetTcpGenericConnect(err,addr,port,source_addr,
+            ANET_CONNECT_NONBLOCK);
+}
+
+int anetTcpNonBlockBestEffortBindConnect(char *err, char *addr, int port,
+                                         char *source_addr)
+{
+    return anetTcpGenericConnect(err,addr,port,source_addr,
+            ANET_CONNECT_NONBLOCK|ANET_CONNECT_BE_BINDING);
 }
 
 int anetUnixGenericConnect(char *err, char *path, int flags)
@@ -391,7 +423,7 @@ int anetUnixNonBlockConnect(char *err, char *path)
  * (unless error or EOF condition is encountered) */
 int anetRead(int fd, char *buf, int count)
 {
-    int nread, totlen = 0;
+    ssize_t nread, totlen = 0;
     while(totlen != count) {
         nread = read(fd,buf,count-totlen);
         if (nread == 0) return totlen;
@@ -402,11 +434,11 @@ int anetRead(int fd, char *buf, int count)
     return totlen;
 }
 
-/* Like write(2) but make sure 'count' is read before to return
+/* Like write(2) but make sure 'count' is written before to return
  * (unless error is encountered) */
 int anetWrite(int fd, char *buf, int count)
 {
-    int nwritten, totlen = 0;
+    ssize_t nwritten, totlen = 0;
     while(totlen != count) {
         nwritten = write(fd,buf,count-totlen);
         if (nwritten == 0) return totlen;

src/anet.h

@@ -50,6 +50,7 @@
 int anetTcpConnect(char *err, char *addr, int port);
 int anetTcpNonBlockConnect(char *err, char *addr, int port);
 int anetTcpNonBlockBindConnect(char *err, char *addr, int port, char *source_addr);
+int anetTcpNonBlockBestEffortBindConnect(char *err, char *addr, int port, char *source_addr);
 int anetUnixConnect(char *err, char *path);
 int anetUnixNonBlockConnect(char *err, char *path);
 int anetRead(int fd, char *buf, int count);
@@ -67,6 +68,7 @@ int anetEnableTcpNoDelay(char *err, int fd);
 int anetDisableTcpNoDelay(char *err, int fd);
 int anetTcpKeepAlive(char *err, int fd);
 int anetSendTimeout(char *err, int fd, long long ms);
+int anetRecvTimeout(char *err, int fd, long long ms);
 int anetPeerToString(int fd, char *ip, size_t ip_len, int *port);
 int anetKeepAlive(char *err, int fd, int interval);
 int anetSockName(int fd, char *ip, size_t ip_len, int *port);

src/aof.c

@@ -620,7 +620,7 @@ int loadAppendOnlyFile(char *filename) {
     server.aof_state = REDIS_AOF_OFF;
 
     fakeClient = createFakeClient();
-    startLoading(fp);
+    startLoadingFile(fp);
 
     while(1) {
         int argc, j;
@@ -1146,9 +1146,9 @@ int rewriteAppendOnlyFile(char *filename) {
     return REDIS_OK;
 
 werr:
+    redisLog(REDIS_WARNING,"Write error writing append only file on disk: %s", strerror(errno));
     fclose(fp);
     unlink(tmpfile);
-    redisLog(REDIS_WARNING,"Write error writing append only file on disk: %s", strerror(errno));
     if (di) dictReleaseIterator(di);
     return REDIS_ERR;
 }

src/bio.c

@@ -142,6 +142,13 @@ void *bioProcessBackgroundJobs(void *arg) {
     unsigned long type = (unsigned long) arg;
     sigset_t sigset;
 
+    /* Check that the type is within the right interval. */
+    if (type >= REDIS_BIO_NUM_OPS) {
+        redisLog(REDIS_WARNING,
+            "Warning: bio thread started with wrong type %lu",type);
+        return NULL;
+    }
+
     /* Make the thread killable at any time, so that bioKillThreads()
      * can work reliably. */
     pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);

src/bitops.c

@@ -229,19 +229,17 @@ void setbitCommand(redisClient *c) {
         return;
     }
 
+    byte = bitoffset >> 3;
     o = lookupKeyWrite(c->db,c->argv[1]);
     if (o == NULL) {
-        o = createObject(REDIS_STRING,sdsempty());
+        o = createObject(REDIS_STRING,sdsnewlen(NULL, byte+1));
         dbAdd(c->db,c->argv[1],o);
     } else {
         if (checkType(c,o,REDIS_STRING)) return;
         o = dbUnshareStringValue(c->db,c->argv[1],o);
+        o->ptr = sdsgrowzero(o->ptr,byte+1);
     }
 
-    /* Grow sds value to the right length if necessary */
-    byte = bitoffset >> 3;
-    o->ptr = sdsgrowzero(o->ptr,byte+1);
-
     /* Get current values */
     byteval = ((uint8_t*)o->ptr)[byte];
     bit = 7 - (bitoffset & 0x7);

src/blocked.c

@@ -59,6 +59,8 @@
  * When implementing a new type of blocking opeation, the implementation
  * should modify unblockClient() and replyToBlockedClientTimedOut() in order
  * to handle the btype-specific behavior of this two functions.
+ * If the blocking operation waits for certain keys to change state, the
+ * clusterRedirectBlockedClientIfNeeded() function should also be updated.
  */
 
 #include "redis.h"
@@ -114,13 +116,15 @@ void processUnblockedClients(void) {
         c = ln->value;
         listDelNode(server.unblocked_clients,ln);
         c->flags &= ~REDIS_UNBLOCKED;
-        c->btype = REDIS_BLOCKED_NONE;
 
-        /* Process remaining data in the input buffer. */
-        if (c->querybuf && sdslen(c->querybuf) > 0) {
-            server.current_client = c;
-            processInputBuffer(c);
-            server.current_client = NULL;
+        /* Process remaining data in the input buffer, unless the client
+         * is blocked again. Actually processInputBuffer() checks that the
+         * client is not blocked before to proceed, but things may change and
+         * the code is conceptually more correct this way. */
+        if (!(c->flags & REDIS_BLOCKED)) {
+            if (c->querybuf && sdslen(c->querybuf) > 0) {
+                processInputBuffer(c);
+            }
         }
     }
 }
@@ -138,10 +142,14 @@ void unblockClient(redisClient *c) {
     /* Clear the flags, and put the client in the unblocked list so that
      * we'll process new commands in its query buffer ASAP. */
     c->flags &= ~REDIS_BLOCKED;
-    c->flags |= REDIS_UNBLOCKED;
     c->btype = REDIS_BLOCKED_NONE;
     server.bpop_blocked_clients--;
-    listAddNodeTail(server.unblocked_clients,c);
+    /* The client may already be into the unblocked list because of a previous
+     * blocking operation, don't add back it into the list multiple times. */
+    if (!(c->flags & REDIS_UNBLOCKED)) {
+        c->flags |= REDIS_UNBLOCKED;
+        listAddNodeTail(server.unblocked_clients,c);
+    }
 }
 
 /* This function gets called when a blocked client timed out in order to
@@ -156,3 +164,27 @@ void replyToBlockedClientTimedOut(redisClient *c) {
     }
 }
 
+/* Mass-unblock clients because something changed in the instance that makes
+ * blocking no longer safe. For example clients blocked in list operations
+ * in an instance which turns from master to slave is unsafe, so this function
+ * is called when a master turns into a slave.
+ *
+ * The semantics is to send an -UNBLOCKED error to the client, disconnecting
+ * it at the same time. */
+void disconnectAllBlockedClients(void) {
+    listNode *ln;
+    listIter li;
+
+    listRewind(server.clients,&li);
+    while((ln = listNext(&li))) {
+        redisClient *c = listNodeValue(ln);
+
+        if (c->flags & REDIS_BLOCKED) {
+            addReplySds(c,sdsnew(
+                "-UNBLOCKED force unblock from blocking operation, "
+                "instance state changed (master -> slave?)\r\n"));
+            unblockClient(c);
+            c->flags |= REDIS_CLOSE_AFTER_REPLY;
+        }
+    }
+}

src/cluster.c

@@ -74,27 +74,13 @@ void clusterCloseAllSlots(void);
 void clusterSetNodeAsMaster(clusterNode *n);
 void clusterDelNode(clusterNode *delnode);
 sds representRedisNodeFlags(sds ci, uint16_t flags);
+uint64_t clusterGetMaxEpoch(void);
+int clusterBumpConfigEpochWithoutConsensus(void);
 
 /* -----------------------------------------------------------------------------
  * Initialization
  * -------------------------------------------------------------------------- */
 
-/* Return the greatest configEpoch found in the cluster. */
-uint64_t clusterGetMaxEpoch(void) {
-    uint64_t max = 0;
-    dictIterator *di;
-    dictEntry *de;
-
-    di = dictGetSafeIterator(server.cluster->nodes);
-    while((de = dictNext(di)) != NULL) {
-        clusterNode *node = dictGetVal(de);
-        if (node->configEpoch > max) max = node->configEpoch;
-    }
-    dictReleaseIterator(di);
-    if (max < server.cluster->currentEpoch) max = server.cluster->currentEpoch;
-    return max;
-}
-
 /* Load the cluster config from 'filename'.
  *
  * If the file does not exist or is zero-length (this may happen because
@@ -372,6 +358,11 @@ void clusterSaveConfigOrDie(int do_fsync) {
  * On success REDIS_OK is returned, otherwise an error is logged and
  * the function returns REDIS_ERR to signal a lock was not acquired. */
 int clusterLockConfig(char *filename) {
+/* flock() does not exist on Solaris
+ * and a fcntl-based solution won't help, as we constantly re-open that file,
+ * which will release _all_ locks anyway
+ */
+#if !defined(__sun)
     /* To lock it, we need to open the file in a way it is created if
      * it does not exist, otherwise there is a race condition with other
      * processes. */
@@ -399,6 +390,8 @@ int clusterLockConfig(char *filename) {
     }
     /* Lock acquired: leak the 'fd' by not closing it, so that we'll retain the
      * lock to the file as long as the process exists. */
+#endif /* __sun */
+
     return REDIS_OK;
 }
 
@@ -927,6 +920,138 @@ void clusterRenameNode(clusterNode *node, char *newname) {
     clusterAddNode(node);
 }
 
+/* -----------------------------------------------------------------------------
+ * CLUSTER config epoch handling
+ * -------------------------------------------------------------------------- */
+
+/* Return the greatest configEpoch found in the cluster, or the current
+ * epoch if greater than any node configEpoch. */
+uint64_t clusterGetMaxEpoch(void) {
+    uint64_t max = 0;
+    dictIterator *di;
+    dictEntry *de;
+
+    di = dictGetSafeIterator(server.cluster->nodes);
+    while((de = dictNext(di)) != NULL) {
+        clusterNode *node = dictGetVal(de);
+        if (node->configEpoch > max) max = node->configEpoch;
+    }
+    dictReleaseIterator(di);
+    if (max < server.cluster->currentEpoch) max = server.cluster->currentEpoch;
+    return max;
+}
+
+/* If this node epoch is zero or is not already the greatest across the
+ * cluster (from the POV of the local configuration), this function will:
+ *
+ * 1) Generate a new config epoch increment the current epoch.
+ * 2) Assign the new epoch to this node, WITHOUT any consensus.
+ * 3) Persist the configuration on disk before sending packets with the
+ *    new configuration.
+ *
+ * If the new config epoch is generated and assigend, REDIS_OK is returned,
+ * otherwise REDIS_ERR is returned (since the node has already the greatest
+ * configuration around) and no operation is performed.
+ *
+ * Important note: this function violates the principle that config epochs
+ * should be generated with consensus and should be unique across the cluster.
+ * However Redis Cluster uses this auto-generated new config epochs in two
+ * cases:
+ *
+ * 1) When slots are closed after importing. Otherwise resharding would be
+ *    too exansive.
+ * 2) When CLUSTER FAILOVER is called with options that force a slave to
+ *    failover its master even if there is not master majority able to
+ *    create a new configuration epoch.
+ *
+ * Redis Cluster does not explode using this function, even in the case of
+ * a collision between this node and another node, generating the same
+ * configuration epoch unilaterally, because the config epoch conflict
+ * resolution algorithm will eventually move colliding nodes to different
+ * config epochs. However usign this function may violate the "last failover
+ * wins" rule, so should only be used with care. */
+int clusterBumpConfigEpochWithoutConsensus(void) {
+    uint64_t maxEpoch = clusterGetMaxEpoch();
+
+    if (myself->configEpoch == 0 ||
+        myself->configEpoch != maxEpoch)
+    {
+        server.cluster->currentEpoch++;
+        myself->configEpoch = server.cluster->currentEpoch;
+        clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG|
+                             CLUSTER_TODO_FSYNC_CONFIG);
+        redisLog(REDIS_WARNING,
+            "New configEpoch set to %llu",
+            (unsigned long long) myself->configEpoch);
+        return REDIS_OK;
+    } else {
+        return REDIS_ERR;
+    }
+}
+
+/* This function is called when this node is a master, and we receive from
+ * another master a configuration epoch that is equal to our configuration
+ * epoch.
+ *
+ * BACKGROUND
+ *
+ * It is not possible that different slaves get the same config
+ * epoch during a failover election, because the slaves need to get voted
+ * by a majority. However when we perform a manual resharding of the cluster
+ * the node will assign a configuration epoch to itself without to ask
+ * for agreement. Usually resharding happens when the cluster is working well
+ * and is supervised by the sysadmin, however it is possible for a failover
+ * to happen exactly while the node we are resharding a slot to assigns itself
+ * a new configuration epoch, but before it is able to propagate it.
+ *
+ * So technically it is possible in this condition that two nodes end with
+ * the same configuration epoch.
+ *
+ * Another possibility is that there are bugs in the implementation causing
+ * this to happen.
+ *
+ * Moreover when a new cluster is created, all the nodes start with the same
+ * configEpoch. This collision resolution code allows nodes to automatically
+ * end with a different configEpoch at startup automatically.
+ *
+ * In all the cases, we want a mechanism that resolves this issue automatically
+ * as a safeguard. The same configuration epoch for masters serving different
+ * set of slots is not harmful, but it is if the nodes end serving the same
+ * slots for some reason (manual errors or software bugs) without a proper
+ * failover procedure.
+ *
+ * In general we want a system that eventually always ends with different
+ * masters having different configuration epochs whatever happened, since
+ * nothign is worse than a split-brain condition in a distributed system.
+ *
+ * BEHAVIOR
+ *
+ * When this function gets called, what happens is that if this node
+ * has the lexicographically smaller Node ID compared to the other node
+ * with the conflicting epoch (the 'sender' node), it will assign itself
+ * the greatest configuration epoch currently detected among nodes plus 1.
+ *
+ * This means that even if there are multiple nodes colliding, the node
+ * with the greatest Node ID never moves forward, so eventually all the nodes
+ * end with a different configuration epoch.
+ */
+void clusterHandleConfigEpochCollision(clusterNode *sender) {
+    /* Prerequisites: nodes have the same configEpoch and are both masters. */
+    if (sender->configEpoch != myself->configEpoch ||
+        !nodeIsMaster(sender) || !nodeIsMaster(myself)) return;
+    /* Don't act if the colliding node has a smaller Node ID. */
+    if (memcmp(sender->name,myself->name,REDIS_CLUSTER_NAMELEN) <= 0) return;
+    /* Get the next ID available at the best of this node knowledge. */
+    server.cluster->currentEpoch++;
+    myself->configEpoch = server.cluster->currentEpoch;
+    clusterSaveConfigOrDie(1);
+    redisLog(REDIS_VERBOSE,
+        "WARNING: configEpoch collision with node %.40s."
+        " configEpoch set to %llu",
+        sender->name,
+        (unsigned long long) myself->configEpoch);
+}
+
 /* -----------------------------------------------------------------------------
  * CLUSTER nodes blacklist
  *
@@ -1399,69 +1524,6 @@ void clusterUpdateSlotsConfigWith(clusterNode *sender, uint64_t senderConfigEpoc
     }
 }
 
-/* This function is called when this node is a master, and we receive from
- * another master a configuration epoch that is equal to our configuration
- * epoch.
- *
- * BACKGROUND
- *
- * It is not possible that different slaves get the same config
- * epoch during a failover election, because the slaves need to get voted
- * by a majority. However when we perform a manual resharding of the cluster
- * the node will assign a configuration epoch to itself without to ask
- * for agreement. Usually resharding happens when the cluster is working well
- * and is supervised by the sysadmin, however it is possible for a failover
- * to happen exactly while the node we are resharding a slot to assigns itself
- * a new configuration epoch, but before it is able to propagate it.
- *
- * So technically it is possible in this condition that two nodes end with
- * the same configuration epoch.
- *
- * Another possibility is that there are bugs in the implementation causing
- * this to happen.
- *
- * Moreover when a new cluster is created, all the nodes start with the same
- * configEpoch. This collision resolution code allows nodes to automatically
- * end with a different configEpoch at startup automatically.
- *
- * In all the cases, we want a mechanism that resolves this issue automatically
- * as a safeguard. The same configuration epoch for masters serving different
- * set of slots is not harmful, but it is if the nodes end serving the same
- * slots for some reason (manual errors or software bugs) without a proper
- * failover procedure.
- *
- * In general we want a system that eventually always ends with different
- * masters having different configuration epochs whatever happened, since
- * nothign is worse than a split-brain condition in a distributed system.
- *
- * BEHAVIOR
- *
- * When this function gets called, what happens is that if this node
- * has the lexicographically smaller Node ID compared to the other node
- * with the conflicting epoch (the 'sender' node), it will assign itself
- * the greatest configuration epoch currently detected among nodes plus 1.
- *
- * This means that even if there are multiple nodes colliding, the node
- * with the greatest Node ID never moves forward, so eventually all the nodes
- * end with a different configuration epoch.
- */
-void clusterHandleConfigEpochCollision(clusterNode *sender) {
-    /* Prerequisites: nodes have the same configEpoch and are both masters. */
-    if (sender->configEpoch != myself->configEpoch ||
-        !nodeIsMaster(sender) || !nodeIsMaster(myself)) return;
-    /* Don't act if the colliding node has a smaller Node ID. */
-    if (memcmp(sender->name,myself->name,REDIS_CLUSTER_NAMELEN) <= 0) return;
-    /* Get the next ID available at the best of this node knowledge. */
-    server.cluster->currentEpoch++;
-    myself->configEpoch = server.cluster->currentEpoch;
-    clusterSaveConfigOrDie(1);
-    redisLog(REDIS_VERBOSE,
-        "WARNING: configEpoch collision with node %.40s."
-        " configEpoch set to %llu",
-        sender->name,
-        (unsigned long long) myself->configEpoch);
-}
-
 /* When this function is called, there is a packet to process starting
  * at node->rcvbuf. Releasing the buffer is up to the caller, so this
  * function should just handle the higher level stuff of processing the
@@ -2582,6 +2644,42 @@ void clusterLogCantFailover(int reason) {
     redisLog(REDIS_WARNING,"Currently unable to failover: %s", msg);
 }
 
+/* This function implements the final part of automatic and manual failovers,
+ * where the slave grabs its master's hash slots, and propagates the new
+ * configuration.
+ *
+ * Note that it's up to the caller to be sure that the node got a new
+ * configuration epoch already. */
+void clusterFailoverReplaceYourMaster(void) {
+    int j;
+    clusterNode *oldmaster = myself->slaveof;
+
+    if (nodeIsMaster(myself) || oldmaster == NULL) return;
+
+    /* 1) Turn this node into a master. */
+    clusterSetNodeAsMaster(myself);
+    replicationUnsetMaster();
+
+    /* 2) Claim all the slots assigned to our master. */
+    for (j = 0; j < REDIS_CLUSTER_SLOTS; j++) {
+        if (clusterNodeGetSlotBit(oldmaster,j)) {
+            clusterDelSlot(j);
+            clusterAddSlot(myself,j);
+        }
+    }
+
+    /* 3) Update state and save config. */
+    clusterUpdateState();
+    clusterSaveConfigOrDie(1);
+
+    /* 4) Pong all the other nodes so that they can update the state
+     *    accordingly and detect that we switched to master role. */
+    clusterBroadcastPong(CLUSTER_BROADCAST_ALL);
+
+    /* 5) If there was a manual failover in progress, clear the state. */
+    resetManualFailover();
+}
+
 /* This function is called if we are a slave node and our master serving
  * a non-zero amount of hash slots is in FAIL state.
  *
@@ -2596,7 +2694,6 @@ void clusterHandleSlaveFailover(void) {
     int needed_quorum = (server.cluster->size / 2) + 1;
     int manual_failover = server.cluster->mf_end != 0 &&
                           server.cluster->mf_can_start;
-    int j;
     mstime_t auth_timeout, auth_retry_time;
 
     server.cluster->todo_before_sleep &= ~CLUSTER_TODO_HANDLE_FAILOVER;
@@ -2738,26 +2835,12 @@ void clusterHandleSlaveFailover(void) {
 
     /* Check if we reached the quorum. */
     if (server.cluster->failover_auth_count >= needed_quorum) {
-        clusterNode *oldmaster = myself->slaveof;
+        /* We have the quorum, we can finally failover the master. */
 
         redisLog(REDIS_WARNING,
             "Failover election won: I'm the new master.");
-        /* We have the quorum, perform all the steps to correctly promote
-         * this slave to a master.
-         *
-         * 1) Turn this node into a master. */
-        clusterSetNodeAsMaster(myself);
-        replicationUnsetMaster();
 
-        /* 2) Claim all the slots assigned to our master. */
-        for (j = 0; j < REDIS_CLUSTER_SLOTS; j++) {
-            if (clusterNodeGetSlotBit(oldmaster,j)) {
-                clusterDelSlot(j);
-                clusterAddSlot(myself,j);
-            }
-        }
-
-        /* 3) Update my configEpoch to the epoch of the election. */
+        /* Update my configEpoch to the epoch of the election. */
         if (myself->configEpoch < server.cluster->failover_auth_epoch) {
             myself->configEpoch = server.cluster->failover_auth_epoch;
             redisLog(REDIS_WARNING,
@@ -2765,16 +2848,8 @@ void clusterHandleSlaveFailover(void) {
                 (unsigned long long) myself->configEpoch);
         }
 
-        /* 4) Update state and save config. */
-        clusterUpdateState();
-        clusterSaveConfigOrDie(1);
-
-        /* 5) Pong all the other nodes so that they can update the state
-         *    accordingly and detect that we switched to master role. */
-        clusterBroadcastPong(CLUSTER_BROADCAST_ALL);
-
-        /* 6) If there was a manual failover in progress, clear the state. */
-        resetManualFailover();
+        /* Take responsability for the cluster slots. */
+        clusterFailoverReplaceYourMaster();
     } else {
         clusterLogCantFailover(REDIS_CLUSTER_CANT_FAILOVER_WAITING_VOTES);
     }
@@ -3567,7 +3642,7 @@ sds clusterGenNodeDescription(clusterNode *node) {
     else
         ci = sdscatlen(ci," - ",3);
 
-    /* Latency from the POV of this node, link status */
+    /* Latency from the POV of this node, config epoch, link status */
     ci = sdscatprintf(ci,"%lld %lld %llu %s",
         (long long) node->ping_sent,
         (long long) node->pong_received,
@@ -3760,6 +3835,9 @@ void clusterCommand(redisClient *c) {
         o = createObject(REDIS_STRING,ci);
         addReplyBulk(c,o);
         decrRefCount(o);
+    } else if (!strcasecmp(c->argv[1]->ptr,"myid") && c->argc == 2) {
+        /* CLUSTER MYID */
+        addReplyBulkCBuffer(c,myself->name, REDIS_CLUSTER_NAMELEN);
     } else if (!strcasecmp(c->argv[1]->ptr,"slots") && c->argc == 2) {
         /* CLUSTER SLOTS */
         clusterReplyMultiBulkSlots(c);
@@ -3899,17 +3977,9 @@ void clusterCommand(redisClient *c) {
                  * failover happens at the same time we close the slot, the
                  * configEpoch collision resolution will fix it assigning
                  * a different epoch to each node. */
-                uint64_t maxEpoch = clusterGetMaxEpoch();
-
-                if (myself->configEpoch == 0 ||
-                    myself->configEpoch != maxEpoch)
-                {
-                    server.cluster->currentEpoch++;
-                    myself->configEpoch = server.cluster->currentEpoch;
-                    clusterDoBeforeSleep(CLUSTER_TODO_FSYNC_CONFIG);
+                if (clusterBumpConfigEpochWithoutConsensus() == REDIS_OK) {
                     redisLog(REDIS_WARNING,
-                        "configEpoch set to %llu after importing slot %d",
-                        (unsigned long long) myself->configEpoch, slot);
+                        "configEpoch updated after importing slot %d", slot);
                 }
                 server.cluster->importing_slots_from[slot] = NULL;
             }
@@ -3970,7 +4040,10 @@ void clusterCommand(redisClient *c) {
             server.cluster->stats_bus_messages_sent,
             server.cluster->stats_bus_messages_received
         );
-        addReplyBulkSds(c, info);
+        addReplySds(c,sdscatprintf(sdsempty(),"$%lu\r\n",
+            (unsigned long)sdslen(info)));
+        addReplySds(c,info);
+        addReply(c,shared.crlf);
     } else if (!strcasecmp(c->argv[1]->ptr,"saveconfig") && c->argc == 2) {
         int retval = clusterSaveConfig(1);
 
@@ -4052,7 +4125,7 @@ void clusterCommand(redisClient *c) {
         }
 
         /* Can't replicate a slave. */
-        if (n->slaveof != NULL) {
+        if (nodeIsSlave(n)) {
             addReplyError(c,"I can only replicate a master, not a slave.");
             return;
         }
@@ -4109,24 +4182,31 @@ void clusterCommand(redisClient *c) {
     } else if (!strcasecmp(c->argv[1]->ptr,"failover") &&
                (c->argc == 2 || c->argc == 3))
     {
-        /* CLUSTER FAILOVER [FORCE] */
-        int force = 0;
+        /* CLUSTER FAILOVER [FORCE|TAKEOVER] */
+        int force = 0, takeover = 0;
 
         if (c->argc == 3) {
             if (!strcasecmp(c->argv[2]->ptr,"force")) {
                 force = 1;
+            } else if (!strcasecmp(c->argv[2]->ptr,"takeover")) {
+                takeover = 1;
+                force = 1; /* Takeover also implies force. */
             } else {
                 addReply(c,shared.syntaxerr);
                 return;
             }
         }
 
+        /* Check preconditions. */
         if (nodeIsMaster(myself)) {
             addReplyError(c,"You should send CLUSTER FAILOVER to a slave");
             return;
+        } else if (myself->slaveof == NULL) {
+            addReplyError(c,"I'm a slave but my master is unknown to me");
+            return;
         } else if (!force &&
-                   (myself->slaveof == NULL || nodeFailed(myself->slaveof) ||
-                   myself->slaveof->link == NULL))
+                   (nodeFailed(myself->slaveof) ||
+                    myself->slaveof->link == NULL))
         {
             addReplyError(c,"Master is down or failed, "
                             "please use CLUSTER FAILOVER FORCE");
@@ -4135,15 +4215,24 @@ void clusterCommand(redisClient *c) {
         resetManualFailover();
         server.cluster->mf_end = mstime() + REDIS_CLUSTER_MF_TIMEOUT;
 
-        /* If this is a forced failover, we don't need to talk with our master
-         * to agree about the offset. We just failover taking over it without
-         * coordination. */
-        if (force) {
+        if (takeover) {
+            /* A takeover does not perform any initial check. It just
+             * generates a new configuration epoch for this node without
+             * consensus, claims the master's slots, and broadcast the new
+             * configuration. */
+            redisLog(REDIS_WARNING,"Taking over the master (user request).");
+            clusterBumpConfigEpochWithoutConsensus();
+            clusterFailoverReplaceYourMaster();
+        } else if (force) {
+            /* If this is a forced failover, we don't need to talk with our
+             * master to agree about the offset. We just failover taking over
+             * it without coordination. */
+            redisLog(REDIS_WARNING,"Forced failover user request accepted.");
             server.cluster->mf_can_start = 1;
         } else {
+            redisLog(REDIS_WARNING,"Manual failover user request accepted.");
             clusterSendMFStart(myself->slaveof);
         }
-        redisLog(REDIS_WARNING,"Manual failover user request accepted.");
         addReply(c,shared.ok);
     } else if (!strcasecmp(c->argv[1]->ptr,"set-config-epoch") && c->argc == 3)
     {
@@ -4362,11 +4451,12 @@ void restoreCommand(redisClient *c) {
 
 typedef struct migrateCachedSocket {
     int fd;
+    long last_dbid;
     time_t last_use_time;
 } migrateCachedSocket;
 
-/* Return a TCP socket connected with the target instance, possibly returning
- * a cached one.
+/* Return a migrateCachedSocket containing a TCP socket connected with the
+ * target instance, possibly returning a cached one.
  *
  * This function is responsible of sending errors to the client if a
  * connection can't be established. In this case -1 is returned.
@@ -4376,7 +4466,7 @@ typedef struct migrateCachedSocket {
  * If the caller detects an error while using the socket, migrateCloseSocket()
  * should be called so that the connection will be created from scratch
  * the next time. */
-int migrateGetSocket(redisClient *c, robj *host, robj *port, long timeout) {
+migrateCachedSocket* migrateGetSocket(redisClient *c, robj *host, robj *port, long timeout) {
     int fd;
     sds name = sdsempty();
     migrateCachedSocket *cs;
@@ -4389,7 +4479,7 @@ int migrateGetSocket(redisClient *c, robj *host, robj *port, long timeout) {
     if (cs) {
         sdsfree(name);
         cs->last_use_time = server.unixtime;
-        return cs->fd;
+        return cs;
     }
 
     /* No cached socket, create one. */
@@ -4403,13 +4493,13 @@ int migrateGetSocket(redisClient *c, robj *host, robj *port, long timeout) {
     }
 
     /* Create the socket */
-    fd = anetTcpNonBlockBindConnect(server.neterr,c->argv[1]->ptr,
-                atoi(c->argv[2]->ptr),REDIS_BIND_ADDR);
+    fd = anetTcpNonBlockConnect(server.neterr,c->argv[1]->ptr,
+                                atoi(c->argv[2]->ptr));
     if (fd == -1) {
         sdsfree(name);
         addReplyErrorFormat(c,"Can't connect to target node: %s",
             server.neterr);
-        return -1;
+        return NULL;
     }
     anetEnableTcpNoDelay(server.neterr,fd);
 
@@ -4419,15 +4509,16 @@ int migrateGetSocket(redisClient *c, robj *host, robj *port, long timeout) {
         addReplySds(c,
             sdsnew("-IOERR error or timeout connecting to the client\r\n"));
         close(fd);
-        return -1;
+        return NULL;
     }
 
     /* Add to the cache and return it to the caller. */
     cs = zmalloc(sizeof(*cs));
     cs->fd = fd;
+    cs->last_dbid = -1;
     cs->last_use_time = server.unixtime;
     dictAdd(server.migrate_cached_sockets,name,cs);
-    return fd;
+    return cs;
 }
 
 /* Free a migrate cached connection. */
@@ -4468,7 +4559,8 @@ void migrateCloseTimedoutSockets(void) {
 
 /* MIGRATE host port key dbid timeout [COPY | REPLACE] */
 void migrateCommand(redisClient *c) {
-    int fd, copy, replace, j;
+    migrateCachedSocket *cs;
+    int copy, replace, j;
     long timeout;
     long dbid;
     long long ttl, expireat;
@@ -4504,21 +4596,26 @@ try_again:
     /* Check if the key is here. If not we reply with success as there is
      * nothing to migrate (for instance the key expired in the meantime), but
      * we include such information in the reply string. */
-    if ((o = lookupKeyWrite(c->db,c->argv[3])) == NULL) {
+    if ((o = lookupKeyRead(c->db,c->argv[3])) == NULL) {
         addReplySds(c,sdsnew("+NOKEY\r\n"));
         return;
     }
 
     /* Connect */
-    fd = migrateGetSocket(c,c->argv[1],c->argv[2],timeout);
-    if (fd == -1) return; /* error sent to the client by migrateGetSocket() */
+    cs = migrateGetSocket(c,c->argv[1],c->argv[2],timeout);
+    if (cs == NULL) return; /* error sent to the client by migrateGetSocket() */
+
+    rioInitWithBuffer(&cmd,sdsempty());
+
+    /* Send the SELECT command if the current DB is not already selected. */
+    int select = cs->last_dbid != dbid; /* Should we emit SELECT? */
+    if (select) {
+        redisAssertWithInfo(c,NULL,rioWriteBulkCount(&cmd,'*',2));
+        redisAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,"SELECT",6));
+        redisAssertWithInfo(c,NULL,rioWriteBulkLongLong(&cmd,dbid));
+    }
 
     /* Create RESTORE payload and generate the protocol to call the command. */
-    rioInitWithBuffer(&cmd,sdsempty());
-    redisAssertWithInfo(c,NULL,rioWriteBulkCount(&cmd,'*',2));
-    redisAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,"SELECT",6));
-    redisAssertWithInfo(c,NULL,rioWriteBulkLongLong(&cmd,dbid));
-
     expireat = getExpire(c->db,c->argv[3]);
     if (expireat != -1) {
         ttl = expireat-mstime();
@@ -4556,7 +4653,7 @@ try_again:
 
         while ((towrite = sdslen(buf)-pos) > 0) {
             towrite = (towrite > (64*1024) ? (64*1024) : towrite);
-            nwritten = syncWrite(fd,buf+pos,towrite,timeout);
+            nwritten = syncWrite(cs->fd,buf+pos,towrite,timeout);
             if (nwritten != (signed)towrite) goto socket_wr_err;
             pos += nwritten;
         }
@@ -4568,28 +4665,33 @@ try_again:
         char buf2[1024];
 
         /* Read the two replies */
-        if (syncReadLine(fd, buf1, sizeof(buf1), timeout) <= 0)
+        if (select && syncReadLine(cs->fd, buf1, sizeof(buf1), timeout) <= 0)
             goto socket_rd_err;
-        if (syncReadLine(fd, buf2, sizeof(buf2), timeout) <= 0)
+        if (syncReadLine(cs->fd, buf2, sizeof(buf2), timeout) <= 0)
             goto socket_rd_err;
-        if (buf1[0] == '-' || buf2[0] == '-') {
+        if ((select && buf1[0] == '-') || buf2[0] == '-') {
+            /* On error assume that last_dbid is no longer valid. */
+            cs->last_dbid = -1;
             addReplyErrorFormat(c,"Target instance replied with error: %s",
-                (buf1[0] == '-') ? buf1+1 : buf2+1);
+                (select && buf1[0] == '-') ? buf1+1 : buf2+1);
         } else {
+            /* Update the last_dbid in migrateCachedSocket */
+            cs->last_dbid = dbid;
             robj *aux;
 
+            addReply(c,shared.ok);
+
             if (!copy) {
                 /* No COPY option: remove the local key, signal the change. */
                 dbDelete(c->db,c->argv[3]);
                 signalModifiedKey(c->db,c->argv[3]);
-            }
-            addReply(c,shared.ok);
-            server.dirty++;
+                server.dirty++;
 
-            /* Translate MIGRATE as DEL for replication/AOF. */
-            aux = createStringObject("DEL",3);
-            rewriteClientCommandVector(c,2,aux,c->argv[3]);
-            decrRefCount(aux);
+                /* Translate MIGRATE as DEL for replication/AOF. */
+                aux = createStringObject("DEL",3);
+                rewriteClientCommandVector(c,2,aux,c->argv[3]);
+                decrRefCount(aux);
+            }
         }
     }
 
@@ -4671,7 +4773,12 @@ void readwriteCommand(redisClient *c) {
  *
  * REDIS_CLUSTER_REDIR_UNSTABLE if the request contains mutliple keys
  * belonging to the same slot, but the slot is not stable (in migration or
- * importing state, likely because a resharding is in progress). */
+ * importing state, likely because a resharding is in progress).
+ *
+ * REDIS_CLUSTER_REDIR_DOWN_UNBOUND if the request addresses a slot which is
+ * not bound to any node. In this case the cluster global state should be
+ * already "down" but it is fragile to rely on the update of the global state,
+ * so we also handle it here. */
 clusterNode *getNodeByQuery(redisClient *c, struct redisCommand *cmd, robj **argv, int argc, int *hashslot, int *error_code) {
     clusterNode *n = NULL;
     robj *firstkey = NULL;
@@ -4725,7 +4832,18 @@ clusterNode *getNodeByQuery(redisClient *c, struct redisCommand *cmd, robj **arg
                 firstkey = thiskey;
                 slot = thisslot;
                 n = server.cluster->slots[slot];
-                redisAssertWithInfo(c,firstkey,n != NULL);
+
+                /* Error: If a slot is not served, we are in "cluster down"
+                 * state. However the state is yet to be updated, so this was
+                 * not trapped earlier in processCommand(). Report the same
+                 * error to the client. */
+                if (n == NULL) {
+                    getKeysFreeResult(keyindex);
+                    if (error_code)
+                        *error_code = REDIS_CLUSTER_REDIR_DOWN_UNBOUND;
+                    return NULL;
+                }
+
                 /* If we are migrating or importing this slot, we need to check
                  * if we have all the keys in the request (the only way we
                  * can safely serve the request, otherwise we return a TRYAGAIN
@@ -4814,3 +4932,83 @@ clusterNode *getNodeByQuery(redisClient *c, struct redisCommand *cmd, robj **arg
     if (n != myself && error_code) *error_code = REDIS_CLUSTER_REDIR_MOVED;
     return n;
 }
+
+/* Send the client the right redirection code, according to error_code
+ * that should be set to one of REDIS_CLUSTER_REDIR_* macros.
+ *
+ * If REDIS_CLUSTER_REDIR_ASK or REDIS_CLUSTER_REDIR_MOVED error codes
+ * are used, then the node 'n' should not be NULL, but should be the
+ * node we want to mention in the redirection. Moreover hashslot should
+ * be set to the hash slot that caused the redirection. */
+void clusterRedirectClient(redisClient *c, clusterNode *n, int hashslot, int error_code) {
+    if (error_code == REDIS_CLUSTER_REDIR_CROSS_SLOT) {
+        addReplySds(c,sdsnew("-CROSSSLOT Keys in request don't hash to the same slot\r\n"));
+    } else if (error_code == REDIS_CLUSTER_REDIR_UNSTABLE) {
+        /* The request spawns mutliple keys in the same slot,
+         * but the slot is not "stable" currently as there is
+         * a migration or import in progress. */
+        addReplySds(c,sdsnew("-TRYAGAIN Multiple keys request during rehashing of slot\r\n"));
+    } else if (error_code == REDIS_CLUSTER_REDIR_DOWN_STATE) {
+        addReplySds(c,sdsnew("-CLUSTERDOWN The cluster is down\r\n"));
+    } else if (error_code == REDIS_CLUSTER_REDIR_DOWN_UNBOUND) {
+        addReplySds(c,sdsnew("-CLUSTERDOWN Hash slot not served\r\n"));
+    } else if (error_code == REDIS_CLUSTER_REDIR_MOVED ||
+               error_code == REDIS_CLUSTER_REDIR_ASK)
+    {
+        addReplySds(c,sdscatprintf(sdsempty(),
+            "-%s %d %s:%d\r\n",
+            (error_code == REDIS_CLUSTER_REDIR_ASK) ? "ASK" : "MOVED",
+            hashslot,n->ip,n->port));
+    } else {
+        redisPanic("getNodeByQuery() unknown error.");
+    }
+}
+
+/* This function is called by the function processing clients incrementally
+ * to detect timeouts, in order to handle the following case:
+ *
+ * 1) A client blocks with BLPOP or similar blocking operation.
+ * 2) The master migrates the hash slot elsewhere or turns into a slave.
+ * 3) The client may remain blocked forever (or up to the max timeout time)
+ *    waiting for a key change that will never happen.
+ *
+ * If the client is found to be blocked into an hash slot this node no
+ * longer handles, the client is sent a redirection error, and the function
+ * returns 1. Otherwise 0 is returned and no operation is performed. */
+int clusterRedirectBlockedClientIfNeeded(redisClient *c) {
+    if (c->flags & REDIS_BLOCKED && c->btype == REDIS_BLOCKED_LIST) {
+        dictEntry *de;
+        dictIterator *di;
+
+        /* If the cluster is down, unblock the client with the right error. */
+        if (server.cluster->state == REDIS_CLUSTER_FAIL) {
+            clusterRedirectClient(c,NULL,0,REDIS_CLUSTER_REDIR_DOWN_STATE);
+            return 1;
+        }
+
+        di = dictGetIterator(c->bpop.keys);
+        while((de = dictNext(di)) != NULL) {
+            robj *key = dictGetKey(de);
+            int slot = keyHashSlot((char*)key->ptr, sdslen(key->ptr));
+            clusterNode *node = server.cluster->slots[slot];
+
+            /* We send an error and unblock the client if:
+             * 1) The slot is unassigned, emitting a cluster down error.
+             * 2) The slot is not handled by this node, nor being imported. */
+            if (node != myself &&
+                server.cluster->importing_slots_from[slot] == NULL)
+            {
+                if (node == NULL) {
+                    clusterRedirectClient(c,NULL,0,
+                        REDIS_CLUSTER_REDIR_DOWN_UNBOUND);
+                } else {
+                    clusterRedirectClient(c,node,slot,
+                        REDIS_CLUSTER_REDIR_MOVED);
+                }
+                return 1;
+            }
+        }
+        dictReleaseIterator(di);
+    }
+    return 0;
+}

src/cluster.h

@@ -26,10 +26,12 @@
 
 /* Redirection errors returned by getNodeByQuery(). */
 #define REDIS_CLUSTER_REDIR_NONE 0          /* Node can serve the request. */
-#define REDIS_CLUSTER_REDIR_CROSS_SLOT 1    /* Keys in different slots. */
-#define REDIS_CLUSTER_REDIR_UNSTABLE 2      /* Keys in slot resharding. */
+#define REDIS_CLUSTER_REDIR_CROSS_SLOT 1    /* -CROSSSLOT request. */
+#define REDIS_CLUSTER_REDIR_UNSTABLE 2      /* -TRYAGAIN redirection required */
 #define REDIS_CLUSTER_REDIR_ASK 3           /* -ASK redirection required. */
 #define REDIS_CLUSTER_REDIR_MOVED 4         /* -MOVED redirection required. */
+#define REDIS_CLUSTER_REDIR_DOWN_STATE 5    /* -CLUSTERDOWN, global state. */
+#define REDIS_CLUSTER_REDIR_DOWN_UNBOUND 6  /* -CLUSTERDOWN, unbound slot. */
 
 struct clusterNode;
 
@@ -248,5 +250,7 @@ typedef struct {
 
 /* ---------------------- API exported outside cluster.c -------------------- */
 clusterNode *getNodeByQuery(redisClient *c, struct redisCommand *cmd, robj **argv, int argc, int *hashslot, int *ask);
+int clusterRedirectBlockedClientIfNeeded(redisClient *c);
+void clusterRedirectClient(redisClient *c, clusterNode *n, int hashslot, int error_code);
 
 #endif /* __REDIS_CLUSTER_H */

src/config.c

@@ -34,10 +34,26 @@
 #include <fcntl.h>
 #include <sys/stat.h>
 
-static struct {
-    const char     *name;
-    const int       value;
-} validSyslogFacilities[] = {
+/*-----------------------------------------------------------------------------
+ * Config file name-value maps.
+ *----------------------------------------------------------------------------*/
+
+typedef struct configEnum {
+    const char *name;
+    const int val;
+} configEnum;
+
+configEnum maxmemory_policy_enum[] = {
+    {"volatile-lru", REDIS_MAXMEMORY_VOLATILE_LRU},
+    {"volatile-random",REDIS_MAXMEMORY_VOLATILE_RANDOM},
+    {"volatile-ttl",REDIS_MAXMEMORY_VOLATILE_TTL},
+    {"allkeys-lru",REDIS_MAXMEMORY_ALLKEYS_LRU},
+    {"allkeys-random",REDIS_MAXMEMORY_ALLKEYS_RANDOM},
+    {"noeviction",REDIS_MAXMEMORY_NO_EVICTION},
+    {NULL, 0}
+};
+
+configEnum syslog_facility_enum[] = {
     {"user",    LOG_USER},
     {"local0",  LOG_LOCAL0},
     {"local1",  LOG_LOCAL1},
@@ -50,6 +66,30 @@ static struct {
     {NULL, 0}
 };
 
+configEnum loglevel_enum[] = {
+    {"debug", REDIS_DEBUG},
+    {"verbose", REDIS_VERBOSE},
+    {"notice", REDIS_NOTICE},
+    {"warning", REDIS_WARNING},
+    {NULL,0}
+};
+
+configEnum supervised_mode_enum[] = {
+    {"upstart", REDIS_SUPERVISED_UPSTART},
+    {"systemd", REDIS_SUPERVISED_SYSTEMD},
+    {"auto", REDIS_SUPERVISED_AUTODETECT},
+    {"no", REDIS_SUPERVISED_NONE},
+    {NULL, 0}
+};
+
+configEnum aof_fsync_enum[] = {
+    {"everysec", AOF_FSYNC_EVERYSEC},
+    {"always", AOF_FSYNC_ALWAYS},
+    {"no", AOF_FSYNC_NO},
+    {NULL, 0}
+};
+
+/* Output buffer limits presets. */
 clientBufferLimitsConfig clientBufferLimitsDefaults[REDIS_CLIENT_TYPE_COUNT] = {
     {0, 0, 0}, /* normal */
     {1024*1024*256, 1024*1024*64, 60}, /* slave */
@@ -57,10 +97,42 @@ clientBufferLimitsConfig clientBufferLimitsDefaults[REDIS_CLIENT_TYPE_COUNT] = {
 };
 
 /*-----------------------------------------------------------------------------
- * Config file parsing
+ * Enum access functions
  *----------------------------------------------------------------------------*/
 
-int supervisedToMode(const char *str);
+/* Get enum value from name. If there is no match INT_MIN is returned. */
+int configEnumGetValue(configEnum *ce, char *name) {
+    while(ce->name != NULL) {
+        if (!strcasecmp(ce->name,name)) return ce->val;
+        ce++;
+    }
+    return INT_MIN;
+}
+
+/* Get enum name from value. If no match is found NULL is returned. */
+const char *configEnumGetName(configEnum *ce, int val) {
+    while(ce->name != NULL) {
+        if (ce->val == val) return ce->name;
+        ce++;
+    }
+    return NULL;
+}
+
+/* Wrapper for configEnumGetName() returning "unknown" insetad of NULL if
+ * there is no match. */
+const char *configEnumGetNameOrUnknown(configEnum *ce, int val) {
+    const char *name = configEnumGetName(ce,val);
+    return name ? name : "unknown";
+}
+
+/* Used for INFO generation. */
+const char *maxmemoryToString(void) {
+    return configEnumGetNameOrUnknown(maxmemory_policy_enum,server.maxmemory);
+}
+
+/*-----------------------------------------------------------------------------
+ * Config file parsing
+ *----------------------------------------------------------------------------*/
 
 int yesnotoi(char *s) {
     if (!strcasecmp(s,"yes")) return 1;
@@ -169,12 +241,10 @@ void loadServerConfigFromString(char *config) {
                 exit(1);
             }
         } else if (!strcasecmp(argv[0],"loglevel") && argc == 2) {
-            if (!strcasecmp(argv[1],"debug")) server.verbosity = REDIS_DEBUG;
-            else if (!strcasecmp(argv[1],"verbose")) server.verbosity = REDIS_VERBOSE;
-            else if (!strcasecmp(argv[1],"notice")) server.verbosity = REDIS_NOTICE;
-            else if (!strcasecmp(argv[1],"warning")) server.verbosity = REDIS_WARNING;
-            else {
-                err = "Invalid log level. Must be one of debug, notice, warning";
+            server.verbosity = configEnumGetValue(loglevel_enum,argv[1]);
+            if (server.verbosity == INT_MIN) {
+                err = "Invalid log level. "
+                      "Must be one of debug, verbose, notice, warning";
                 goto loaderr;
             }
         } else if (!strcasecmp(argv[0],"logfile") && argc == 2) {
@@ -201,16 +271,9 @@ void loadServerConfigFromString(char *config) {
             if (server.syslog_ident) zfree(server.syslog_ident);
             server.syslog_ident = zstrdup(argv[1]);
         } else if (!strcasecmp(argv[0],"syslog-facility") && argc == 2) {
-            int i;
-
-            for (i = 0; validSyslogFacilities[i].name; i++) {
-                if (!strcasecmp(validSyslogFacilities[i].name, argv[1])) {
-                    server.syslog_facility = validSyslogFacilities[i].value;
-                    break;
-                }
-            }
-
-            if (!validSyslogFacilities[i].name) {
+            server.syslog_facility =
+                configEnumGetValue(syslog_facility_enum,argv[1]);
+            if (server.syslog_facility == INT_MIN) {
                 err = "Invalid log facility. Must be one of USER or between LOCAL0-LOCAL7";
                 goto loaderr;
             }
@@ -229,19 +292,9 @@ void loadServerConfigFromString(char *config) {
         } else if (!strcasecmp(argv[0],"maxmemory") && argc == 2) {
             server.maxmemory = memtoll(argv[1],NULL);
         } else if (!strcasecmp(argv[0],"maxmemory-policy") && argc == 2) {
-            if (!strcasecmp(argv[1],"volatile-lru")) {
-                server.maxmemory_policy = REDIS_MAXMEMORY_VOLATILE_LRU;
-            } else if (!strcasecmp(argv[1],"volatile-random")) {
-                server.maxmemory_policy = REDIS_MAXMEMORY_VOLATILE_RANDOM;
-            } else if (!strcasecmp(argv[1],"volatile-ttl")) {
-                server.maxmemory_policy = REDIS_MAXMEMORY_VOLATILE_TTL;
-            } else if (!strcasecmp(argv[1],"allkeys-lru")) {
-                server.maxmemory_policy = REDIS_MAXMEMORY_ALLKEYS_LRU;
-            } else if (!strcasecmp(argv[1],"allkeys-random")) {
-                server.maxmemory_policy = REDIS_MAXMEMORY_ALLKEYS_RANDOM;
-            } else if (!strcasecmp(argv[1],"noeviction")) {
-                server.maxmemory_policy = REDIS_MAXMEMORY_NO_EVICTION;
-            } else {
+            server.maxmemory_policy =
+                configEnumGetValue(maxmemory_policy_enum,argv[1]);
+            if (server.maxmemory_policy == INT_MIN) {
                 err = "Invalid maxmemory policy";
                 goto loaderr;
             }
@@ -276,6 +329,10 @@ void loadServerConfigFromString(char *config) {
             if ((server.repl_diskless_sync = yesnotoi(argv[1])) == -1) {
                 err = "argument must be 'yes' or 'no'"; goto loaderr;
             }
+        } else if (!strcasecmp(argv[0],"repl-diskless-load") && argc==2) {
+            if ((server.repl_diskless_load = yesnotoi(argv[1])) == -1) {
+                err = "argument must be 'yes' or 'no'"; goto loaderr;
+            }
         } else if (!strcasecmp(argv[0],"repl-diskless-sync-delay") && argc==2) {
             server.repl_diskless_sync_delay = atoi(argv[1]);
             if (server.repl_diskless_sync_delay < 0) {
@@ -345,13 +402,8 @@ void loadServerConfigFromString(char *config) {
                 err = "argument must be 'yes' or 'no'"; goto loaderr;
             }
         } else if (!strcasecmp(argv[0],"appendfsync") && argc == 2) {
-            if (!strcasecmp(argv[1],"no")) {
-                server.aof_fsync = AOF_FSYNC_NO;
-            } else if (!strcasecmp(argv[1],"always")) {
-                server.aof_fsync = AOF_FSYNC_ALWAYS;
-            } else if (!strcasecmp(argv[1],"everysec")) {
-                server.aof_fsync = AOF_FSYNC_EVERYSEC;
-            } else {
+            server.aof_fsync = configEnumGetValue(aof_fsync_enum,argv[1]);
+            if (server.aof_fsync == INT_MIN) {
                 err = "argument must be 'no', 'always' or 'everysec'";
                 goto loaderr;
             }
@@ -536,14 +588,14 @@ void loadServerConfigFromString(char *config) {
             }
             server.notify_keyspace_events = flags;
         } else if (!strcasecmp(argv[0],"supervised") && argc == 2) {
-            int mode = supervisedToMode(argv[1]);
+            server.supervised_mode =
+                configEnumGetValue(supervised_mode_enum,argv[1]);
 
-            if (mode == -1) {
+            if (server.supervised_mode == INT_MIN) {
                 err = "Invalid option for 'supervised'. "
                     "Allowed values: 'upstart', 'systemd', 'auto', or 'no'";
                 goto loaderr;
             }
-            server.supervised_mode = mode;
         } else if (!strcasecmp(argv[0],"sentinel")) {
             /* argc == 1 is handled by main() as we need to enter the sentinel
              * mode ASAP. */
@@ -621,38 +673,62 @@ void loadServerConfig(char *filename, char *options) {
  * CONFIG SET implementation
  *----------------------------------------------------------------------------*/
 
+#define config_set_bool_field(_name,_var) \
+    } else if (!strcasecmp(c->argv[2]->ptr,_name)) { \
+        int yn = yesnotoi(o->ptr); \
+        if (yn == -1) goto badfmt; \
+        _var = yn;
+
+#define config_set_numerical_field(_name,_var,min,max) \
+    } else if (!strcasecmp(c->argv[2]->ptr,_name)) { \
+        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt; \
+        if (min != LLONG_MIN && ll < min) goto badfmt; \
+        if (max != LLONG_MAX && ll > max) goto badfmt; \
+        _var = ll;
+
+#define config_set_memory_field(_name,_var) \
+    } else if (!strcasecmp(c->argv[2]->ptr,_name)) { \
+        ll = memtoll(o->ptr,&err); \
+        if (err || ll < 0) goto badfmt; \
+        _var = ll;
+
+#define config_set_enum_field(_name,_var,_enumvar) \
+    } else if (!strcasecmp(c->argv[2]->ptr,_name)) { \
+        int enumval = configEnumGetValue(_enumvar,o->ptr); \
+        if (enumval == INT_MIN) goto badfmt; \
+        _var = enumval;
+
+#define config_set_special_field(_name) \
+    } else if (!strcasecmp(c->argv[2]->ptr,_name)) {
+
+#define config_set_else } else
+
 void configSetCommand(redisClient *c) {
     robj *o;
     long long ll;
+    int err;
     redisAssertWithInfo(c,c->argv[2],sdsEncodedObject(c->argv[2]));
     redisAssertWithInfo(c,c->argv[3],sdsEncodedObject(c->argv[3]));
     o = c->argv[3];
 
-    if (!strcasecmp(c->argv[2]->ptr,"dbfilename")) {
+    if (0) { /* this starts the config_set macros else-if chain. */
+
+    /* Special fields that can't be handled with general macros. */
+    config_set_special_field("dbfilename") {
         if (!pathIsBaseName(o->ptr)) {
             addReplyError(c, "dbfilename can't be a path, just a filename");
             return;
         }
         zfree(server.rdb_filename);
         server.rdb_filename = zstrdup(o->ptr);
-    } else if (!strcasecmp(c->argv[2]->ptr,"requirepass")) {
+    } config_set_special_field("requirepass") {
         if (sdslen(o->ptr) > REDIS_AUTHPASS_MAX_LEN) goto badfmt;
         zfree(server.requirepass);
         server.requirepass = ((char*)o->ptr)[0] ? zstrdup(o->ptr) : NULL;
-    } else if (!strcasecmp(c->argv[2]->ptr,"masterauth")) {
+    } config_set_special_field("masterauth") {
         zfree(server.masterauth);
         server.masterauth = ((char*)o->ptr)[0] ? zstrdup(o->ptr) : NULL;
-    } else if (!strcasecmp(c->argv[2]->ptr,"maxmemory")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll < 0) goto badfmt;
-        server.maxmemory = ll;
-        if (server.maxmemory) {
-            if (server.maxmemory < zmalloc_used_memory()) {
-                redisLog(REDIS_WARNING,"WARNING: the new maxmemory value set via CONFIG SET is smaller than the current memory usage. This will result in keys eviction and/or inability to accept new write commands depending on the maxmemory-policy.");
-            }
-            freeMemoryIfNeeded();
-        }
-    } else if (!strcasecmp(c->argv[2]->ptr,"maxclients")) {
+    } config_set_special_field("maxclients") {
         int orig_value = server.maxclients;
 
         if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 1) goto badfmt;
@@ -678,55 +754,7 @@ void configSetCommand(redisClient *c) {
                 }
             }
         }
-    } else if (!strcasecmp(c->argv[2]->ptr,"hz")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.hz = ll;
-        if (server.hz < REDIS_MIN_HZ) server.hz = REDIS_MIN_HZ;
-        if (server.hz > REDIS_MAX_HZ) server.hz = REDIS_MAX_HZ;
-    } else if (!strcasecmp(c->argv[2]->ptr,"maxmemory-policy")) {
-        if (!strcasecmp(o->ptr,"volatile-lru")) {
-            server.maxmemory_policy = REDIS_MAXMEMORY_VOLATILE_LRU;
-        } else if (!strcasecmp(o->ptr,"volatile-random")) {
-            server.maxmemory_policy = REDIS_MAXMEMORY_VOLATILE_RANDOM;
-        } else if (!strcasecmp(o->ptr,"volatile-ttl")) {
-            server.maxmemory_policy = REDIS_MAXMEMORY_VOLATILE_TTL;
-        } else if (!strcasecmp(o->ptr,"allkeys-lru")) {
-            server.maxmemory_policy = REDIS_MAXMEMORY_ALLKEYS_LRU;
-        } else if (!strcasecmp(o->ptr,"allkeys-random")) {
-            server.maxmemory_policy = REDIS_MAXMEMORY_ALLKEYS_RANDOM;
-        } else if (!strcasecmp(o->ptr,"noeviction")) {
-            server.maxmemory_policy = REDIS_MAXMEMORY_NO_EVICTION;
-        } else {
-            goto badfmt;
-        }
-    } else if (!strcasecmp(c->argv[2]->ptr,"maxmemory-samples")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll <= 0) goto badfmt;
-        server.maxmemory_samples = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"timeout")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll < 0 || ll > LONG_MAX) goto badfmt;
-        server.maxidletime = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"tcp-keepalive")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll < 0 || ll > INT_MAX) goto badfmt;
-        server.tcpkeepalive = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"appendfsync")) {
-        if (!strcasecmp(o->ptr,"no")) {
-            server.aof_fsync = AOF_FSYNC_NO;
-        } else if (!strcasecmp(o->ptr,"everysec")) {
-            server.aof_fsync = AOF_FSYNC_EVERYSEC;
-        } else if (!strcasecmp(o->ptr,"always")) {
-            server.aof_fsync = AOF_FSYNC_ALWAYS;
-        } else {
-            goto badfmt;
-        }
-    } else if (!strcasecmp(c->argv[2]->ptr,"no-appendfsync-on-rewrite")) {
-        int yn = yesnotoi(o->ptr);
-
-        if (yn == -1) goto badfmt;
-        server.aof_no_fsync_on_rewrite = yn;
-    } else if (!strcasecmp(c->argv[2]->ptr,"appendonly")) {
+    } config_set_special_field("appendonly") {
         int enable = yesnotoi(o->ptr);
 
         if (enable == -1) goto badfmt;
@@ -739,23 +767,7 @@ void configSetCommand(redisClient *c) {
                 return;
             }
         }
-    } else if (!strcasecmp(c->argv[2]->ptr,"auto-aof-rewrite-percentage")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.aof_rewrite_perc = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"auto-aof-rewrite-min-size")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.aof_rewrite_min_size = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"aof-rewrite-incremental-fsync")) {
-        int yn = yesnotoi(o->ptr);
-
-        if (yn == -1) goto badfmt;
-        server.aof_rewrite_incremental_fsync = yn;
-    } else if (!strcasecmp(c->argv[2]->ptr,"aof-load-truncated")) {
-        int yn = yesnotoi(o->ptr);
-
-        if (yn == -1) goto badfmt;
-        server.aof_load_truncated = yn;
-    } else if (!strcasecmp(c->argv[2]->ptr,"save")) {
+    } config_set_special_field("save") {
         int vlen, j;
         sds *v = sdssplitlen(o->ptr,sdslen(o->ptr)," ",1,&vlen);
 
@@ -789,70 +801,12 @@ void configSetCommand(redisClient *c) {
             appendServerSaveParams(seconds, changes);
         }
         sdsfreesplitres(v,vlen);
-    } else if (!strcasecmp(c->argv[2]->ptr,"slave-serve-stale-data")) {
-        int yn = yesnotoi(o->ptr);
-
-        if (yn == -1) goto badfmt;
-        server.repl_serve_stale_data = yn;
-    } else if (!strcasecmp(c->argv[2]->ptr,"slave-read-only")) {
-        int yn = yesnotoi(o->ptr);
-
-        if (yn == -1) goto badfmt;
-        server.repl_slave_ro = yn;
-    } else if (!strcasecmp(c->argv[2]->ptr,"dir")) {
+    } config_set_special_field("dir") {
         if (chdir((char*)o->ptr) == -1) {
             addReplyErrorFormat(c,"Changing directory: %s", strerror(errno));
             return;
         }
-    } else if (!strcasecmp(c->argv[2]->ptr,"hash-max-ziplist-entries")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.hash_max_ziplist_entries = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"hash-max-ziplist-value")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.hash_max_ziplist_value = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"list-max-ziplist-size")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.list_max_ziplist_size = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"list-compress-depth")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.list_compress_depth = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"set-max-intset-entries")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.set_max_intset_entries = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"zset-max-ziplist-entries")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.zset_max_ziplist_entries = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"zset-max-ziplist-value")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.zset_max_ziplist_value = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"hll-sparse-max-bytes")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.hll_sparse_max_bytes = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"lua-time-limit")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.lua_time_limit = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"slowlog-log-slower-than")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR) goto badfmt;
-        server.slowlog_log_slower_than = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"slowlog-max-len")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.slowlog_max_len = (unsigned)ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"latency-monitor-threshold")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.latency_monitor_threshold = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"loglevel")) {
-        if (!strcasecmp(o->ptr,"warning")) {
-            server.verbosity = REDIS_WARNING;
-        } else if (!strcasecmp(o->ptr,"notice")) {
-            server.verbosity = REDIS_NOTICE;
-        } else if (!strcasecmp(o->ptr,"verbose")) {
-            server.verbosity = REDIS_VERBOSE;
-        } else if (!strcasecmp(o->ptr,"debug")) {
-            server.verbosity = REDIS_DEBUG;
-        } else {
-            goto badfmt;
-        }
-    } else if (!strcasecmp(c->argv[2]->ptr,"client-output-buffer-limit")) {
+    } config_set_special_field("client-output-buffer-limit") {
         int vlen, j;
         sds *v = sdssplitlen(o->ptr,sdslen(o->ptr)," ",1,&vlen);
 
@@ -866,7 +820,6 @@ void configSetCommand(redisClient *c) {
          * whole configuration string or accept it all, even if a single
          * error in a single client class is present. */
         for (j = 0; j < vlen; j++) {
-            char *eptr;
             long val;
 
             if ((j % 4) == 0) {
@@ -875,8 +828,8 @@ void configSetCommand(redisClient *c) {
                     goto badfmt;
                 }
             } else {
-                val = strtoll(v[j], &eptr, 10);
-                if (eptr[0] != '\0' || val < 0) {
+                val = memtoll(v[j], &err);
+                if (err || val < 0) {
                     sdsfreesplitres(v,vlen);
                     goto badfmt;
                 }
@@ -898,89 +851,139 @@ void configSetCommand(redisClient *c) {
             server.client_obuf_limits[class].soft_limit_seconds = soft_seconds;
         }
         sdsfreesplitres(v,vlen);
-    } else if (!strcasecmp(c->argv[2]->ptr,"stop-writes-on-bgsave-error")) {
-        int yn = yesnotoi(o->ptr);
-
-        if (yn == -1) goto badfmt;
-        server.stop_writes_on_bgsave_err = yn;
-    } else if (!strcasecmp(c->argv[2]->ptr,"repl-ping-slave-period")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll <= 0) goto badfmt;
-        server.repl_ping_slave_period = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"repl-timeout")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll <= 0) goto badfmt;
-        server.repl_timeout = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"repl-backlog-size")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll <= 0) goto badfmt;
-        resizeReplicationBacklog(ll);
-    } else if (!strcasecmp(c->argv[2]->ptr,"repl-backlog-ttl")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        server.repl_backlog_time_limit = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"watchdog-period")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
-        if (ll)
-            enableWatchdog(ll);
-        else
-            disableWatchdog();
-    } else if (!strcasecmp(c->argv[2]->ptr,"rdbcompression")) {
-        int yn = yesnotoi(o->ptr);
-
-        if (yn == -1) goto badfmt;
-        server.rdb_compression = yn;
-    } else if (!strcasecmp(c->argv[2]->ptr,"notify-keyspace-events")) {
+    } config_set_special_field("notify-keyspace-events") {
         int flags = keyspaceEventsStringToFlags(o->ptr);
 
         if (flags == -1) goto badfmt;
         server.notify_keyspace_events = flags;
-    } else if (!strcasecmp(c->argv[2]->ptr,"repl-disable-tcp-nodelay")) {
-        int yn = yesnotoi(o->ptr);
 
-        if (yn == -1) goto badfmt;
-        server.repl_disable_tcp_nodelay = yn;
-    } else if (!strcasecmp(c->argv[2]->ptr,"repl-diskless-sync")) {
-        int yn = yesnotoi(o->ptr);
+    /* Boolean fields.
+     * config_set_bool_field(name,var). */
+    } config_set_bool_field(
+      "rdbcompression", server.rdb_compression) {
+    } config_set_bool_field(
+      "repl-disable-tcp-nodelay",server.repl_disable_tcp_nodelay) {
+    } config_set_bool_field(
+      "repl-diskless-sync",server.repl_diskless_sync) {
+    } config_set_bool_field(
+      "repl-diskless-load",server.repl_diskless_load) {
+    } config_set_bool_field(
+      "cluster-require-full-coverage",server.cluster_require_full_coverage) {
+    } config_set_bool_field(
+      "aof-rewrite-incremental-fsync",server.aof_rewrite_incremental_fsync) {
+    } config_set_bool_field(
+      "aof-load-truncated",server.aof_load_truncated) {
+    } config_set_bool_field(
+      "slave-serve-stale-data",server.repl_serve_stale_data) {
+    } config_set_bool_field(
+      "slave-read-only",server.repl_slave_ro) {
+    } config_set_bool_field(
+      "activerehashing",server.activerehashing) {
+    } config_set_bool_field(
+      "stop-writes-on-bgsave-error",server.stop_writes_on_bgsave_err) {
 
-        if (yn == -1) goto badfmt;
-        server.repl_diskless_sync = yn;
-    } else if (!strcasecmp(c->argv[2]->ptr,"repl-diskless-sync-delay")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll < 0) goto badfmt;
-        server.repl_diskless_sync_delay = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"slave-priority")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll < 0) goto badfmt;
-        server.slave_priority = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"min-slaves-to-write")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll < 0) goto badfmt;
-        server.repl_min_slaves_to_write = ll;
+    /* Numerical fields.
+     * config_set_numerical_field(name,var,min,max) */
+    } config_set_numerical_field(
+      "tcp-keepalive",server.tcpkeepalive,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "maxmemory-samples",server.maxmemory_samples,1,LLONG_MAX) {
+    } config_set_numerical_field(
+      "timeout",server.maxidletime,0,LONG_MAX) {
+    } config_set_numerical_field(
+      "auto-aof-rewrite-percentage",server.aof_rewrite_perc,0,LLONG_MAX){
+    } config_set_numerical_field(
+      "auto-aof-rewrite-min-size",server.aof_rewrite_min_size,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "hash-max-ziplist-entries",server.hash_max_ziplist_entries,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "hash-max-ziplist-value",server.hash_max_ziplist_value,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "list-max-ziplist-size",server.list_max_ziplist_size,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "list-compress-depth",server.list_compress_depth,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "set-max-intset-entries",server.set_max_intset_entries,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "zset-max-ziplist-entries",server.zset_max_ziplist_entries,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "zset-max-ziplist-value",server.zset_max_ziplist_value,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "hll-sparse-max-bytes",server.hll_sparse_max_bytes,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "lua-time-limit",server.lua_time_limit,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "slowlog-log-slower-than",server.slowlog_log_slower_than,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "slowlog-max-len",ll,0,LLONG_MAX) {
+      /* Cast to unsigned. */
+        server.slowlog_max_len = (unsigned)ll;
+    } config_set_numerical_field(
+      "latency-monitor-threshold",server.latency_monitor_threshold,0,LLONG_MAX){
+    } config_set_numerical_field(
+      "repl-ping-slave-period",server.repl_ping_slave_period,1,LLONG_MAX) {
+    } config_set_numerical_field(
+      "repl-timeout",server.repl_timeout,1,LLONG_MAX) {
+    } config_set_numerical_field(
+      "repl-backlog-ttl",server.repl_backlog_time_limit,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "repl-diskless-sync-delay",server.repl_diskless_sync_delay,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "slave-priority",server.slave_priority,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "min-slaves-to-write",server.repl_min_slaves_to_write,0,LLONG_MAX) {
         refreshGoodSlavesCount();
-    } else if (!strcasecmp(c->argv[2]->ptr,"min-slaves-max-lag")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll < 0) goto badfmt;
-        server.repl_min_slaves_max_lag = ll;
+    } config_set_numerical_field(
+      "min-slaves-max-lag",server.repl_min_slaves_max_lag,0,LLONG_MAX) {
         refreshGoodSlavesCount();
-    } else if (!strcasecmp(c->argv[2]->ptr,"cluster-require-full-coverage")) {
-        int yn = yesnotoi(o->ptr);
+    } config_set_numerical_field(
+      "cluster-node-timeout",server.cluster_node_timeout,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "cluster-migration-barrier",server.cluster_migration_barrier,0,LLONG_MAX){
+    } config_set_numerical_field(
+      "cluster-slave-validity-factor",server.cluster_slave_validity_factor,0,LLONG_MAX) {
+    } config_set_numerical_field(
+      "hz",server.hz,0,LLONG_MAX) {
+        /* Hz is more an hint from the user, so we accept values out of range
+         * but cap them to reasonable values. */
+        if (server.hz < REDIS_MIN_HZ) server.hz = REDIS_MIN_HZ;
+        if (server.hz > REDIS_MAX_HZ) server.hz = REDIS_MAX_HZ;
+    } config_set_numerical_field(
+      "watchdog-period",ll,0,LLONG_MAX) {
+        if (ll)
+            enableWatchdog(ll);
+        else
+            disableWatchdog();
 
-        if (yn == -1) goto badfmt;
-        server.cluster_require_full_coverage = yn;
-    } else if (!strcasecmp(c->argv[2]->ptr,"cluster-node-timeout")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll <= 0) goto badfmt;
-        server.cluster_node_timeout = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"cluster-migration-barrier")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll < 0) goto badfmt;
-        server.cluster_migration_barrier = ll;
-    } else if (!strcasecmp(c->argv[2]->ptr,"cluster-slave-validity-factor")) {
-        if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
-            ll < 0) goto badfmt;
-        server.cluster_slave_validity_factor = ll;
-    } else {
+    /* Memory fields.
+     * config_set_memory_field(name,var) */
+    } config_set_memory_field("maxmemory",server.maxmemory) {
+        if (server.maxmemory) {
+            if (server.maxmemory < zmalloc_used_memory()) {
+                redisLog(REDIS_WARNING,"WARNING: the new maxmemory value set via CONFIG SET is smaller than the current memory usage. This will result in keys eviction and/or inability to accept new write commands depending on the maxmemory-policy.");
+            }
+            freeMemoryIfNeeded();
+        }
+    } config_set_memory_field("repl-backlog-size",ll) {
+        resizeReplicationBacklog(ll);
+
+    /* Enumeration fields.
+     * config_set_enum_field(name,var,enum_var) */
+    } config_set_enum_field(
+      "loglevel",server.verbosity,loglevel_enum) {
+    } config_set_enum_field(
+      "maxmemory-policy",server.maxmemory_policy,maxmemory_policy_enum) {
+    } config_set_enum_field(
+      "appendfsync",server.aof_fsync,aof_fsync_enum) {
+
+    /* Everyhing else is an error... */
+    } config_set_else {
         addReplyErrorFormat(c,"Unsupported CONFIG parameter: %s",
             (char*)c->argv[2]->ptr);
         return;
     }
+
+    /* On success we just return a generic OK for all the options. */
     addReply(c,shared.ok);
     return;
 
@@ -1019,47 +1022,14 @@ badfmt: /* Bad format errors */
     } \
 } while(0);
 
-char *maxmemoryToString() {
-    char *s;
-    switch(server.maxmemory_policy) {
-    case REDIS_MAXMEMORY_VOLATILE_LRU: s = "volatile-lru"; break;
-    case REDIS_MAXMEMORY_VOLATILE_TTL: s = "volatile-ttl"; break;
-    case REDIS_MAXMEMORY_VOLATILE_RANDOM: s = "volatile-random"; break;
-    case REDIS_MAXMEMORY_ALLKEYS_LRU: s = "allkeys-lru"; break;
-    case REDIS_MAXMEMORY_ALLKEYS_RANDOM: s = "allkeys-random"; break;
-    case REDIS_MAXMEMORY_NO_EVICTION: s = "noeviction"; break;
-    default: s = "unknown"; break;
-    }
-    return s;
-}
+#define config_get_enum_field(_name,_var,_enumvar) do { \
+    if (stringmatch(pattern,_name,0)) { \
+        addReplyBulkCString(c,_name); \
+        addReplyBulkCString(c,configEnumGetNameOrUnknown(_enumvar,_var)); \
+        matches++; \
+    } \
+} while(0);
 
-int supervisedToMode(const char *str) {
-    int mode;
-    if (!strcasecmp(str,"upstart")) {
-        mode = REDIS_SUPERVISED_UPSTART;
-    } else if (!strcasecmp(str,"systemd")) {
-        mode = REDIS_SUPERVISED_SYSTEMD;
-    } else if (!strcasecmp(str,"auto")) {
-        mode = REDIS_SUPERVISED_AUTODETECT;
-    } else if (!strcasecmp(str,"no")) {
-        mode = REDIS_SUPERVISED_NONE;
-    } else {
-        mode = -1;
-    }
-    return mode;
-}
-
-char *supervisedToString(void) {
-    char *s;
-    switch(server.supervised_mode) {
-    case REDIS_SUPERVISED_UPSTART: s = "upstart"; break;
-    case REDIS_SUPERVISED_SYSTEMD: s = "systemd"; break;
-    case REDIS_SUPERVISED_AUTODETECT: s = "auto"; break;
-    case REDIS_SUPERVISED_NONE: s = "no"; break;
-    default: s = "no"; break;
-    }
-    return s;
-}
 void configGetCommand(redisClient *c) {
     robj *o = c->argv[2];
     void *replylen = addDeferredMultiBulkLength(c);
@@ -1080,7 +1050,6 @@ void configGetCommand(redisClient *c) {
     config_get_numerical_field("maxmemory",server.maxmemory);
     config_get_numerical_field("maxmemory-samples",server.maxmemory_samples);
     config_get_numerical_field("timeout",server.maxidletime);
-    config_get_numerical_field("tcp-keepalive",server.tcpkeepalive);
     config_get_numerical_field("auto-aof-rewrite-percentage",
             server.aof_rewrite_perc);
     config_get_numerical_field("auto-aof-rewrite-min-size",
@@ -1125,6 +1094,7 @@ void configGetCommand(redisClient *c) {
     config_get_numerical_field("cluster-migration-barrier",server.cluster_migration_barrier);
     config_get_numerical_field("cluster-slave-validity-factor",server.cluster_slave_validity_factor);
     config_get_numerical_field("repl-diskless-sync-delay",server.repl_diskless_sync_delay);
+    config_get_numerical_field("tcp-keepalive",server.tcpkeepalive);
 
     /* Bool (yes/no) values */
     config_get_bool_field("cluster-require-full-coverage",
@@ -1145,11 +1115,25 @@ void configGetCommand(redisClient *c) {
             server.repl_disable_tcp_nodelay);
     config_get_bool_field("repl-diskless-sync",
             server.repl_diskless_sync);
+    config_get_bool_field("repl-diskless-load",
+            server.repl_diskless_load);
     config_get_bool_field("aof-rewrite-incremental-fsync",
             server.aof_rewrite_incremental_fsync);
     config_get_bool_field("aof-load-truncated",
             server.aof_load_truncated);
 
+    /* Enum values */
+    config_get_enum_field("maxmemory-policy",
+            server.maxmemory_policy,maxmemory_policy_enum);
+    config_get_enum_field("loglevel",
+            server.verbosity,loglevel_enum);
+    config_get_enum_field("supervised",
+            server.supervised_mode,supervised_mode_enum);
+    config_get_enum_field("appendfsync",
+            server.aof_fsync,aof_fsync_enum);
+    config_get_enum_field("syslog-facility",
+            server.syslog_facility,syslog_facility_enum);
+
     /* Everything we can't handle with macros follows. */
 
     if (stringmatch(pattern,"appendonly",0)) {
@@ -1167,24 +1151,6 @@ void configGetCommand(redisClient *c) {
         addReplyBulkCString(c,buf);
         matches++;
     }
-    if (stringmatch(pattern,"maxmemory-policy",0)) {
-        addReplyBulkCString(c,"maxmemory-policy");
-        addReplyBulkCString(c,maxmemoryToString());
-        matches++;
-    }
-    if (stringmatch(pattern,"appendfsync",0)) {
-        char *policy;
-
-        switch(server.aof_fsync) {
-        case AOF_FSYNC_NO: policy = "no"; break;
-        case AOF_FSYNC_EVERYSEC: policy = "everysec"; break;
-        case AOF_FSYNC_ALWAYS: policy = "always"; break;
-        default: policy = "unknown"; break; /* too harmless to panic */
-        }
-        addReplyBulkCString(c,"appendfsync");
-        addReplyBulkCString(c,policy);
-        matches++;
-    }
     if (stringmatch(pattern,"save",0)) {
         sds buf = sdsempty();
         int j;
@@ -1201,25 +1167,6 @@ void configGetCommand(redisClient *c) {
         sdsfree(buf);
         matches++;
     }
-    if (stringmatch(pattern,"loglevel",0)) {
-        char *s;
-
-        switch(server.verbosity) {
-        case REDIS_WARNING: s = "warning"; break;
-        case REDIS_VERBOSE: s = "verbose"; break;
-        case REDIS_NOTICE: s = "notice"; break;
-        case REDIS_DEBUG: s = "debug"; break;
-        default: s = "unknown"; break; /* too harmless to panic */
-        }
-        addReplyBulkCString(c,"loglevel");
-        addReplyBulkCString(c,s);
-        matches++;
-    }
-    if (stringmatch(pattern,"supervised",0)) {
-        addReplyBulkCString(c,"supervised");
-        addReplyBulkCString(c,supervisedToString());
-        matches++;
-    }
     if (stringmatch(pattern,"client-output-buffer-limit",0)) {
         sds buf = sdsempty();
         int j;
@@ -1344,7 +1291,7 @@ void rewriteConfigAddLineNumberToOption(struct rewriteConfigState *state, sds op
  * This is useful as only unused lines of processed options will be blanked
  * in the config file, while options the rewrite process does not understand
  * remain untouched. */
-void rewriteConfigMarkAsProcessed(struct rewriteConfigState *state, char *option) {
+void rewriteConfigMarkAsProcessed(struct rewriteConfigState *state, const char *option) {
     sds opt = sdsnew(option);
 
     if (dictAdd(state->rewritten,opt,NULL) != DICT_OK) sdsfree(opt);
@@ -1428,7 +1375,7 @@ struct rewriteConfigState *rewriteConfigReadOldFile(char *path) {
  *
  * "line" is either used, or freed, so the caller does not need to free it
  * in any way. */
-void rewriteConfigRewriteLine(struct rewriteConfigState *state, char *option, sds line, int force) {
+void rewriteConfigRewriteLine(struct rewriteConfigState *state, const char *option, sds line, int force) {
     sds o = sdsnew(option);
     list *l = dictFetchValue(state->option_to_line,o);
 
@@ -1539,45 +1486,26 @@ void rewriteConfigOctalOption(struct rewriteConfigState *state, char *option, in
     rewriteConfigRewriteLine(state,option,line,force);
 }
 
-/* Rewrite an enumeration option, after the "value" every enum/value pair
- * is specified, terminated by NULL. After NULL the default value is
- * specified. See how the function is used for more information. */
-void rewriteConfigEnumOption(struct rewriteConfigState *state, char *option, int value, ...) {
-    va_list ap;
-    char *enum_name, *matching_name = NULL;
-    int enum_val, def_val, force;
+/* Rewrite an enumeration option. It takes as usually state and option name,
+ * and in addition the enumeration array and the default value for the
+ * option. */
+void rewriteConfigEnumOption(struct rewriteConfigState *state, char *option, int value, configEnum *ce, int defval) {
     sds line;
+    const char *name = configEnumGetNameOrUnknown(ce,value);
+    int force = value != defval;
 
-    va_start(ap, value);
-    while(1) {
-        enum_name = va_arg(ap,char*);
-        enum_val = va_arg(ap,int);
-        if (enum_name == NULL) {
-            def_val = enum_val;
-            break;
-        }
-        if (value == enum_val) matching_name = enum_name;
-    }
-    va_end(ap);
-
-    force = value != def_val;
-    line = sdscatprintf(sdsempty(),"%s %s",option,matching_name);
+    line = sdscatprintf(sdsempty(),"%s %s",option,name);
     rewriteConfigRewriteLine(state,option,line,force);
 }
 
 /* Rewrite the syslog-facility option. */
 void rewriteConfigSyslogfacilityOption(struct rewriteConfigState *state) {
-    int value = server.syslog_facility, j;
+    int value = server.syslog_facility;
     int force = value != LOG_LOCAL0;
-    char *name = NULL, *option = "syslog-facility";
+    const char *name = NULL, *option = "syslog-facility";
     sds line;
 
-    for (j = 0; validSyslogFacilities[j].name; j++) {
-        if (validSyslogFacilities[j].value == value) {
-            name = (char*) validSyslogFacilities[j].name;
-            break;
-        }
-    }
+    name = configEnumGetNameOrUnknown(syslog_facility_enum,value);
     line = sdscatprintf(sdsempty(),"%s %s",option,name);
     rewriteConfigRewriteLine(state,option,line,force);
 }
@@ -1838,12 +1766,7 @@ int rewriteConfig(char *path) {
     rewriteConfigOctalOption(state,"unixsocketperm",server.unixsocketperm,REDIS_DEFAULT_UNIX_SOCKET_PERM);
     rewriteConfigNumericalOption(state,"timeout",server.maxidletime,REDIS_MAXIDLETIME);
     rewriteConfigNumericalOption(state,"tcp-keepalive",server.tcpkeepalive,REDIS_DEFAULT_TCP_KEEPALIVE);
-    rewriteConfigEnumOption(state,"loglevel",server.verbosity,
-        "debug", REDIS_DEBUG,
-        "verbose", REDIS_VERBOSE,
-        "notice", REDIS_NOTICE,
-        "warning", REDIS_WARNING,
-        NULL, REDIS_DEFAULT_VERBOSITY);
+    rewriteConfigEnumOption(state,"loglevel",server.verbosity,loglevel_enum,REDIS_DEFAULT_VERBOSITY);
     rewriteConfigStringOption(state,"logfile",server.logfile,REDIS_DEFAULT_LOGFILE);
     rewriteConfigYesNoOption(state,"syslog-enabled",server.syslog_enabled,REDIS_DEFAULT_SYSLOG_ENABLED);
     rewriteConfigStringOption(state,"syslog-ident",server.syslog_ident,REDIS_DEFAULT_SYSLOG_IDENT);
@@ -1865,6 +1788,7 @@ int rewriteConfig(char *path) {
     rewriteConfigBytesOption(state,"repl-backlog-ttl",server.repl_backlog_time_limit,REDIS_DEFAULT_REPL_BACKLOG_TIME_LIMIT);
     rewriteConfigYesNoOption(state,"repl-disable-tcp-nodelay",server.repl_disable_tcp_nodelay,REDIS_DEFAULT_REPL_DISABLE_TCP_NODELAY);
     rewriteConfigYesNoOption(state,"repl-diskless-sync",server.repl_diskless_sync,REDIS_DEFAULT_REPL_DISKLESS_SYNC);
+    rewriteConfigYesNoOption(state,"repl-diskless-load",server.repl_diskless_load,REDIS_DEFAULT_REPL_DISKLESS_LOAD);
     rewriteConfigNumericalOption(state,"repl-diskless-sync-delay",server.repl_diskless_sync_delay,REDIS_DEFAULT_REPL_DISKLESS_SYNC_DELAY);
     rewriteConfigNumericalOption(state,"slave-priority",server.slave_priority,REDIS_DEFAULT_SLAVE_PRIORITY);
     rewriteConfigNumericalOption(state,"min-slaves-to-write",server.repl_min_slaves_to_write,REDIS_DEFAULT_MIN_SLAVES_TO_WRITE);
@@ -1872,22 +1796,11 @@ int rewriteConfig(char *path) {
     rewriteConfigStringOption(state,"requirepass",server.requirepass,NULL);
     rewriteConfigNumericalOption(state,"maxclients",server.maxclients,REDIS_MAX_CLIENTS);
     rewriteConfigBytesOption(state,"maxmemory",server.maxmemory,REDIS_DEFAULT_MAXMEMORY);
-    rewriteConfigEnumOption(state,"maxmemory-policy",server.maxmemory_policy,
-        "volatile-lru", REDIS_MAXMEMORY_VOLATILE_LRU,
-        "allkeys-lru", REDIS_MAXMEMORY_ALLKEYS_LRU,
-        "volatile-random", REDIS_MAXMEMORY_VOLATILE_RANDOM,
-        "allkeys-random", REDIS_MAXMEMORY_ALLKEYS_RANDOM,
-        "volatile-ttl", REDIS_MAXMEMORY_VOLATILE_TTL,
-        "noeviction", REDIS_MAXMEMORY_NO_EVICTION,
-        NULL, REDIS_DEFAULT_MAXMEMORY_POLICY);
+    rewriteConfigEnumOption(state,"maxmemory-policy",server.maxmemory_policy,maxmemory_policy_enum,REDIS_DEFAULT_MAXMEMORY_POLICY);
     rewriteConfigNumericalOption(state,"maxmemory-samples",server.maxmemory_samples,REDIS_DEFAULT_MAXMEMORY_SAMPLES);
     rewriteConfigYesNoOption(state,"appendonly",server.aof_state != REDIS_AOF_OFF,0);
     rewriteConfigStringOption(state,"appendfilename",server.aof_filename,REDIS_DEFAULT_AOF_FILENAME);
-    rewriteConfigEnumOption(state,"appendfsync",server.aof_fsync,
-        "everysec", AOF_FSYNC_EVERYSEC,
-        "always", AOF_FSYNC_ALWAYS,
-        "no", AOF_FSYNC_NO,
-        NULL, REDIS_DEFAULT_AOF_FSYNC);
+    rewriteConfigEnumOption(state,"appendfsync",server.aof_fsync,aof_fsync_enum,REDIS_DEFAULT_AOF_FSYNC);
     rewriteConfigYesNoOption(state,"no-appendfsync-on-rewrite",server.aof_no_fsync_on_rewrite,REDIS_DEFAULT_AOF_NO_FSYNC_ON_REWRITE);
     rewriteConfigNumericalOption(state,"auto-aof-rewrite-percentage",server.aof_rewrite_perc,REDIS_AOF_REWRITE_PERC);
     rewriteConfigBytesOption(state,"auto-aof-rewrite-min-size",server.aof_rewrite_min_size,REDIS_AOF_REWRITE_MIN_SIZE);
@@ -1915,12 +1828,9 @@ int rewriteConfig(char *path) {
     rewriteConfigNumericalOption(state,"hz",server.hz,REDIS_DEFAULT_HZ);
     rewriteConfigYesNoOption(state,"aof-rewrite-incremental-fsync",server.aof_rewrite_incremental_fsync,REDIS_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC);
     rewriteConfigYesNoOption(state,"aof-load-truncated",server.aof_load_truncated,REDIS_DEFAULT_AOF_LOAD_TRUNCATED);
-    rewriteConfigEnumOption(state,"supervised",server.supervised_mode,
-        "upstart", REDIS_SUPERVISED_UPSTART,
-        "systemd", REDIS_SUPERVISED_SYSTEMD,
-        "auto", REDIS_SUPERVISED_AUTODETECT,
-        "no", REDIS_SUPERVISED_NONE,
-        NULL, REDIS_SUPERVISED_NONE);
+    rewriteConfigEnumOption(state,"supervised",server.supervised_mode,supervised_mode_enum,REDIS_SUPERVISED_NONE);
+
+    /* Rewrite Sentinel config if in Sentinel mode. */
     if (server.sentinel_mode) rewriteConfigSentinelOption(state);
 
     /* Step 3: remove all the orphaned lines in the old file, that is, lines

src/config.h

@@ -34,6 +34,11 @@
 #include <AvailabilityMacros.h>
 #endif
 
+#ifdef __linux__
+#include <linux/version.h>
+#include <features.h>
+#endif
+
 /* Define redis_fstat to fstat or fstat64() */
 #if defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
 #define redis_fstat fstat64
@@ -92,8 +97,6 @@
 /* Define rdb_fsync_range to sync_file_range() on Linux, otherwise we use
  * the plain fsync() call. */
 #ifdef __linux__
-#include <linux/version.h>
-#include <features.h>
 #if defined(__GLIBC__) && defined(__GLIBC_PREREQ)
 #if (LINUX_VERSION_CODE >= 0x020611 && __GLIBC_PREREQ(2, 6))
 #define HAVE_SYNC_FILE_RANGE 1

src/db.c

@@ -318,13 +318,17 @@ void delCommand(redisClient *c) {
     addReplyLongLong(c,deleted);
 }
 
+/* EXISTS key1 key2 ... key_N.
+ * Return value is the number of keys existing. */
 void existsCommand(redisClient *c) {
-    expireIfNeeded(c->db,c->argv[1]);
-    if (dbExists(c->db,c->argv[1])) {
-        addReply(c, shared.cone);
-    } else {
-        addReply(c, shared.czero);
+    long long count = 0;
+    int j;
+
+    for (j = 1; j < c->argc; j++) {
+        expireIfNeeded(c->db,c->argv[j]);
+        if (dbExists(c->db,c->argv[j])) count++;
     }
+    addReplyLongLong(c,count);
 }
 
 void selectCommand(redisClient *c) {
@@ -688,16 +692,20 @@ void shutdownCommand(redisClient *c) {
 void renameGenericCommand(redisClient *c, int nx) {
     robj *o;
     long long expire;
+    int samekey = 0;
 
-    /* To use the same key as src and dst is probably an error */
-    if (sdscmp(c->argv[1]->ptr,c->argv[2]->ptr) == 0) {
-        addReply(c,shared.sameobjecterr);
-        return;
-    }
+    /* When source and dest key is the same, no operation is performed,
+     * if the key exists, however we still return an error on unexisting key. */
+    if (sdscmp(c->argv[1]->ptr,c->argv[2]->ptr) == 0) samekey = 1;
 
     if ((o = lookupKeyWriteOrReply(c,c->argv[1],shared.nokeyerr)) == NULL)
         return;
 
+    if (samekey) {
+        addReply(c,nx ? shared.czero : shared.ok);
+        return;
+    }
+
     incrRefCount(o);
     expire = getExpire(c->db,c->argv[1]);
     if (lookupKeyWrite(c->db,c->argv[2]) != NULL) {

src/debug.c

@@ -336,10 +336,10 @@ void debugCommand(redisClient *c) {
 
         addReplyStatusFormat(c,
             "Value at:%p refcount:%d "
-            "encoding:%s serializedlength:%lld "
+            "encoding:%s serializedlength:%zu "
             "lru:%d lru_seconds_idle:%llu%s",
             (void*)val, val->refcount,
-            strenc, (long long) rdbSavedObjectLen(val),
+            strenc, rdbSavedObjectLen(val),
             val->lru, estimateObjectIdleTime(val)/1000, extra);
     } else if (!strcasecmp(c->argv[1]->ptr,"sdslen") && c->argc == 3) {
         dictEntry *de;
@@ -425,6 +425,27 @@ void debugCommand(redisClient *c) {
         sizes = sdscatprintf(sizes,"dictentry:%d ", (int)sizeof(dictEntry));
         sizes = sdscatprintf(sizes,"sdshdr:%d", (int)sizeof(struct sdshdr));
         addReplyBulkSds(c,sizes);
+    } else if (!strcasecmp(c->argv[1]->ptr,"htstats") && c->argc == 3) {
+        long dbid;
+        sds stats = sdsempty();
+        char buf[4096];
+
+        if (getLongFromObjectOrReply(c, c->argv[2], &dbid, NULL) != REDIS_OK)
+            return;
+        if (dbid < 0 || dbid >= server.dbnum) {
+            addReplyError(c,"Out of range database");
+            return;
+        }
+
+        stats = sdscatprintf(stats,"[Dictionary HT]\n");
+        dictGetStats(buf,sizeof(buf),server.db[dbid].dict);
+        stats = sdscat(stats,buf);
+
+        stats = sdscatprintf(stats,"[Expires HT]\n");
+        dictGetStats(buf,sizeof(buf),server.db[dbid].expires);
+        stats = sdscat(stats,buf);
+
+        addReplyBulkSds(c,stats);
     } else if (!strcasecmp(c->argv[1]->ptr,"jemalloc") && c->argc == 3) {
 #if defined(USE_JEMALLOC)
         if (!strcasecmp(c->argv[2]->ptr, "info")) {

src/dict.c

@@ -211,6 +211,9 @@ int dictExpand(dict *d, unsigned long size)
     if (dictIsRehashing(d) || d->ht[0].used > size)
         return DICT_ERR;
 
+    /* Rehashing to the same table size is not useful. */
+    if (realsize == d->ht[0].size) return DICT_ERR;
+
     /* Allocate the new hash table and initialize all pointers to NULL */
     n.size = realsize;
     n.sizemask = realsize-1;
@@ -232,27 +235,27 @@ int dictExpand(dict *d, unsigned long size)
 
 /* Performs N steps of incremental rehashing. Returns 1 if there are still
  * keys to move from the old to the new hash table, otherwise 0 is returned.
+ *
  * Note that a rehashing step consists in moving a bucket (that may have more
- * than one key as we use chaining) from the old to the new hash table. */
+ * than one key as we use chaining) from the old to the new hash table, however
+ * since part of the hash table may be composed of empty spaces, it is not
+ * guaranteed that this function will rehash even a single bucket, since it
+ * will visit at max N*10 empty buckets in total, otherwise the amount of
+ * work it does would be unbound and the function may block for a long time. */
 int dictRehash(dict *d, int n) {
+    int empty_visits = n*10; /* Max number of empty buckets to visit. */
     if (!dictIsRehashing(d)) return 0;
 
-    while(n--) {
+    while(n-- && d->ht[0].used != 0) {
         dictEntry *de, *nextde;
 
-        /* Check if we already rehashed the whole table... */
-        if (d->ht[0].used == 0) {
-            zfree(d->ht[0].table);
-            d->ht[0] = d->ht[1];
-            _dictReset(&d->ht[1]);
-            d->rehashidx = -1;
-            return 0;
-        }
-
         /* Note that rehashidx can't overflow as we are sure there are more
          * elements because ht[0].used != 0 */
         assert(d->ht[0].size > (unsigned long)d->rehashidx);
-        while(d->ht[0].table[d->rehashidx] == NULL) d->rehashidx++;
+        while(d->ht[0].table[d->rehashidx] == NULL) {
+            d->rehashidx++;
+            if (--empty_visits == 0) return 1;
+        }
         de = d->ht[0].table[d->rehashidx];
         /* Move all the keys in this bucket from the old to the new hash HT */
         while(de) {
@@ -270,6 +273,17 @@ int dictRehash(dict *d, int n) {
         d->ht[0].table[d->rehashidx] = NULL;
         d->rehashidx++;
     }
+
+    /* Check if we already rehashed the whole table... */
+    if (d->ht[0].used == 0) {
+        zfree(d->ht[0].table);
+        d->ht[0] = d->ht[1];
+        _dictReset(&d->ht[1]);
+        d->rehashidx = -1;
+        return 0;
+    }
+
+    /* More to rehash... */
     return 1;
 }
 
@@ -619,7 +633,11 @@ dictEntry *dictGetRandomKey(dict *d)
     if (dictIsRehashing(d)) _dictRehashStep(d);
     if (dictIsRehashing(d)) {
         do {
-            h = random() % (d->ht[0].size+d->ht[1].size);
+            /* We are sure there are no elements in indexes from 0
+             * to rehashidx-1 */
+            h = d->rehashidx + (random() % (d->ht[0].size +
+                                            d->ht[1].size -
+                                            d->rehashidx));
             he = (h >= d->ht[0].size) ? d->ht[1].table[h - d->ht[0].size] :
                                       d->ht[0].table[h];
         } while(he == NULL);
@@ -646,9 +664,12 @@ dictEntry *dictGetRandomKey(dict *d)
     return he;
 }
 
-/* This is a version of dictGetRandomKey() that is modified in order to
- * return multiple entries by jumping at a random place of the hash table
- * and scanning linearly for entries.
+/* This function samples the dictionary to return a few keys from random
+ * locations.
+ *
+ * It does not guarantee to return all the keys specified in 'count', nor
+ * it does guarantee to return non-duplicated elements, however it will make
+ * some effort to do both things.
  *
  * Returned pointers to hash table entries are stored into 'des' that
  * points to an array of dictEntry pointers. The array must have room for
@@ -657,28 +678,65 @@ dictEntry *dictGetRandomKey(dict *d)
  *
  * The function returns the number of items stored into 'des', that may
  * be less than 'count' if the hash table has less than 'count' elements
- * inside.
+ * inside, or if not enough elements were found in a reasonable amount of
+ * steps.
  *
  * Note that this function is not suitable when you need a good distribution
  * of the returned items, but only when you need to "sample" a given number
  * of continuous elements to run some kind of algorithm or to produce
  * statistics. However the function is much faster than dictGetRandomKey()
- * at producing N elements, and the elements are guaranteed to be non
- * repeating. */
-unsigned int dictGetRandomKeys(dict *d, dictEntry **des, unsigned int count) {
-    int j; /* internal hash table id, 0 or 1. */
-    unsigned int stored = 0;
+ * at producing N elements. */
+unsigned int dictGetSomeKeys(dict *d, dictEntry **des, unsigned int count) {
+    unsigned long j; /* internal hash table id, 0 or 1. */
+    unsigned long tables; /* 1 or 2 tables? */
+    unsigned long stored = 0, maxsizemask;
+    unsigned long maxsteps;
 
     if (dictSize(d) < count) count = dictSize(d);
-    while(stored < count) {
-        for (j = 0; j < 2; j++) {
-            /* Pick a random point inside the hash table 0 or 1. */
-            unsigned int i = random() & d->ht[j].sizemask;
-            int size = d->ht[j].size;
+    maxsteps = count*10;
 
-            /* Make sure to visit every bucket by iterating 'size' times. */
-            while(size--) {
-                dictEntry *he = d->ht[j].table[i];
+    /* Try to do a rehashing work proportional to 'count'. */
+    for (j = 0; j < count; j++) {
+        if (dictIsRehashing(d))
+            _dictRehashStep(d);
+        else
+            break;
+    }
+
+    tables = dictIsRehashing(d) ? 2 : 1;
+    maxsizemask = d->ht[0].sizemask;
+    if (tables > 1 && maxsizemask < d->ht[1].sizemask)
+        maxsizemask = d->ht[1].sizemask;
+
+    /* Pick a random point inside the larger table. */
+    unsigned long i = random() & maxsizemask;
+    unsigned long emptylen = 0; /* Continuous empty entries so far. */
+    while(stored < count && maxsteps--) {
+        for (j = 0; j < tables; j++) {
+            /* Invariant of the dict.c rehashing: up to the indexes already
+             * visited in ht[0] during the rehashing, there are no populated
+             * buckets, so we can skip ht[0] for indexes between 0 and idx-1. */
+            if (tables == 2 && j == 0 && i < (unsigned long) d->rehashidx) {
+                /* Moreover, if we are currently out of range in the second
+                 * table, there will be no elements in both tables up to
+                 * the current rehashing index, so we jump if possible.
+                 * (this happens when going from big to small table). */
+                if (i >= d->ht[1].size) i = d->rehashidx;
+                continue;
+            }
+            if (i >= d->ht[j].size) continue; /* Out of range for this table. */
+            dictEntry *he = d->ht[j].table[i];
+
+            /* Count contiguous empty buckets, and jump to other
+             * locations if they reach 'count' (with a minimum of 5). */
+            if (he == NULL) {
+                emptylen++;
+                if (emptylen >= 5 && emptylen > count) {
+                    i = random() & maxsizemask;
+                    emptylen = 0;
+                }
+            } else {
+                emptylen = 0;
                 while (he) {
                     /* Collect all the elements of the buckets found non
                      * empty while iterating. */
@@ -688,14 +746,11 @@ unsigned int dictGetRandomKeys(dict *d, dictEntry **des, unsigned int count) {
                     stored++;
                     if (stored == count) return stored;
                 }
-                i = (i+1) & d->ht[j].sizemask;
             }
-            /* If there is only one table and we iterated it all, we should
-             * already have 'count' elements. Assert this condition. */
-            assert(dictIsRehashing(d) != 0);
         }
+        i = (i+1) & maxsizemask;
     }
-    return stored; /* Never reached. */
+    return stored;
 }
 
 /* Function to reverse bits. Algorithm from:
@@ -947,24 +1002,21 @@ void dictDisableResize(void) {
     dict_can_resize = 0;
 }
 
-#if 0
-
-/* The following is code that we don't use for Redis currently, but that is part
-of the library. */
-
-/* ----------------------- Debugging ------------------------*/
+/* ------------------------------- Debugging ---------------------------------*/
 
 #define DICT_STATS_VECTLEN 50
-static void _dictPrintStatsHt(dictht *ht) {
+size_t _dictGetStatsHt(char *buf, size_t bufsize, dictht *ht, int tableid) {
     unsigned long i, slots = 0, chainlen, maxchainlen = 0;
     unsigned long totchainlen = 0;
     unsigned long clvector[DICT_STATS_VECTLEN];
+    size_t l = 0;
 
     if (ht->used == 0) {
-        printf("No stats available for empty dictionaries\n");
-        return;
+        return snprintf(buf,bufsize,
+            "No stats available for empty dictionaries\n");
     }
 
+    /* Compute stats. */
     for (i = 0; i < DICT_STATS_VECTLEN; i++) clvector[i] = 0;
     for (i = 0; i < ht->size; i++) {
         dictEntry *he;
@@ -985,89 +1037,46 @@ static void _dictPrintStatsHt(dictht *ht) {
         if (chainlen > maxchainlen) maxchainlen = chainlen;
         totchainlen += chainlen;
     }
-    printf("Hash table stats:\n");
-    printf(" table size: %ld\n", ht->size);
-    printf(" number of elements: %ld\n", ht->used);
-    printf(" different slots: %ld\n", slots);
-    printf(" max chain length: %ld\n", maxchainlen);
-    printf(" avg chain length (counted): %.02f\n", (float)totchainlen/slots);
-    printf(" avg chain length (computed): %.02f\n", (float)ht->used/slots);
-    printf(" Chain length distribution:\n");
+
+    /* Generate human readable stats. */
+    l += snprintf(buf+l,bufsize-l,
+        "Hash table %d stats (%s):\n"
+        " table size: %ld\n"
+        " number of elements: %ld\n"
+        " different slots: %ld\n"
+        " max chain length: %ld\n"
+        " avg chain length (counted): %.02f\n"
+        " avg chain length (computed): %.02f\n"
+        " Chain length distribution:\n",
+        tableid, (tableid == 0) ? "main hash table" : "rehashing target",
+        ht->size, ht->used, slots, maxchainlen,
+        (float)totchainlen/slots, (float)ht->used/slots);
+
     for (i = 0; i < DICT_STATS_VECTLEN-1; i++) {
         if (clvector[i] == 0) continue;
-        printf("   %s%ld: %ld (%.02f%%)\n",(i == DICT_STATS_VECTLEN-1)?">= ":"", i, clvector[i], ((float)clvector[i]/ht->size)*100);
+        if (l >= bufsize) break;
+        l += snprintf(buf+l,bufsize-l,
+            "   %s%ld: %ld (%.02f%%)\n",
+            (i == DICT_STATS_VECTLEN-1)?">= ":"",
+            i, clvector[i], ((float)clvector[i]/ht->size)*100);
     }
+
+    /* Unlike snprintf(), teturn the number of characters actually written. */
+    if (bufsize) buf[bufsize-1] = '\0';
+    return strlen(buf);
 }
 
-void dictPrintStats(dict *d) {
-    _dictPrintStatsHt(&d->ht[0]);
-    if (dictIsRehashing(d)) {
-        printf("-- Rehashing into ht[1]:\n");
-        _dictPrintStatsHt(&d->ht[1]);
+void dictGetStats(char *buf, size_t bufsize, dict *d) {
+    size_t l;
+    char *orig_buf = buf;
+    size_t orig_bufsize = bufsize;
+
+    l = _dictGetStatsHt(buf,bufsize,&d->ht[0],0);
+    buf += l;
+    bufsize -= l;
+    if (dictIsRehashing(d) && bufsize > 0) {
+        _dictGetStatsHt(buf,bufsize,&d->ht[1],1);
     }
+    /* Make sure there is a NULL term at the end. */
+    if (orig_bufsize) orig_buf[orig_bufsize-1] = '\0';
 }
-
-/* ----------------------- StringCopy Hash Table Type ------------------------*/
-
-static unsigned int _dictStringCopyHTHashFunction(const void *key)
-{
-    return dictGenHashFunction(key, strlen(key));
-}
-
-static void *_dictStringDup(void *privdata, const void *key)
-{
-    int len = strlen(key);
-    char *copy = zmalloc(len+1);
-    DICT_NOTUSED(privdata);
-
-    memcpy(copy, key, len);
-    copy[len] = '\0';
-    return copy;
-}
-
-static int _dictStringCopyHTKeyCompare(void *privdata, const void *key1,
-        const void *key2)
-{
-    DICT_NOTUSED(privdata);
-
-    return strcmp(key1, key2) == 0;
-}
-
-static void _dictStringDestructor(void *privdata, void *key)
-{
-    DICT_NOTUSED(privdata);
-
-    zfree(key);
-}
-
-dictType dictTypeHeapStringCopyKey = {
-    _dictStringCopyHTHashFunction, /* hash function */
-    _dictStringDup,                /* key dup */
-    NULL,                          /* val dup */
-    _dictStringCopyHTKeyCompare,   /* key compare */
-    _dictStringDestructor,         /* key destructor */
-    NULL                           /* val destructor */
-};
-
-/* This is like StringCopy but does not auto-duplicate the key.
- * It's used for intepreter's shared strings. */
-dictType dictTypeHeapStrings = {
-    _dictStringCopyHTHashFunction, /* hash function */
-    NULL,                          /* key dup */
-    NULL,                          /* val dup */
-    _dictStringCopyHTKeyCompare,   /* key compare */
-    _dictStringDestructor,         /* key destructor */
-    NULL                           /* val destructor */
-};
-
-/* This is like StringCopy but also automatically handle dynamic
- * allocated C strings as values. */
-dictType dictTypeHeapStringCopyKeyValue = {
-    _dictStringCopyHTHashFunction, /* hash function */
-    _dictStringDup,                /* key dup */
-    _dictStringDup,                /* val dup */
-    _dictStringCopyHTKeyCompare,   /* key compare */
-    _dictStringDestructor,         /* key destructor */
-    _dictStringDestructor,         /* val destructor */
-};
-#endif

src/dict.h

@@ -164,8 +164,8 @@ dictIterator *dictGetSafeIterator(dict *d);
 dictEntry *dictNext(dictIterator *iter);
 void dictReleaseIterator(dictIterator *iter);
 dictEntry *dictGetRandomKey(dict *d);
-unsigned int dictGetRandomKeys(dict *d, dictEntry **des, unsigned int count);
-void dictPrintStats(dict *d);
+unsigned int dictGetSomeKeys(dict *d, dictEntry **des, unsigned int count);
+void dictGetStats(char *buf, size_t bufsize, dict *d);
 unsigned int dictGenHashFunction(const void *key, int len);
 unsigned int dictGenCaseHashFunction(const unsigned char *buf, int len);
 void dictEmpty(dict *d, void(callback)(void*));

src/geo.c

@@ -0,0 +1,706 @@
+/*
+ * Copyright (c) 2014, Matt Stancliff <matt@genges.com>.
+ * Copyright (c) 2015, Salvatore Sanfilippo <antirez@gmail.com>.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ *   * Redistributions of source code must retain the above copyright notice,
+ *     this list of conditions and the following disclaimer.
+ *   * Redistributions in binary form must reproduce the above copyright
+ *     notice, this list of conditions and the following disclaimer in the
+ *     documentation and/or other materials provided with the distribution.
+ *   * Neither the name of Redis nor the names of its contributors may be used
+ *     to endorse or promote products derived from this software without
+ *     specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "geo.h"
+#include "geohash_helper.h"
+
+/* Things exported from t_zset.c only for geo.c, since it is the only other
+ * part of Redis that requires close zset introspection. */
+unsigned char *zzlFirstInRange(unsigned char *zl, zrangespec *range);
+int zslValueLteMax(double value, zrangespec *spec);
+
+/* ====================================================================
+ * This file implements the following commands:
+ *
+ *   - geoadd - add coordinates for value to geoset
+ *   - georadius - search radius by coordinates in geoset
+ *   - georadiusbymember - search radius based on geoset member position
+ * ==================================================================== */
+
+/* ====================================================================
+ * geoArray implementation
+ * ==================================================================== */
+
+/* Create a new array of geoPoints. */
+geoArray *geoArrayCreate(void) {
+    geoArray *ga = zmalloc(sizeof(*ga));
+    /* It gets allocated on first geoArrayAppend() call. */
+    ga->array = NULL;
+    ga->buckets = 0;
+    ga->used = 0;
+    return ga;
+}
+
+/* Add a new entry and return its pointer so that the caller can populate
+ * it with data. */
+geoPoint *geoArrayAppend(geoArray *ga) {
+    if (ga->used == ga->buckets) {
+        ga->buckets = (ga->buckets == 0) ? 8 : ga->buckets*2;
+        ga->array = zrealloc(ga->array,sizeof(geoPoint)*ga->buckets);
+    }
+    geoPoint *gp = ga->array+ga->used;
+    ga->used++;
+    return gp;
+}
+
+/* Destroy a geoArray created with geoArrayCreate(). */
+void geoArrayFree(geoArray *ga) {
+    size_t i;
+    for (i = 0; i < ga->used; i++) sdsfree(ga->array[i].member);
+    zfree(ga->array);
+    zfree(ga);
+}
+
+/* ====================================================================
+ * Helpers
+ * ==================================================================== */
+int decodeGeohash(double bits, double *xy) {
+    GeoHashBits hash = { .bits = (uint64_t)bits, .step = GEO_STEP_MAX };
+    return geohashDecodeToLongLatWGS84(hash, xy);
+}
+
+/* Input Argument Helper */
+/* Take a pointer to the latitude arg then use the next arg for longitude.
+ * On parse error REDIS_ERR is returned, otherwise REDIS_OK. */
+int extractLongLatOrReply(redisClient *c, robj **argv,
+                                        double *xy) {
+    for (int i = 0; i < 2; i++) {
+        if (getDoubleFromObjectOrReply(c, argv[i], xy + i, NULL) !=
+            REDIS_OK) {
+            return REDIS_ERR;
+        }
+        if (xy[0] < GEO_LONG_MIN || xy[0] > GEO_LONG_MAX ||
+            xy[1] < GEO_LAT_MIN  || xy[1] > GEO_LAT_MAX) {
+            addReplySds(c, sdscatprintf(sdsempty(),
+                "-ERR invalid longitude,latitude pair %f,%f\r\n",xy[0],xy[1]));
+            return REDIS_ERR;
+        }
+    }
+    return REDIS_OK;
+}
+
+/* Input Argument Helper */
+/* Decode lat/long from a zset member's score.
+ * Returns REDIS_OK on successful decoding, otherwise REDIS_ERR is returned. */
+int longLatFromMember(robj *zobj, robj *member, double *xy) {
+    double score = 0;
+
+    if (zsetScore(zobj, member, &score) == REDIS_ERR) return REDIS_ERR;
+    if (!decodeGeohash(score, xy)) return REDIS_ERR;
+    return REDIS_OK;
+}
+
+/* Check that the unit argument matches one of the known units, and returns
+ * the conversion factor to meters (you need to divide meters by the conversion
+ * factor to convert to the right unit).
+ *
+ * If the unit is not valid, an error is reported to the client, and a value
+ * less than zero is returned. */
+double extractUnitOrReply(redisClient *c, robj *unit) {
+    char *u = unit->ptr;
+
+    if (!strcmp(u, "m")) {
+        return 1;
+    } else if (!strcmp(u, "km")) {
+        return 1000;
+    } else if (!strcmp(u, "ft")) {
+        return 0.3048;
+    } else if (!strcmp(u, "mi")) {
+        return 1609.34;
+    } else {
+        addReplyError(c,
+            "unsupported unit provided. please use m, km, ft, mi");
+        return -1;
+    }
+}
+
+/* Input Argument Helper.
+ * Extract the dinstance from the specified two arguments starting at 'argv'
+ * that shouldbe in the form: <number> <unit> and return the dinstance in the
+ * specified unit on success. *conversino is populated with the coefficient
+ * to use in order to convert meters to the unit.
+ *
+ * On error a value less than zero is returned. */
+double extractDistanceOrReply(redisClient *c, robj **argv,
+                                     double *conversion) {
+    double distance;
+    if (getDoubleFromObjectOrReply(c, argv[0], &distance,
+                                   "need numeric radius") != REDIS_OK) {
+        return -1;
+    }
+
+    double to_meters = extractUnitOrReply(c,argv[1]);
+    if (to_meters < 0) return -1;
+
+    if (conversion) *conversion = to_meters;
+    return distance * to_meters;
+}
+
+/* The defailt addReplyDouble has too much accuracy.  We use this
+ * for returning location distances. "5.2145 meters away" is nicer
+ * than "5.2144992818115 meters away." We provide 4 digits after the dot
+ * so that the returned value is decently accurate even when the unit is
+ * the kilometer. */
+void addReplyDoubleDistance(redisClient *c, double d) {
+    char dbuf[128];
+    int dlen = snprintf(dbuf, sizeof(dbuf), "%.4f", d);
+    addReplyBulkCBuffer(c, dbuf, dlen);
+}
+
+/* Helper function for geoGetPointsInRange(): given a sorted set score
+ * representing a point, and another point (the center of our search) and
+ * a radius, appends this entry as a geoPoint into the specified geoArray
+ * only if the point is within the search area.
+ *
+ * returns REDIS_OK if the point is included, or REIDS_ERR if it is outside. */
+int geoAppendIfWithinRadius(geoArray *ga, double lon, double lat, double radius, double score, sds member) {
+    double distance, xy[2];
+
+    if (!decodeGeohash(score,xy)) return REDIS_ERR; /* Can't decode. */
+    /* Note that geohashGetDistanceIfInRadiusWGS84() takes arguments in
+     * reverse order: longitude first, latitude later. */
+    if (!geohashGetDistanceIfInRadiusWGS84(lon,lat, xy[0], xy[1],
+                                           radius, &distance))
+    {
+        return REDIS_ERR;
+    }
+
+    /* Append the new element. */
+    geoPoint *gp = geoArrayAppend(ga);
+    gp->longitude = xy[0];
+    gp->latitude = xy[1];
+    gp->dist = distance;
+    gp->member = member;
+    gp->score = score;
+    return REDIS_OK;
+}
+
+/* Query a Redis sorted set to extract all the elements between 'min' and
+ * 'max', appending them into the array of geoPoint structures 'gparray'.
+ * The command returns the number of elements added to the array.
+ *
+ * Elements which are farest than 'radius' from the specified 'x' and 'y'
+ * coordinates are not included.
+ *
+ * The ability of this function to append to an existing set of points is
+ * important for good performances because querying by radius is performed
+ * using multiple queries to the sorted set, that we later need to sort
+ * via qsort. Similarly we need to be able to reject points outside the search
+ * radius area ASAP in order to allocate and process more points than needed. */
+int geoGetPointsInRange(robj *zobj, double min, double max, double lon, double lat, double radius, geoArray *ga) {
+    /* minex 0 = include min in range; maxex 1 = exclude max in range */
+    /* That's: min <= val < max */
+    zrangespec range = { .min = min, .max = max, .minex = 0, .maxex = 1 };
+    size_t origincount = ga->used;
+    sds member;
+
+    if (zobj->encoding == REDIS_ENCODING_ZIPLIST) {
+        unsigned char *zl = zobj->ptr;
+        unsigned char *eptr, *sptr;
+        unsigned char *vstr = NULL;
+        unsigned int vlen = 0;
+        long long vlong = 0;
+        double score = 0;
+
+        if ((eptr = zzlFirstInRange(zl, &range)) == NULL) {
+            /* Nothing exists starting at our min.  No results. */
+            return 0;
+        }
+
+        sptr = ziplistNext(zl, eptr);
+        while (eptr) {
+            score = zzlGetScore(sptr);
+
+            /* If we fell out of range, break. */
+            if (!zslValueLteMax(score, &range))
+                break;
+
+            /* We know the element exists. ziplistGet should always succeed */
+            ziplistGet(eptr, &vstr, &vlen, &vlong);
+            member = (vstr == NULL) ? sdsfromlonglong(vlong) :
+                                      sdsnewlen(vstr,vlen);
+            if (geoAppendIfWithinRadius(ga,lon,lat,radius,score,member)
+                == REDIS_ERR) sdsfree(member);
+            zzlNext(zl, &eptr, &sptr);
+        }
+    } else if (zobj->encoding == REDIS_ENCODING_SKIPLIST) {
+        zset *zs = zobj->ptr;
+        zskiplist *zsl = zs->zsl;
+        zskiplistNode *ln;
+
+        if ((ln = zslFirstInRange(zsl, &range)) == NULL) {
+            /* Nothing exists starting at our min.  No results. */
+            return 0;
+        }
+
+        while (ln) {
+            robj *o = ln->obj;
+            /* Abort when the node is no longer in range. */
+            if (!zslValueLteMax(ln->score, &range))
+                break;
+
+            member = (o->encoding == REDIS_ENCODING_INT) ?
+                        sdsfromlonglong((long)o->ptr) :
+                        sdsdup(o->ptr);
+            if (geoAppendIfWithinRadius(ga,lon,lat,radius,ln->score,member)
+                == REDIS_ERR) sdsfree(member);
+            ln = ln->level[0].forward;
+        }
+    }
+    return ga->used - origincount;
+}
+
+/* Compute the sorted set scores min (inclusive), max (exclusive) we should
+ * query in order to retrieve all the elements inside the specified area
+ * 'hash'. The two scores are returned by reference in *min and *max. */
+void scoresOfGeoHashBox(GeoHashBits hash, GeoHashFix52Bits *min, GeoHashFix52Bits *max) {
+    /* We want to compute the sorted set scores that will include all the
+     * elements inside the specified Geohash 'hash', which has as many
+     * bits as specified by hash.step * 2.
+     *
+     * So if step is, for example, 3, and the hash value in binary
+     * is 101010, since our score is 52 bits we want every element which
+     * is in binary: 101010?????????????????????????????????????????????
+     * Where ? can be 0 or 1.
+     *
+     * To get the min score we just use the initial hash value left
+     * shifted enough to get the 52 bit value. Later we increment the
+     * 6 bit prefis (see the hash.bits++ statement), and get the new
+     * prefix: 101011, which we align again to 52 bits to get the maximum
+     * value (which is excluded from the search). So we get everything
+     * between the two following scores (represented in binary):
+     *
+     * 1010100000000000000000000000000000000000000000000000 (included)
+     * and
+     * 1010110000000000000000000000000000000000000000000000 (excluded).
+     */
+    *min = geohashAlign52Bits(hash);
+    hash.bits++;
+    *max = geohashAlign52Bits(hash);
+}
+
+/* Obtain all members between the min/max of this geohash bounding box.
+ * Populate a geoArray of GeoPoints by calling geoGetPointsInRange().
+ * Return the number of points added to the array. */
+int membersOfGeoHashBox(robj *zobj, GeoHashBits hash, geoArray *ga, double lon, double lat, double radius) {
+    GeoHashFix52Bits min, max;
+
+    scoresOfGeoHashBox(hash,&min,&max);
+    return geoGetPointsInRange(zobj, min, max, lon, lat, radius, ga);
+}
+
+/* Search all eight neighbors + self geohash box */
+int membersOfAllNeighbors(robj *zobj, GeoHashRadius n, double lon, double lat, double radius, geoArray *ga) {
+    GeoHashBits neighbors[9];
+    unsigned int i, count = 0;
+
+    neighbors[0] = n.hash;
+    neighbors[1] = n.neighbors.north;
+    neighbors[2] = n.neighbors.south;
+    neighbors[3] = n.neighbors.east;
+    neighbors[4] = n.neighbors.west;
+    neighbors[5] = n.neighbors.north_east;
+    neighbors[6] = n.neighbors.north_west;
+    neighbors[7] = n.neighbors.south_east;
+    neighbors[8] = n.neighbors.south_west;
+
+    /* For each neighbor (*and* our own hashbox), get all the matching
+     * members and add them to the potential result list. */
+    for (i = 0; i < sizeof(neighbors) / sizeof(*neighbors); i++) {
+        if (HASHISZERO(neighbors[i]))
+            continue;
+        count += membersOfGeoHashBox(zobj, neighbors[i], ga, lon, lat, radius);
+    }
+    return count;
+}
+
+/* Sort comparators for qsort() */
+static int sort_gp_asc(const void *a, const void *b) {
+    const struct geoPoint *gpa = a, *gpb = b;
+    /* We can't do adist - bdist because they are doubles and
+     * the comparator returns an int. */
+    if (gpa->dist > gpb->dist)
+        return 1;
+    else if (gpa->dist == gpb->dist)
+        return 0;
+    else
+        return -1;
+}
+
+static int sort_gp_desc(const void *a, const void *b) {
+    return -sort_gp_asc(a, b);
+}
+
+/* ====================================================================
+ * Commands
+ * ==================================================================== */
+
+/* GEOADD key long lat name [long2 lat2 name2 ... longN latN nameN] */
+void geoaddCommand(redisClient *c) {
+    /* Check arguments number for sanity. */
+    if ((c->argc - 2) % 3 != 0) {
+        /* Need an odd number of arguments if we got this far... */
+        addReplyError(c, "syntax error. Try GEOADD key [x1] [y1] [name1] "
+                         "[x2] [y2] [name2] ... ");
+        return;
+    }
+
+    int elements = (c->argc - 2) / 3;
+    int argc = 2+elements*2; /* ZADD key score ele ... */
+    robj **argv = zcalloc(argc*sizeof(robj*));
+    argv[0] = createRawStringObject("zadd",4);
+    argv[1] = c->argv[1]; /* key */
+    incrRefCount(argv[1]);
+
+    /* Create the argument vector to call ZADD in order to add all
+     * the score,value pairs to the requested zset, where score is actually
+     * an encoded version of lat,long. */
+    int i;
+    for (i = 0; i < elements; i++) {
+        double xy[2];
+
+        if (extractLongLatOrReply(c, (c->argv+2)+(i*3),xy) == REDIS_ERR) {
+            for (i = 0; i < argc; i++)
+                if (argv[i]) decrRefCount(argv[i]);
+            zfree(argv);
+            return;
+        }
+
+        /* Turn the coordinates into the score of the element. */
+        GeoHashBits hash;
+        geohashEncodeWGS84(xy[0], xy[1], GEO_STEP_MAX, &hash);
+        GeoHashFix52Bits bits = geohashAlign52Bits(hash);
+        robj *score = createObject(REDIS_STRING, sdsfromlonglong(bits));
+        robj *val = c->argv[2 + i * 3 + 2];
+        argv[2+i*2] = score;
+        argv[3+i*2] = val;
+        incrRefCount(val);
+    }
+
+    /* Finally call ZADD that will do the work for us. */
+    replaceClientCommandVector(c,argc,argv);
+    zaddCommand(c);
+}
+
+#define SORT_NONE 0
+#define SORT_ASC 1
+#define SORT_DESC 2
+
+#define RADIUS_COORDS 1
+#define RADIUS_MEMBER 2
+
+/* GEORADIUS key x y radius unit [WITHDIST] [WITHHASH] [WITHCOORD] [ASC|DESC]
+ *                               [COUNT count]
+ * GEORADIUSBYMEMBER key member radius unit ... options ... */
+void georadiusGeneric(redisClient *c, int type) {
+    robj *key = c->argv[1];
+
+    /* Look up the requested zset */
+    robj *zobj = NULL;
+    if ((zobj = lookupKeyReadOrReply(c, key, shared.emptymultibulk)) == NULL ||
+        checkType(c, zobj, REDIS_ZSET)) {
+        return;
+    }
+
+    /* Find long/lat to use for radius search based on inquiry type */
+    int base_args;
+    double xy[2] = { 0 };
+    if (type == RADIUS_COORDS) {
+        base_args = 6;
+        if (extractLongLatOrReply(c, c->argv + 2, xy) == REDIS_ERR)
+            return;
+    } else if (type == RADIUS_MEMBER) {
+        base_args = 5;
+        robj *member = c->argv[2];
+        if (longLatFromMember(zobj, member, xy) == REDIS_ERR) {
+            addReplyError(c, "could not decode requested zset member");
+            return;
+        }
+    } else {
+        addReplyError(c, "unknown georadius search type");
+        return;
+    }
+
+    /* Extract radius and units from arguments */
+    double radius_meters = 0, conversion = 1;
+    if ((radius_meters = extractDistanceOrReply(c, c->argv + base_args - 2,
+                                                &conversion)) < 0) {
+        return;
+    }
+
+    /* Discover and populate all optional parameters. */
+    int withdist = 0, withhash = 0, withcoords = 0;
+    int sort = SORT_NONE;
+    long long count = 0;
+    if (c->argc > base_args) {
+        int remaining = c->argc - base_args;
+        for (int i = 0; i < remaining; i++) {
+            char *arg = c->argv[base_args + i]->ptr;
+            if (!strcasecmp(arg, "withdist")) {
+                withdist = 1;
+            } else if (!strcasecmp(arg, "withhash")) {
+                withhash = 1;
+            } else if (!strcasecmp(arg, "withcoord")) {
+                withcoords = 1;
+            } else if (!strcasecmp(arg, "asc")) {
+                sort = SORT_ASC;
+            } else if (!strcasecmp(arg, "desc")) {
+                sort = SORT_DESC;
+            } else if (!strcasecmp(arg, "count") && remaining > 0) {
+                if (getLongLongFromObjectOrReply(c, c->argv[base_args+i+1],
+                    &count, NULL) != REDIS_OK) return;
+                if (count <= 0) {
+                    addReplyError(c,"COUNT must be > 0");
+                    return;
+                }
+                i++;
+            } else {
+                addReply(c, shared.syntaxerr);
+                return;
+            }
+        }
+    }
+
+    /* COUNT without ordering does not make much sense, force ASC
+     * ordering if COUNT was specified but no sorting was requested. */
+    if (count != 0 && sort == SORT_NONE) sort = SORT_ASC;
+
+    /* Get all neighbor geohash boxes for our radius search */
+    GeoHashRadius georadius =
+        geohashGetAreasByRadiusWGS84(xy[0], xy[1], radius_meters);
+
+    /* Search the zset for all matching points */
+    geoArray *ga = geoArrayCreate();
+    membersOfAllNeighbors(zobj, georadius, xy[0], xy[1], radius_meters, ga);
+
+    /* If no matching results, the user gets an empty reply. */
+    if (ga->used == 0) {
+        addReply(c, shared.emptymultibulk);
+        geoArrayFree(ga);
+        return;
+    }
+
+    long result_length = ga->used;
+    long option_length = 0;
+
+    /* Our options are self-contained nested multibulk replies, so we
+     * only need to track how many of those nested replies we return. */
+    if (withdist)
+        option_length++;
+
+    if (withcoords)
+        option_length++;
+
+    if (withhash)
+        option_length++;
+
+    /* The multibulk len we send is exactly result_length. The result is either
+     * all strings of just zset members  *or* a nested multi-bulk reply
+     * containing the zset member string _and_ all the additional options the
+     * user enabled for this request. */
+    addReplyMultiBulkLen(c, (count == 0 || result_length < count) ?
+                            result_length : count);
+
+    /* Process [optional] requested sorting */
+    if (sort == SORT_ASC) {
+        qsort(ga->array, result_length, sizeof(geoPoint), sort_gp_asc);
+    } else if (sort == SORT_DESC) {
+        qsort(ga->array, result_length, sizeof(geoPoint), sort_gp_desc);
+    }
+
+    /* Finally send results back to the caller */
+    int i;
+    for (i = 0; i < result_length; i++) {
+        geoPoint *gp = ga->array+i;
+        gp->dist /= conversion; /* Fix according to unit. */
+
+        /* If we have options in option_length, return each sub-result
+         * as a nested multi-bulk.  Add 1 to account for result value itself. */
+        if (option_length)
+            addReplyMultiBulkLen(c, option_length + 1);
+
+        addReplyBulkSds(c,gp->member);
+        gp->member = NULL;
+
+        if (withdist)
+            addReplyDoubleDistance(c, gp->dist);
+
+        if (withhash)
+            addReplyLongLong(c, gp->score);
+
+        if (withcoords) {
+            addReplyMultiBulkLen(c, 2);
+            addReplyDouble(c, gp->longitude);
+            addReplyDouble(c, gp->latitude);
+        }
+
+        /* Stop if COUNT was specified and we already provided the
+         * specified number of elements. */
+        if (count != 0 && count == i+1) break;
+    }
+    geoArrayFree(ga);
+}
+
+/* GEORADIUS wrapper function. */
+void georadiusCommand(redisClient *c) {
+    georadiusGeneric(c, RADIUS_COORDS);
+}
+
+/* GEORADIUSBYMEMBER wrapper function. */
+void georadiusByMemberCommand(redisClient *c) {
+    georadiusGeneric(c, RADIUS_MEMBER);
+}
+
+/* GEOHASH key ele1 ele2 ... eleN
+ *
+ * Returns an array with an 11 characters geohash representation of the
+ * position of the specified elements. */
+void geohashCommand(redisClient *c) {
+    char *geoalphabet= "0123456789bcdefghjkmnpqrstuvwxyz";
+    int j;
+
+    /* Look up the requested zset */
+    robj *zobj = NULL;
+    if ((zobj = lookupKeyReadOrReply(c, c->argv[1], shared.emptymultibulk))
+        == NULL || checkType(c, zobj, REDIS_ZSET)) return;
+
+    /* Geohash elements one after the other, using a null bulk reply for
+     * missing elements. */
+    addReplyMultiBulkLen(c,c->argc-2);
+    for (j = 2; j < c->argc; j++) {
+        double score;
+        if (zsetScore(zobj, c->argv[j], &score) == REDIS_ERR) {
+            addReply(c,shared.nullbulk);
+        } else {
+            /* The internal format we use for geocoding is a bit different
+             * than the standard, since we use as initial latitude range
+             * -85,85, while the normal geohashing algorithm uses -90,90.
+             * So we have to decode our position and re-encode using the
+             * standard ranges in order to output a valid geohash string. */
+
+            /* Decode... */
+            double xy[2];
+            if (!decodeGeohash(score,xy)) {
+                addReply(c,shared.nullbulk);
+                continue;
+            }
+
+            /* Re-encode */
+            GeoHashRange r[2];
+            GeoHashBits hash;
+            r[0].min = -180;
+            r[0].max = 180;
+            r[1].min = -90;
+            r[1].max = 90;
+            geohashEncode(&r[0],&r[1],xy[0],xy[1],26,&hash);
+
+            char buf[12];
+            int i;
+            for (i = 0; i < 11; i++) {
+                int idx = (hash.bits >> (52-((i+1)*5))) & 0x1f;
+                buf[i] = geoalphabet[idx];
+            }
+            buf[11] = '\0';
+            addReplyBulkCBuffer(c,buf,11);
+        }
+    }
+}
+
+/* GEOPOS key ele1 ele2 ... eleN
+ *
+ * Returns an array of two-items arrays representing the x,y position of each
+ * element specified in the arguments. For missing elements NULL is returned. */
+void geoposCommand(redisClient *c) {
+    int j;
+
+    /* Look up the requested zset */
+    robj *zobj = NULL;
+    if ((zobj = lookupKeyReadOrReply(c, c->argv[1], shared.emptymultibulk))
+        == NULL || checkType(c, zobj, REDIS_ZSET)) return;
+
+    /* Report elements one after the other, using a null bulk reply for
+     * missing elements. */
+    addReplyMultiBulkLen(c,c->argc-2);
+    for (j = 2; j < c->argc; j++) {
+        double score;
+        if (zsetScore(zobj, c->argv[j], &score) == REDIS_ERR) {
+            addReply(c,shared.nullmultibulk);
+        } else {
+            /* Decode... */
+            double xy[2];
+            if (!decodeGeohash(score,xy)) {
+                addReply(c,shared.nullmultibulk);
+                continue;
+            }
+            addReplyMultiBulkLen(c,2);
+            addReplyDouble(c,xy[0]);
+            addReplyDouble(c,xy[1]);
+        }
+    }
+}
+
+/* GEODIST key ele1 ele2 [unit]
+ *
+ * Return the distance, in meters by default, otherwise accordig to "unit",
+ * between points ele1 and ele2. If one or more elements are missing NULL
+ * is returned. */
+void geodistCommand(redisClient *c) {
+    double to_meter = 1;
+
+    /* Check if there is the unit to extract, otherwise assume meters. */
+    if (c->argc == 5) {
+        to_meter = extractUnitOrReply(c,c->argv[4]);
+        if (to_meter < 0) return;
+    } else if (c->argc > 5) {
+        addReply(c,shared.syntaxerr);
+        return;
+    }
+
+    /* Look up the requested zset */
+    robj *zobj = NULL;
+    if ((zobj = lookupKeyReadOrReply(c, c->argv[1], shared.emptybulk))
+        == NULL || checkType(c, zobj, REDIS_ZSET)) return;
+
+    /* Get the scores. We need both otherwise NULL is returned. */
+    double score1, score2, xyxy[4];
+    if (zsetScore(zobj, c->argv[2], &score1) == REDIS_ERR ||
+        zsetScore(zobj, c->argv[3], &score2) == REDIS_ERR)
+    {
+        addReply(c,shared.nullbulk);
+        return;
+    }
+
+    /* Decode & compute the distance. */
+    if (!decodeGeohash(score1,xyxy) || !decodeGeohash(score2,xyxy+2))
+        addReply(c,shared.nullbulk);
+    else
+        addReplyDouble(c,
+            geohashGetDistance(xyxy[0],xyxy[1],xyxy[2],xyxy[3]) / to_meter);
+}

src/geo.h

@@ -0,0 +1,22 @@
+#ifndef __GEO_H__
+#define __GEO_H__
+
+#include "redis.h"
+
+/* Structures used inside geo.c in order to represent points and array of
+ * points on the earth. */
+typedef struct geoPoint {
+    double longitude;
+    double latitude;
+    double dist;
+    double score;
+    char *member;
+} geoPoint;
+
+typedef struct geoArray {
+    struct geoPoint *array;
+    size_t buckets;
+    size_t used;
+} geoArray;
+
+#endif

src/intset.c

@@ -261,90 +261,6 @@ int64_t intsetRandom(intset *is) {
     return _intsetGet(is,rand()%intrev32ifbe(is->length));
 }
 
-/* How many times bigger should the set length be compared to the requested
- * count of members for us to use the Floyd algorithm instead of
- * the Knuth algorithm */
-#define RANDOMMEMBERS_ALGORITHM_SELECTION_RATIO (2)
-
-/* Copies 'count' random members from the set into the 'values' array.
- * 'values' must be an array of int64_t values, of length 'count'.
- * Returns the amount of items returned. If this amount is less than 'count',
- * then the remaining 'values' are left uninitialized. */
-int intsetRandomMembers(intset *is, int64_t* values, int count) {
-
-    /* We don't check that is and values are non-NULL - the caller must
-     * play nice. */
-
-    int length = intsetLen(is);
-
-    if (count > length) {
-        /* Return everything in the set */
-        count = length;
-    }
-
-    /* Choose between the Knuth shuffle algorithm, O(1) space, O(length) time,
-     * and the Floyd algorithm, O(length) space, O(count) time. */
-    if ((RANDOMMEMBERS_ALGORITHM_SELECTION_RATIO * count) > length) {
-
-        /* If the count of members requested is almost the length of the set,
-         * use the Knuth shuffle algorithm, O(1) space, O(length) time. */
-
-        /* First, fill the values array with unique random indexes inside
-         * the set. */
-        int in, im, rn, rm;
-        im = 0;
-        for (in = 0; in < length && im < count; in++) {
-
-            rn = length - in;
-            rm = count - im;
-            if (rand() % rn < rm) {
-                values[im++] = in;
-            }
-        }
-
-    } else {
-
-        /* If the length is considerably more than the count of members
-         * requested, use Robert Floyd's algorithm, O(length) space,
-         * O(count) time.
-         * Based on Jon Bentley's Programming Pearls */
-
-        int64_t *is_used = zcalloc(sizeof(int64_t) * length);
-        int in, im, r;
-
-        r = 0;
-        im = 0;
-
-        for (in = length - count; in < length && im < count; in++) {
-
-            /* Generate a random number r */
-            r = rand() % (in + 1);
-
-            /* Do we already have the value in r? */
-            if (is_used[r]) {
-                /* Use in instead of the generated number */
-                r = in;
-            }
-
-            values[im++] = r ;
-
-            /* Mark it as used */
-            is_used[r] = 1;
-        }
-
-        zfree(is_used);
-    }
-
-    /* Replace each random index with the value stored there in the intset */
-    uint8_t encoding = intrev32ifbe(is->encoding);
-    for (int currentValue = 0; currentValue < count; currentValue++) {
-        values[currentValue] =
-            _intsetGetEncoded(is, values[currentValue], encoding);
-    }
-
-    return count;
-}
-
 /* Sets the value to the value at the given position. When this position is
  * out of range the function returns 0, when in range it returns 1. */
 uint8_t intsetGet(intset *is, uint32_t pos, int64_t *value) {

src/intset.h

@@ -43,7 +43,6 @@ intset *intsetAdd(intset *is, int64_t value, uint8_t *success);
 intset *intsetRemove(intset *is, int64_t value, int *success);
 uint8_t intsetFind(intset *is, int64_t value);
 int64_t intsetRandom(intset *is);
-int intsetRandomMembers(intset *is, int64_t* value, int count);
 uint8_t intsetGet(intset *is, uint32_t pos, int64_t *value);
 uint32_t intsetLen(intset *is);
 size_t intsetBlobLen(intset *is);

src/latency.c

@@ -228,6 +228,7 @@ sds createLatencyReport(void) {
     int advise_write_load_info = 0; /* Print info about AOF and write load. */
     int advise_hz = 0;              /* Use higher HZ. */
     int advise_large_objects = 0;   /* Deletion of large objects. */
+    int advise_mass_eviction = 0;   /* Avoid mass eviction of keys. */
     int advise_relax_fsync_policy = 0; /* appendfsync always is slow. */
     int advise_disable_thp = 0;     /* AnonHugePages detected. */
     int advices = 0;
@@ -247,7 +248,7 @@ sds createLatencyReport(void) {
     dictEntry *de;
     int eventnum = 0;
 
-    di = dictGetIterator(server.latency_events);
+    di = dictGetSafeIterator(server.latency_events);
     while((de = dictNext(di)) != NULL) {
         char *event = dictGetKey(de);
         struct latencyTimeSeries *ts = dictGetVal(de);
@@ -364,11 +365,16 @@ sds createLatencyReport(void) {
         }
 
         /* Eviction cycle. */
-        if (!strcasecmp(event,"eviction-cycle")) {
+        if (!strcasecmp(event,"eviction-del")) {
             advise_large_objects = 1;
             advices++;
         }
 
+        if (!strcasecmp(event,"eviction-cycle")) {
+            advise_mass_eviction = 1;
+            advices++;
+        }
+
         report = sdscatlen(report,"\n",1);
     }
     dictReleaseIterator(di);
@@ -452,6 +458,10 @@ sds createLatencyReport(void) {
             report = sdscat(report,"- Deleting, expiring or evicting (because of maxmemory policy) large objects is a blocking operation. If you have very large objects that are often deleted, expired, or evicted, try to fragment those objects into multiple smaller objects.\n");
         }
 
+        if (advise_mass_eviction) {
+            report = sdscat(report,"- Sudden changes to the 'maxmemory' setting via 'CONFIG SET', or allocation of large objects via sets or sorted sets intersections, STORE option of SORT, Redis Cluster large keys migrations (RESTORE command), may create sudden memory pressure forcing the server to block trying to evict keys. \n");
+        }
+
         if (advise_disable_thp) {
             report = sdscat(report,"- I detected a non zero amount of anonymous huge pages used by your process. This creates very serious latency events in different conditions, especially when Redis is persisting on disk. To disable THP support use the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled', make sure to also add it into /etc/rc.local so that the command will be executed again after a reboot. Note that even if you have already disabled THP, you still need to restart the Redis process to get rid of the huge pages already created.\n");
         }

src/latency.h

@@ -86,4 +86,8 @@ int THPIsEnabled(void);
         (var) >= server.latency_monitor_threshold) \
           latencyAddSample((event),(var));
 
+/* Remove time from a nested event. */
+#define latencyRemoveNestedEvent(event_var,nested_var) \
+    event_var += nested_var;
+
 #endif /* __LATENCY_H */

src/networking.c

@@ -121,6 +121,7 @@ redisClient *createClient(int fd) {
     c->pubsub_channels = dictCreate(&setDictType,NULL);
     c->pubsub_patterns = listCreate();
     c->peerid = NULL;
+    c->repl_eof_supported = 0;
     listSetFreeMethod(c->pubsub_patterns,decrRefCountVoid);
     listSetMatchMethod(c->pubsub_patterns,listMatchObjects);
     if (fd != -1) listAddNodeTail(server.clients,c);
@@ -135,23 +136,49 @@ redisClient *createClient(int fd) {
  * returns REDIS_OK, and make sure to install the write handler in our event
  * loop so that when the socket is writable new data gets written.
  *
- * If the client should not receive new data, because it is a fake client,
- * a master, a slave not yet online, or because the setup of the write handler
- * failed, the function returns REDIS_ERR.
+ * If the client should not receive new data, because it is a fake client
+ * (used to load AOF in memory), a master or because the setup of the write
+ * handler failed, the function returns REDIS_ERR.
+ *
+ * The function may return REDIS_OK without actually installing the write
+ * event handler in the following cases:
+ *
+ * 1) The event handler should already be installed since the output buffer
+ *    already contained something.
+ * 2) The client is a slave but not yet online, so we want to just accumulate
+ *    writes in the buffer but not actually sending them yet.
  *
  * Typically gets called every time a reply is built, before adding more
  * data to the clients output buffers. If the function returns REDIS_ERR no
  * data should be appended to the output buffers. */
 int prepareClientToWrite(redisClient *c) {
+    /* If it's the Lua client we always return ok without installing any
+     * handler since there is no socket at all. */
     if (c->flags & REDIS_LUA_CLIENT) return REDIS_OK;
+
+    /* Masters don't receive replies, unless REDIS_MASTER_FORCE_REPLY flag
+     * is set. */
     if ((c->flags & REDIS_MASTER) &&
         !(c->flags & REDIS_MASTER_FORCE_REPLY)) return REDIS_ERR;
-    if (c->fd <= 0) return REDIS_ERR; /* Fake client */
+
+    if (c->fd <= 0) return REDIS_ERR; /* Fake client for AOF loading. */
+
+    /* Only install the handler if not already installed and, in case of
+     * slaves, if the client can actually receive writes. */
     if (c->bufpos == 0 && listLength(c->reply) == 0 &&
         (c->replstate == REDIS_REPL_NONE ||
-         c->replstate == REDIS_REPL_ONLINE) &&
-        aeCreateFileEvent(server.el, c->fd, AE_WRITABLE,
-        sendReplyToClient, c) == AE_ERR) return REDIS_ERR;
+         (c->replstate == REDIS_REPL_ONLINE && !c->repl_put_online_on_ack)))
+    {
+        /* Try to install the write handler. */
+        if (aeCreateFileEvent(server.el, c->fd, AE_WRITABLE,
+                sendReplyToClient, c) == AE_ERR)
+        {
+            freeClientAsync(c);
+            return REDIS_ERR;
+        }
+    }
+
+    /* Authorize the caller to queue in the output buffer of this client. */
     return REDIS_OK;
 }
 
@@ -175,7 +202,7 @@ robj *dupLastObjectIfNeeded(list *reply) {
  * Low level functions to add more data to output buffers.
  * -------------------------------------------------------------------------- */
 
-int _addReplyToBuffer(redisClient *c, char *s, size_t len) {
+int _addReplyToBuffer(redisClient *c, const char *s, size_t len) {
     size_t available = sizeof(c->buf)-c->bufpos;
 
     if (c->flags & REDIS_CLOSE_AFTER_REPLY) return REDIS_OK;
@@ -255,7 +282,7 @@ void _addReplySdsToList(redisClient *c, sds s) {
     asyncCloseClientOnOutputBufferLimitReached(c);
 }
 
-void _addReplyStringToList(redisClient *c, char *s, size_t len) {
+void _addReplyStringToList(redisClient *c, const char *s, size_t len) {
     robj *tail;
 
     if (c->flags & REDIS_CLOSE_AFTER_REPLY) return;
@@ -341,19 +368,19 @@ void addReplySds(redisClient *c, sds s) {
     }
 }
 
-void addReplyString(redisClient *c, char *s, size_t len) {
+void addReplyString(redisClient *c, const char *s, size_t len) {
     if (prepareClientToWrite(c) != REDIS_OK) return;
     if (_addReplyToBuffer(c,s,len) != REDIS_OK)
         _addReplyStringToList(c,s,len);
 }
 
-void addReplyErrorLength(redisClient *c, char *s, size_t len) {
+void addReplyErrorLength(redisClient *c, const char *s, size_t len) {
     addReplyString(c,"-ERR ",5);
     addReplyString(c,s,len);
     addReplyString(c,"\r\n",2);
 }
 
-void addReplyError(redisClient *c, char *err) {
+void addReplyError(redisClient *c, const char *err) {
     addReplyErrorLength(c,err,strlen(err));
 }
 
@@ -373,13 +400,13 @@ void addReplyErrorFormat(redisClient *c, const char *fmt, ...) {
     sdsfree(s);
 }
 
-void addReplyStatusLength(redisClient *c, char *s, size_t len) {
+void addReplyStatusLength(redisClient *c, const char *s, size_t len) {
     addReplyString(c,"+",1);
     addReplyString(c,s,len);
     addReplyString(c,"\r\n",2);
 }
 
-void addReplyStatus(redisClient *c, char *status) {
+void addReplyStatus(redisClient *c, const char *status) {
     addReplyStatusLength(c,status,strlen(status));
 }
 
@@ -454,10 +481,10 @@ void addReplyLongLongWithPrefix(redisClient *c, long long ll, char prefix) {
     /* Things like $3\r\n or *2\r\n are emitted very often by the protocol
      * so we have a few shared objects to use if the integer is small
      * like it is most of the times. */
-    if (prefix == '*' && ll < REDIS_SHARED_BULKHDR_LEN) {
+    if (prefix == '*' && ll < REDIS_SHARED_BULKHDR_LEN && ll >= 0) {
         addReply(c,shared.mbulkhdr[ll]);
         return;
-    } else if (prefix == '$' && ll < REDIS_SHARED_BULKHDR_LEN) {
+    } else if (prefix == '$' && ll < REDIS_SHARED_BULKHDR_LEN && ll >= 0) {
         addReply(c,shared.bulkhdr[ll]);
         return;
     }
@@ -519,7 +546,7 @@ void addReplyBulk(redisClient *c, robj *obj) {
 }
 
 /* Add a C buffer as bulk reply */
-void addReplyBulkCBuffer(redisClient *c, void *p, size_t len) {
+void addReplyBulkCBuffer(redisClient *c, const void *p, size_t len) {
     addReplyLongLongWithPrefix(c,len,'$');
     addReplyString(c,p,len);
     addReply(c,shared.crlf);
@@ -534,7 +561,7 @@ void addReplyBulkSds(redisClient *c, sds s)  {
 }
 
 /* Add a C nul term string as bulk reply */
-void addReplyBulkCString(redisClient *c, char *s) {
+void addReplyBulkCString(redisClient *c, const char *s) {
     if (s == NULL) {
         addReply(c,shared.nullbulk);
     } else {
@@ -779,7 +806,7 @@ void freeClient(redisClient *c) {
  * a context where calling freeClient() is not possible, because the client
  * should be valid for the continuation of the flow of the program. */
 void freeClientAsync(redisClient *c) {
-    if (c->flags & REDIS_CLOSE_ASAP) return;
+    if (c->flags & REDIS_CLOSE_ASAP || c->flags & REDIS_LUA_CLIENT) return;
     c->flags |= REDIS_CLOSE_ASAP;
     listAddNodeTail(server.clients_to_close,c);
 }
@@ -797,7 +824,8 @@ void freeClientsInAsyncFreeQueue(void) {
 
 void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask) {
     redisClient *c = privdata;
-    int nwritten = 0, totwritten = 0, objlen;
+    ssize_t nwritten = 0, totwritten = 0;
+    size_t objlen;
     size_t objmem;
     robj *o;
     REDIS_NOTUSED(el);
@@ -812,7 +840,7 @@ void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask) {
 
             /* If the buffer was sent, set bufpos to zero to continue with
              * the remainder of the reply. */
-            if (c->sentlen == c->bufpos) {
+            if ((int)c->sentlen == c->bufpos) {
                 c->bufpos = 0;
                 c->sentlen = 0;
             }
@@ -956,7 +984,7 @@ int processInlineBuffer(redisClient *c) {
 /* Helper function. Trims query buffer to make the function that processes
  * multi bulk requests idempotent. */
 static void setProtocolError(redisClient *c, int pos) {
-    if (server.verbosity >= REDIS_VERBOSE) {
+    if (server.verbosity <= REDIS_VERBOSE) {
         sds client = catClientInfoString(sdsempty(),c);
         redisLog(REDIS_VERBOSE,
             "Protocol error from client: %s", client);
@@ -1105,18 +1133,19 @@ int processMultibulkBuffer(redisClient *c) {
 }
 
 void processInputBuffer(redisClient *c) {
+    server.current_client = c;
     /* Keep processing while there is something in the input buffer */
     while(sdslen(c->querybuf)) {
         /* Return if clients are paused. */
-        if (!(c->flags & REDIS_SLAVE) && clientsArePaused()) return;
+        if (!(c->flags & REDIS_SLAVE) && clientsArePaused()) break;
 
         /* Immediately abort if the client is in the middle of something. */
-        if (c->flags & REDIS_BLOCKED) return;
+        if (c->flags & REDIS_BLOCKED) break;
 
         /* REDIS_CLOSE_AFTER_REPLY closes the connection once the reply is
          * written to the client. Make sure to not let the reply grow after
          * this flag has been set (i.e. don't process more commands). */
-        if (c->flags & REDIS_CLOSE_AFTER_REPLY) return;
+        if (c->flags & REDIS_CLOSE_AFTER_REPLY) break;
 
         /* Determine request type when unknown. */
         if (!c->reqtype) {
@@ -1144,6 +1173,7 @@ void processInputBuffer(redisClient *c) {
                 resetClient(c);
         }
     }
+    server.current_client = NULL;
 }
 
 void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
@@ -1153,7 +1183,6 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
     REDIS_NOTUSED(el);
     REDIS_NOTUSED(mask);
 
-    server.current_client = c;
     readlen = REDIS_IOBUF_LEN;
     /* If this is a multi bulk request, and we are processing a bulk reply
      * that is large enough, try to maximize the probability that the query
@@ -1175,7 +1204,7 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
     nread = read(fd, c->querybuf+qblen, readlen);
     if (nread == -1) {
         if (errno == EAGAIN) {
-            nread = 0;
+            return;
         } else {
             redisLog(REDIS_VERBOSE, "Reading from client: %s",strerror(errno));
             freeClient(c);
@@ -1186,15 +1215,11 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
         freeClient(c);
         return;
     }
-    if (nread) {
-        sdsIncrLen(c->querybuf,nread);
-        c->lastinteraction = server.unixtime;
-        if (c->flags & REDIS_MASTER) c->reploff += nread;
-        server.stat_net_input_bytes += nread;
-    } else {
-        server.current_client = NULL;
-        return;
-    }
+
+    sdsIncrLen(c->querybuf,nread);
+    c->lastinteraction = server.unixtime;
+    if (c->flags & REDIS_MASTER) c->reploff += nread;
+    server.stat_net_input_bytes += nread;
     if (sdslen(c->querybuf) > server.client_max_querybuf_len) {
         sds ci = catClientInfoString(sdsempty(),c), bytes = sdsempty();
 
@@ -1206,7 +1231,6 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
         return;
     }
     processInputBuffer(c);
-    server.current_client = NULL;
 }
 
 void getClientsMaxBuffers(unsigned long *longest_output_list,
@@ -1504,6 +1528,16 @@ void rewriteClientCommandVector(redisClient *c, int argc, ...) {
     va_end(ap);
 }
 
+/* Completely replace the client command vector with the provided one. */
+void replaceClientCommandVector(redisClient *c, int argc, robj **argv) {
+    freeClientArgv(c);
+    zfree(c->argv);
+    c->argv = argv;
+    c->argc = argc;
+    c->cmd = lookupCommandOrOriginal(c->argv[0]->ptr);
+    redisAssertWithInfo(c,NULL,c->cmd != NULL);
+}
+
 /* Rewrite a single item in the command vector.
  * The new val ref count is incremented, and the old decremented. */
 void rewriteClientCommandArgument(redisClient *c, int i, robj *newval) {
@@ -1621,7 +1655,7 @@ int checkClientOutputBufferLimits(redisClient *c) {
  * called from contexts where the client can't be freed safely, i.e. from the
  * lower level functions pushing data inside the client output buffers. */
 void asyncCloseClientOnOutputBufferLimitReached(redisClient *c) {
-    redisAssert(c->reply_bytes < ULONG_MAX-(1024*64));
+    redisAssert(c->reply_bytes < SIZE_MAX-(1024*64));
     if (c->reply_bytes == 0 || c->flags & REDIS_CLOSE_ASAP) return;
     if (checkClientOutputBufferLimits(c)) {
         sds client = catClientInfoString(sdsempty(),c);
@@ -1679,7 +1713,9 @@ void pauseClients(mstime_t end) {
 /* Return non-zero if clients are currently paused. As a side effect the
  * function checks if the pause time was reached and clear it. */
 int clientsArePaused(void) {
-    if (server.clients_paused && server.clients_pause_end_time < server.mstime) {
+    if (server.clients_paused &&
+        server.clients_pause_end_time < server.mstime)
+    {
         listNode *ln;
         listIter li;
         redisClient *c;
@@ -1692,7 +1728,10 @@ int clientsArePaused(void) {
         while ((ln = listNext(&li)) != NULL) {
             c = listNodeValue(ln);
 
-            if (c->flags & REDIS_SLAVE) continue;
+            /* Don't touch slaves and blocked clients. The latter pending
+             * requests be processed when unblocked. */
+            if (c->flags & (REDIS_SLAVE|REDIS_BLOCKED)) continue;
+            c->flags |= REDIS_UNBLOCKED;
             listAddNodeTail(server.unblocked_clients,c);
         }
     }

src/object.c

@@ -50,14 +50,14 @@ robj *createObject(int type, void *ptr) {
 
 /* Create a string object with encoding REDIS_ENCODING_RAW, that is a plain
  * string object where o->ptr points to a proper sds string. */
-robj *createRawStringObject(char *ptr, size_t len) {
+robj *createRawStringObject(const char *ptr, size_t len) {
     return createObject(REDIS_STRING,sdsnewlen(ptr,len));
 }
 
 /* Create a string object with encoding REDIS_ENCODING_EMBSTR, that is
  * an object where the sds string is actually an unmodifiable string
  * allocated in the same chunk as the object itself. */
-robj *createEmbeddedStringObject(char *ptr, size_t len) {
+robj *createEmbeddedStringObject(const char *ptr, size_t len) {
     robj *o = zmalloc(sizeof(robj)+sizeof(struct sdshdr)+len+1);
     struct sdshdr *sh = (void*)(o+1);
 
@@ -85,7 +85,7 @@ robj *createEmbeddedStringObject(char *ptr, size_t len) {
  * The current limit of 39 is chosen so that the biggest string object
  * we allocate as EMBSTR will still fit into the 64 byte arena of jemalloc. */
 #define REDIS_ENCODING_EMBSTR_SIZE_LIMIT 39
-robj *createStringObject(char *ptr, size_t len) {
+robj *createStringObject(const char *ptr, size_t len) {
     if (len <= REDIS_ENCODING_EMBSTR_SIZE_LIMIT)
         return createEmbeddedStringObject(ptr,len);
     else
@@ -529,9 +529,7 @@ size_t stringObjectLen(robj *o) {
     if (sdsEncodedObject(o)) {
         return sdslen(o->ptr);
     } else {
-        char buf[32];
-
-        return ll2string(buf,32,(long)o->ptr);
+        return sdigits10((long)o->ptr);
     }
 }
 

src/quicklist.c

@@ -1372,7 +1372,7 @@ REDIS_STATIC void *_quicklistSaver(unsigned char *data, unsigned int sz) {
     unsigned char *vstr;
     if (data) {
         vstr = zmalloc(sz);
-        memcpy(data, vstr, sz);
+        memcpy(vstr, data, sz);
         return vstr;
     }
     return NULL;
@@ -1757,7 +1757,8 @@ int quicklistTest(int argc, char *argv[]) {
 
         TEST("pop 1 string from 1") {
             quicklist *ql = quicklistNew(-2, options[_i]);
-            quicklistPushHead(ql, genstr("hello", 331), 32);
+            char *populate = genstr("hello", 331);
+            quicklistPushHead(ql, populate, 32);
             unsigned char *data;
             unsigned int sz;
             long long lv;
@@ -1765,6 +1766,9 @@ int quicklistTest(int argc, char *argv[]) {
             quicklistPop(ql, QUICKLIST_HEAD, &data, &sz, &lv);
             assert(data != NULL);
             assert(sz == 32);
+            if (strcmp(populate, (char *)data))
+                ERR("Pop'd value (%.*s) didn't equal original value (%s)", sz,
+                    data, populate);
             zfree(data);
             ql_verify(ql, 0, 0, 0, 0);
             quicklistRelease(ql);
@@ -1797,6 +1801,9 @@ int quicklistTest(int argc, char *argv[]) {
                 assert(ret == 1);
                 assert(data != NULL);
                 assert(sz == 32);
+                if (strcmp(genstr("hello", 499 - i), (char *)data))
+                    ERR("Pop'd value (%.*s) didn't equal original value (%s)",
+                        sz, data, genstr("hello", 499 - i));
                 zfree(data);
             }
             ql_verify(ql, 0, 0, 0, 0);
@@ -1816,6 +1823,10 @@ int quicklistTest(int argc, char *argv[]) {
                     assert(ret == 1);
                     assert(data != NULL);
                     assert(sz == 32);
+                    if (strcmp(genstr("hello", 499 - i), (char *)data))
+                        ERR("Pop'd value (%.*s) didn't equal original value "
+                            "(%s)",
+                            sz, data, genstr("hello", 499 - i));
                     zfree(data);
                 } else {
                     assert(ret == 0);

src/rdb.c

@@ -232,10 +232,10 @@ int rdbTryIntegerEncoding(char *s, size_t len, unsigned char *enc) {
     return rdbEncodeInteger(value,enc);
 }
 
-int rdbSaveLzfBlob(rio *rdb, void *data, size_t compress_len,
-                   size_t original_len) {
+ssize_t rdbSaveLzfBlob(rio *rdb, void *data, size_t compress_len,
+                       size_t original_len) {
     unsigned char byte;
-    int n, nwritten = 0;
+    ssize_t n, nwritten = 0;
 
     /* Data compressed! Let's save it on disk */
     byte = (REDIS_RDB_ENCVAL<<6)|REDIS_RDB_ENC_LZF;
@@ -257,7 +257,7 @@ writeerr:
     return -1;
 }
 
-int rdbSaveLzfStringObject(rio *rdb, unsigned char *s, size_t len) {
+ssize_t rdbSaveLzfStringObject(rio *rdb, unsigned char *s, size_t len) {
     size_t comprlen, outlen;
     void *out;
 
@@ -270,7 +270,7 @@ int rdbSaveLzfStringObject(rio *rdb, unsigned char *s, size_t len) {
         zfree(out);
         return 0;
     }
-    size_t nwritten = rdbSaveLzfBlob(rdb, out, comprlen, len);
+    ssize_t nwritten = rdbSaveLzfBlob(rdb, out, comprlen, len);
     zfree(out);
     return nwritten;
 }
@@ -315,9 +315,9 @@ err:
 
 /* Save a string object as [len][data] on disk. If the object is a string
  * representation of an integer value we try to save it in a special form */
-int rdbSaveRawString(rio *rdb, unsigned char *s, size_t len) {
+ssize_t rdbSaveRawString(rio *rdb, unsigned char *s, size_t len) {
     int enclen;
-    int n, nwritten = 0;
+    ssize_t n, nwritten = 0;
 
     /* Try integer encoding */
     if (len <= 11) {
@@ -348,9 +348,9 @@ int rdbSaveRawString(rio *rdb, unsigned char *s, size_t len) {
 }
 
 /* Save a long long value as either an encoded string or a string. */
-int rdbSaveLongLongAsStringObject(rio *rdb, long long value) {
+ssize_t rdbSaveLongLongAsStringObject(rio *rdb, long long value) {
     unsigned char buf[32];
-    int n, nwritten = 0;
+    ssize_t n, nwritten = 0;
     int enclen = rdbEncodeInteger(value,buf);
     if (enclen > 0) {
         return rdbWriteRaw(rdb,buf,enclen);
@@ -542,8 +542,8 @@ int rdbLoadObjectType(rio *rdb) {
 }
 
 /* Save a Redis object. Returns -1 on error, number of bytes written on success. */
-int rdbSaveObject(rio *rdb, robj *o) {
-    int n = 0, nwritten = 0;
+ssize_t rdbSaveObject(rio *rdb, robj *o) {
+    ssize_t n = 0, nwritten = 0;
 
     if (o->type == REDIS_STRING) {
         /* Save a string value */
@@ -664,8 +664,8 @@ int rdbSaveObject(rio *rdb, robj *o) {
  * the rdbSaveObject() function. Currently we use a trick to get
  * this length with very little changes to the code. In the future
  * we could switch to a faster solution. */
-off_t rdbSavedObjectLen(robj *o) {
-    int len = rdbSaveObject(NULL,o);
+size_t rdbSavedObjectLen(robj *o) {
+    ssize_t len = rdbSaveObject(NULL,o);
     redisAssertWithInfo(NULL,o,len != -1);
     return len;
 }
@@ -869,9 +869,9 @@ int rdbSave(char *filename) {
     return REDIS_OK;
 
 werr:
+    redisLog(REDIS_WARNING,"Write error saving DB on disk: %s", strerror(errno));
     fclose(fp);
     unlink(tmpfile);
-    redisLog(REDIS_WARNING,"Write error saving DB on disk: %s", strerror(errno));
     return REDIS_ERR;
 }
 
@@ -1187,18 +1187,19 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
 
 /* Mark that we are loading in the global state and setup the fields
  * needed to provide loading stats. */
-void startLoading(FILE *fp) {
-    struct stat sb;
-
+void startLoading(size_t size) {
     /* Load the DB */
     server.loading = 1;
     server.loading_start_time = time(NULL);
     server.loading_loaded_bytes = 0;
-    if (fstat(fileno(fp), &sb) == -1) {
-        server.loading_total_bytes = 0;
-    } else {
-        server.loading_total_bytes = sb.st_size;
-    }
+    server.loading_total_bytes = size;
+}
+
+void startLoadingFile(FILE *fp) {
+    struct stat sb;
+    if (fstat(fileno(fp), &sb) == -1)
+        sb.st_size = 0;
+    startLoading(sb.st_size);
 }
 
 /* Refresh the loading progress info */
@@ -1233,66 +1234,74 @@ void rdbLoadProgressCallback(rio *r, const void *buf, size_t len) {
 }
 
 int rdbLoad(char *filename) {
+    FILE *fp;
+    int retval;
+    rio rdb;
+    struct stat sb;
+    if ((fp = fopen(filename,"r")) == NULL) return REDIS_ERR;
+    if (fstat(fileno(fp), &sb) == -1)
+        sb.st_size = 0;
+    rioInitWithFile(&rdb,fp);
+    retval = rdbLoadRio(&rdb, sb.st_size);
+    fclose(fp);
+    return retval;
+}
+
+int rdbLoadRio(rio *rdb, size_t size) {
     uint32_t dbid;
     int type, rdbver;
     redisDb *db = server.db+0;
     char buf[1024];
     long long expiretime, now = mstime();
-    FILE *fp;
-    rio rdb;
 
-    if ((fp = fopen(filename,"r")) == NULL) return REDIS_ERR;
 
-    rioInitWithFile(&rdb,fp);
-    rdb.update_cksum = rdbLoadProgressCallback;
-    rdb.max_processing_chunk = server.loading_process_events_interval_bytes;
-    if (rioRead(&rdb,buf,9) == 0) goto eoferr;
+    rdb->update_cksum = rdbLoadProgressCallback;
+    rdb->max_processing_chunk = server.loading_process_events_interval_bytes;
+    if (rioRead(rdb,buf,9) == 0) goto eoferr;
     buf[9] = '\0';
     if (memcmp(buf,"REDIS",5) != 0) {
-        fclose(fp);
         redisLog(REDIS_WARNING,"Wrong signature trying to load DB from file");
         errno = EINVAL;
         return REDIS_ERR;
     }
     rdbver = atoi(buf+5);
     if (rdbver < 1 || rdbver > REDIS_RDB_VERSION) {
-        fclose(fp);
         redisLog(REDIS_WARNING,"Can't handle RDB format version %d",rdbver);
         errno = EINVAL;
         return REDIS_ERR;
     }
 
-    startLoading(fp);
+    startLoading(size);
     while(1) {
         robj *key, *val;
         expiretime = -1;
 
         /* Read type. */
-        if ((type = rdbLoadType(&rdb)) == -1) goto eoferr;
+        if ((type = rdbLoadType(rdb)) == -1) goto eoferr;
 
         /* Handle special types. */
         if (type == REDIS_RDB_OPCODE_EXPIRETIME) {
             /* EXPIRETIME: load an expire associated with the next key
              * to load. Note that after loading an expire we need to
              * load the actual type, and continue. */
-            if ((expiretime = rdbLoadTime(&rdb)) == -1) goto eoferr;
+            if ((expiretime = rdbLoadTime(rdb)) == -1) goto eoferr;
             /* We read the time so we need to read the object type again. */
-            if ((type = rdbLoadType(&rdb)) == -1) goto eoferr;
+            if ((type = rdbLoadType(rdb)) == -1) goto eoferr;
             /* the EXPIRETIME opcode specifies time in seconds, so convert
              * into milliseconds. */
             expiretime *= 1000;
         } else if (type == REDIS_RDB_OPCODE_EXPIRETIME_MS) {
             /* EXPIRETIME_MS: milliseconds precision expire times introduced
              * with RDB v3. Like EXPIRETIME but no with more precision. */
-            if ((expiretime = rdbLoadMillisecondTime(&rdb)) == -1) goto eoferr;
+            if ((expiretime = rdbLoadMillisecondTime(rdb)) == -1) goto eoferr;
             /* We read the time so we need to read the object type again. */
-            if ((type = rdbLoadType(&rdb)) == -1) goto eoferr;
+            if ((type = rdbLoadType(rdb)) == -1) goto eoferr;
         } else if (type == REDIS_RDB_OPCODE_EOF) {
             /* EOF: End of file, exit the main loop. */
             break;
         } else if (type == REDIS_RDB_OPCODE_SELECTDB) {
             /* SELECTDB: Select the specified database. */
-            if ((dbid = rdbLoadLen(&rdb,NULL)) == REDIS_RDB_LENERR)
+            if ((dbid = rdbLoadLen(rdb,NULL)) == REDIS_RDB_LENERR)
                 goto eoferr;
             if (dbid >= (unsigned)server.dbnum) {
                 redisLog(REDIS_WARNING,
@@ -1307,9 +1316,9 @@ int rdbLoad(char *filename) {
             /* RESIZEDB: Hint about the size of the keys in the currently
              * selected data base, in order to avoid useless rehashing. */
             uint32_t db_size, expires_size;
-            if ((db_size = rdbLoadLen(&rdb,NULL)) == REDIS_RDB_LENERR)
+            if ((db_size = rdbLoadLen(rdb,NULL)) == REDIS_RDB_LENERR)
                 goto eoferr;
-            if ((expires_size = rdbLoadLen(&rdb,NULL)) == REDIS_RDB_LENERR)
+            if ((expires_size = rdbLoadLen(rdb,NULL)) == REDIS_RDB_LENERR)
                 goto eoferr;
             dictExpand(db->dict,db_size);
             dictExpand(db->expires,expires_size);
@@ -1321,8 +1330,8 @@ int rdbLoad(char *filename) {
              *
              * An AUX field is composed of two strings: key and value. */
             robj *auxkey, *auxval;
-            if ((auxkey = rdbLoadStringObject(&rdb)) == NULL) goto eoferr;
-            if ((auxval = rdbLoadStringObject(&rdb)) == NULL) goto eoferr;
+            if ((auxkey = rdbLoadStringObject(rdb)) == NULL) goto eoferr;
+            if ((auxval = rdbLoadStringObject(rdb)) == NULL) goto eoferr;
 
             if (((char*)auxkey->ptr)[0] == '%') {
                 /* All the fields with a name staring with '%' are considered
@@ -1344,9 +1353,9 @@ int rdbLoad(char *filename) {
         }
 
         /* Read key */
-        if ((key = rdbLoadStringObject(&rdb)) == NULL) goto eoferr;
+        if ((key = rdbLoadStringObject(rdb)) == NULL) goto eoferr;
         /* Read value */
-        if ((val = rdbLoadObject(type,&rdb)) == NULL) goto eoferr;
+        if ((val = rdbLoadObject(type,rdb)) == NULL) goto eoferr;
         /* Check if the key already expired. This function is used when loading
          * an RDB file from disk, either at startup, or when an RDB was
          * received from the master. In the latter case, the master is
@@ -1367,9 +1376,9 @@ int rdbLoad(char *filename) {
     }
     /* Verify the checksum if RDB version is >= 5 */
     if (rdbver >= 5 && server.rdb_checksum) {
-        uint64_t cksum, expected = rdb.cksum;
+        uint64_t cksum, expected = rdb->cksum;
 
-        if (rioRead(&rdb,&cksum,8) == 0) goto eoferr;
+        if (rioRead(rdb,&cksum,8) == 0) goto eoferr;
         memrev64ifbe(&cksum);
         if (cksum == 0) {
             redisLog(REDIS_WARNING,"RDB file was saved with checksum disabled: no check performed.");
@@ -1379,7 +1388,6 @@ int rdbLoad(char *filename) {
         }
     }
 
-    fclose(fp);
     stopLoading();
     return REDIS_OK;
 
@@ -1572,7 +1580,7 @@ int rdbSaveToSlavesSockets(void) {
             clientids[numfds] = slave->id;
             fds[numfds++] = slave->fd;
             slave->replstate = REDIS_REPL_WAIT_BGSAVE_END;
-            /* Put the socket in non-blocking mode to simplify RDB transfer.
+            /* Put the socket in blocking mode to simplify RDB transfer.
              * We'll restore it when the children returns (since duped socket
              * will share the O_NONBLOCK attribute with the parent). */
             anetBlock(NULL,slave->fd);
@@ -1646,6 +1654,7 @@ int rdbSaveToSlavesSockets(void) {
             zfree(msg);
         }
         zfree(clientids);
+        rioFreeFdset(&slave_sockets);
         exitFromChild((retval == REDIS_OK) ? 0 : 1);
     } else {
         /* Parent */

src/rdb.h

@@ -105,13 +105,13 @@ uint32_t rdbLoadLen(rio *rdb, int *isencoded);
 int rdbSaveObjectType(rio *rdb, robj *o);
 int rdbLoadObjectType(rio *rdb);
 int rdbLoad(char *filename);
+int rdbLoadRio(rio *rdb, size_t size);
 int rdbSaveBackground(char *filename);
 int rdbSaveToSlavesSockets(void);
 void rdbRemoveTempFile(pid_t childpid);
 int rdbSave(char *filename);
-int rdbSaveObject(rio *rdb, robj *o);
-off_t rdbSavedObjectLen(robj *o);
-off_t rdbSavedObjectPages(robj *o);
+ssize_t rdbSaveObject(rio *rdb, robj *o);
+size_t rdbSavedObjectLen(robj *o);
 robj *rdbLoadObject(int type, rio *rdb);
 void backgroundSaveDoneHandler(int exitcode, int bysignal);
 int rdbSaveKeyValuePair(rio *rdb, robj *key, robj *val, long long expiretime, long long now);

src/redis-benchmark.c

@@ -86,7 +86,7 @@ typedef struct _client {
     char **randptr;         /* Pointers to :rand: strings inside the command buf */
     size_t randlen;         /* Number of pointers in client->randptr */
     size_t randfree;        /* Number of unused pointers in client->randptr */
-    unsigned int written;   /* Bytes of 'obuf' already written */
+    size_t written;         /* Bytes of 'obuf' already written */
     long long start;        /* Start time of a request */
     long long latency;      /* Request latency */
     int pending;            /* Number of pending requests (replies to consume) */
@@ -266,7 +266,7 @@ static void writeHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
 
     if (sdslen(c->obuf) > c->written) {
         void *ptr = c->obuf+c->written;
-        int nwritten = write(c->context->fd,ptr,sdslen(c->obuf)-c->written);
+        ssize_t nwritten = write(c->context->fd,ptr,sdslen(c->obuf)-c->written);
         if (nwritten == -1) {
             if (errno != EPIPE)
                 fprintf(stderr, "Writing to socket: %s\n", strerror(errno));

src/redis-cli.c

@@ -64,6 +64,17 @@
 #define REDIS_CLI_HISTFILE_ENV "REDISCLI_HISTFILE"
 #define REDIS_CLI_HISTFILE_DEFAULT ".rediscli_history"
 
+/* --latency-dist palettes. */
+int spectrum_palette_color_size = 19;
+int spectrum_palette_color[] = {0,233,234,235,237,239,241,243,245,247,144,143,142,184,226,214,208,202,196};
+
+int spectrum_palette_mono_size = 13;
+int spectrum_palette_mono[] = {0,233,234,235,237,239,241,243,245,247,249,251,253};
+
+/* The actual palette in use. */
+int *spectrum_palette;
+int spectrum_palette_size;
+
 static redisContext *context;
 static struct config {
     char *hostip;
@@ -79,6 +90,8 @@ static struct config {
     int latency_mode;
     int latency_dist_mode;
     int latency_history;
+    int lru_test_mode;
+    long long lru_test_sample_size;
     int cluster_mode;
     int cluster_reissue_command;
     int slave_mode;
@@ -631,9 +644,9 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
 
     output_raw = 0;
     if (!strcasecmp(command,"info") ||
-        (argc == 3 && !strcasecmp(command,"debug") &&
-                      (!strcasecmp(argv[1],"jemalloc") &&
-                       !strcasecmp(argv[2],"info"))) ||
+        (argc >= 2 && !strcasecmp(command,"debug") &&
+                      (!strcasecmp(argv[1],"jemalloc") ||
+                       !strcasecmp(argv[1],"htstats"))) ||
         (argc == 2 && !strcasecmp(command,"cluster") &&
                       (!strcasecmp(argv[1],"nodes") ||
                        !strcasecmp(argv[1],"info"))) ||
@@ -702,16 +715,17 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
     return REDIS_OK;
 }
 
-/* Send the INFO command, reconnecting the link if needed. */
-static redisReply *reconnectingInfo(void) {
-    redisContext *c = context;
+/* Send a command reconnecting the link if needed. */
+static redisReply *reconnectingRedisCommand(redisContext *c, const char *fmt, ...) {
     redisReply *reply = NULL;
     int tries = 0;
+    va_list ap;
 
     assert(!c->err);
     while(reply == NULL) {
         while (c->err & (REDIS_ERR_IO | REDIS_ERR_EOF)) {
-            printf("Reconnecting (%d)...\r", ++tries);
+            printf("\r\x1b[0K"); /* Cursor to left edge + clear line. */
+            printf("Reconnecting... %d\r", ++tries);
             fflush(stdout);
 
             redisFree(c);
@@ -719,12 +733,15 @@ static redisReply *reconnectingInfo(void) {
             usleep(1000000);
         }
 
-        reply = redisCommand(c,"INFO");
+        va_start(ap,fmt);
+        reply = redisvCommand(c,fmt,ap);
+        va_end(ap);
+
         if (c->err && !(c->err & (REDIS_ERR_IO | REDIS_ERR_EOF))) {
             fprintf(stderr, "Error: %s\n", c->errstr);
             exit(1);
         } else if (tries > 0) {
-            printf("\n");
+            printf("\r\x1b[0K"); /* Cursor to left edge + clear line. */
         }
     }
 
@@ -774,9 +791,15 @@ static int parseOptions(int argc, char **argv) {
             config.latency_mode = 1;
         } else if (!strcmp(argv[i],"--latency-dist")) {
             config.latency_dist_mode = 1;
+        } else if (!strcmp(argv[i],"--mono")) {
+            spectrum_palette = spectrum_palette_mono;
+            spectrum_palette_size = spectrum_palette_mono_size;
         } else if (!strcmp(argv[i],"--latency-history")) {
             config.latency_mode = 1;
             config.latency_history = 1;
+        } else if (!strcmp(argv[i],"--lru-test") && !lastarg) {
+            config.lru_test_mode = 1;
+            config.lru_test_sample_size = strtoll(argv[++i],NULL,10);
         } else if (!strcmp(argv[i],"--slave")) {
             config.slave_mode = 1;
         } else if (!strcmp(argv[i],"--stat")) {
@@ -868,6 +891,7 @@ static void usage(void) {
 "                     Default time interval is 15 sec. Change it using -i.\n"
 "  --latency-dist     Shows latency as a spectrum, requires xterm 256 colors.\n"
 "                     Default time interval is 1 sec. Change it using -i.\n"
+"  --lru-test <keys>  Simulate a cache workload with an 80-20 distribution.\n"
 "  --slave            Simulate a slave showing commands received from the master.\n"
 "  --rdb <filename>   Transfer an RDB dump from remote server to local file.\n"
 "  --pipe             Transfer raw Redis protocol from stdin to server.\n"
@@ -1083,7 +1107,7 @@ static void latencyMode(void) {
     if (!context) exit(1);
     while(1) {
         start = mstime();
-        reply = redisCommand(context,"PING");
+        reply = reconnectingRedisCommand(context,"PING");
         if (reply == NULL) {
             fprintf(stderr,"\nI/O error\n");
             exit(1);
@@ -1118,13 +1142,6 @@ static void latencyMode(void) {
  *--------------------------------------------------------------------------- */
 
 #define LATENCY_DIST_DEFAULT_INTERVAL 1000 /* milliseconds. */
-#define LATENCY_DIST_MIN_GRAY 233 /* Less than that is too hard to see gray. */
-#define LATENCY_DIST_MAX_GRAY 255
-#define LATENCY_DIST_GRAYS (LATENCY_DIST_MAX_GRAY-LATENCY_DIST_MIN_GRAY+1)
-
-/* Gray palette. */
-int spectrum_palette_size = 24;
-int spectrum_palette[] = {0, 233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255};
 
 /* Structure to store samples distribution. */
 struct distsamples {
@@ -1233,7 +1250,7 @@ static void latencyDistMode(void) {
     if (!context) exit(1);
     while(1) {
         start = ustime();
-        reply = redisCommand(context,"PING");
+        reply = reconnectingRedisCommand(context,"PING");
         if (reply == NULL) {
             fprintf(stderr,"\nI/O error\n");
             exit(1);
@@ -1881,7 +1898,7 @@ static void statMode(void) {
         char buf[64];
         int j;
 
-        reply = reconnectingInfo();
+        reply = reconnectingRedisCommand(context,"INFO");
         if (reply->type == REDIS_REPLY_ERROR) {
             printf("ERROR: %s\n", reply->str);
             exit(1);
@@ -1935,6 +1952,7 @@ static void statMode(void) {
         /* Children */
         aux = getLongInfoField(reply->str,"bgsave_in_progress");
         aux |= getLongInfoField(reply->str,"aof_rewrite_in_progress") << 1;
+        aux |= getLongInfoField(reply->str,"loading") << 2;
         switch(aux) {
         case 0: break;
         case 1:
@@ -1946,6 +1964,9 @@ static void statMode(void) {
         case 3:
             printf("SAVE+AOF");
             break;
+        case 4:
+            printf("LOAD");
+            break;
         }
 
         printf("\n");
@@ -1987,6 +2008,94 @@ static void scanMode(void) {
     exit(0);
 }
 
+/*------------------------------------------------------------------------------
+ * LRU test mode
+ *--------------------------------------------------------------------------- */
+
+/* Return an integer from min to max (both inclusive) using a power-law
+ * distribution, depending on the value of alpha: the greater the alpha
+ * the more bias towards lower values.
+ *
+ * With alpha = 6.2 the output follows the 80-20 rule where 20% of
+ * the returned numbers will account for 80% of the frequency. */
+long long powerLawRand(long long min, long long max, double alpha) {
+    double pl, r;
+
+    max += 1;
+    r = ((double)rand()) / RAND_MAX;
+    pl = pow(
+        ((pow(max,alpha+1) - pow(min,alpha+1))*r + pow(min,alpha+1)),
+        (1.0/(alpha+1)));
+    return (max-1-(long long)pl)+min;
+}
+
+/* Generates a key name among a set of lru_test_sample_size keys, using
+ * an 80-20 distribution. */
+void LRUTestGenKey(char *buf, size_t buflen) {
+    snprintf(buf, buflen, "lru:%lld\n",
+        powerLawRand(1, config.lru_test_sample_size, 6.2));
+}
+
+#define LRU_CYCLE_PERIOD 1000 /* 1000 milliseconds. */
+#define LRU_CYCLE_PIPELINE_SIZE 250
+static void LRUTestMode(void) {
+    redisReply *reply;
+    char key[128];
+    long long start_cycle;
+    int j;
+
+    srand(time(NULL)^getpid());
+    while(1) {
+        /* Perform cycles of 1 second with 50% writes and 50% reads.
+         * We use pipelining batching writes / reads N times per cycle in order
+         * to fill the target instance easily. */
+        start_cycle = mstime();
+        long long hits = 0, misses = 0;
+        while(mstime() - start_cycle < 1000) {
+            /* Write cycle. */
+            for (j = 0; j < LRU_CYCLE_PIPELINE_SIZE; j++) {
+                LRUTestGenKey(key,sizeof(key));
+                redisAppendCommand(context, "SET %s val",key);
+            }
+            for (j = 0; j < LRU_CYCLE_PIPELINE_SIZE; j++)
+                redisGetReply(context, (void**)&reply);
+
+            /* Read cycle. */
+            for (j = 0; j < LRU_CYCLE_PIPELINE_SIZE; j++) {
+                LRUTestGenKey(key,sizeof(key));
+                redisAppendCommand(context, "GET %s",key);
+            }
+            for (j = 0; j < LRU_CYCLE_PIPELINE_SIZE; j++) {
+                if (redisGetReply(context, (void**)&reply) == REDIS_OK) {
+                    switch(reply->type) {
+                        case REDIS_REPLY_ERROR:
+                            printf("%s\n", reply->str);
+                            break;
+                        case REDIS_REPLY_NIL:
+                            misses++;
+                            break;
+                        default:
+                            hits++;
+                            break;
+                    }
+                }
+            }
+
+            if (context->err) {
+                fprintf(stderr,"I/O error during LRU test\n");
+                exit(1);
+            }
+        }
+        /* Print stats. */
+        printf(
+            "%lld Gets/sec | Hits: %lld (%.2f%%) | Misses: %lld (%.2f%%)\n",
+            hits+misses,
+            hits, (double)hits/(hits+misses)*100,
+            misses, (double)misses/(hits+misses)*100);
+    }
+    exit(0);
+}
+
 /*------------------------------------------------------------------------------
  * Intrisic latency mode.
  *
@@ -2080,6 +2189,8 @@ int main(int argc, char **argv) {
     config.latency_mode = 0;
     config.latency_dist_mode = 0;
     config.latency_history = 0;
+    config.lru_test_mode = 0;
+    config.lru_test_sample_size = 0;
     config.cluster_mode = 0;
     config.slave_mode = 0;
     config.getrdb_mode = 0;
@@ -2096,6 +2207,9 @@ int main(int argc, char **argv) {
     config.eval = NULL;
     config.last_cmd_type = -1;
 
+    spectrum_palette = spectrum_palette_color;
+    spectrum_palette_size = spectrum_palette_color_size;
+
     if (!isatty(fileno(stdout)) && (getenv("FAKETTY") == NULL))
         config.output = OUTPUT_RAW;
     else
@@ -2156,6 +2270,12 @@ int main(int argc, char **argv) {
         scanMode();
     }
 
+    /* LRU test mode */
+    if (config.lru_test_mode) {
+        if (cliConnect(0) == REDIS_ERR) exit(1);
+        LRUTestMode();
+    }
+
     /* Intrinsic latency mode */
     if (config.intrinsic_latency_mode) intrinsicLatencyMode();
 

src/redis.c

@@ -53,7 +53,7 @@
 #include <sys/resource.h>
 #include <sys/utsname.h>
 #include <locale.h>
-#include <sys/sysctl.h>
+#include <sys/socket.h>
 
 /* Our shared "common" objects */
 
@@ -131,7 +131,7 @@ struct redisCommand redisCommandTable[] = {
     {"append",appendCommand,3,"wm",0,NULL,1,1,1,0,0},
     {"strlen",strlenCommand,2,"rF",0,NULL,1,1,1,0,0},
     {"del",delCommand,-2,"w",0,NULL,1,-1,1,0,0},
-    {"exists",existsCommand,2,"rF",0,NULL,1,1,1,0,0},
+    {"exists",existsCommand,-2,"rF",0,NULL,1,-1,1,0,0},
     {"setbit",setbitCommand,4,"wm",0,NULL,1,1,1,0,0},
     {"getbit",getbitCommand,3,"rF",0,NULL,1,1,1,0,0},
     {"setrange",setrangeCommand,4,"wm",0,NULL,1,1,1,0,0},
@@ -202,6 +202,7 @@ struct redisCommand redisCommandTable[] = {
     {"hincrbyfloat",hincrbyfloatCommand,4,"wmF",0,NULL,1,1,1,0,0},
     {"hdel",hdelCommand,-3,"wF",0,NULL,1,1,1,0,0},
     {"hlen",hlenCommand,2,"rF",0,NULL,1,1,1,0,0},
+    {"hstrlen",hstrlenCommand,3,"rF",0,NULL,1,1,1,0,0},
     {"hkeys",hkeysCommand,2,"rS",0,NULL,1,1,1,0,0},
     {"hvals",hvalsCommand,2,"rS",0,NULL,1,1,1,0,0},
     {"hgetall",hgetallCommand,2,"r",0,NULL,1,1,1,0,0},
@@ -280,9 +281,15 @@ struct redisCommand redisCommandTable[] = {
     {"bitpos",bitposCommand,-3,"r",0,NULL,1,1,1,0,0},
     {"wait",waitCommand,3,"rs",0,NULL,0,0,0,0,0},
     {"command",commandCommand,0,"rlt",0,NULL,0,0,0,0,0},
+    {"geoadd",geoaddCommand,-5,"wm",0,NULL,1,1,1,0,0},
+    {"georadius",georadiusCommand,-6,"r",0,NULL,1,1,1,0,0},
+    {"georadiusbymember",georadiusByMemberCommand,-5,"r",0,NULL,1,1,1,0,0},
+    {"geohash",geohashCommand,-2,"r",0,NULL,1,1,1,0,0},
+    {"geopos",geoposCommand,-2,"r",0,NULL,1,1,1,0,0},
+    {"geodist",geodistCommand,-4,"r",0,NULL,1,1,1,0,0},
     {"pfselftest",pfselftestCommand,1,"r",0,NULL,0,0,0,0,0},
     {"pfadd",pfaddCommand,-2,"wmF",0,NULL,1,1,1,0,0},
-    {"pfcount",pfcountCommand,-2,"r",0,NULL,1,1,1,0,0},
+    {"pfcount",pfcountCommand,-2,"r",0,NULL,1,-1,1,0,0},
     {"pfmerge",pfmergeCommand,-2,"wm",0,NULL,1,-1,1,0,0},
     {"pfdebug",pfdebugCommand,-3,"w",0,NULL,0,0,0,0,0},
     {"latency",latencyCommand,-2,"arslt",0,NULL,0,0,0,0,0}
@@ -904,9 +911,12 @@ long long getInstantaneousMetric(int metric) {
     return sum / REDIS_METRIC_SAMPLES;
 }
 
-/* Check for timeouts. Returns non-zero if the client was terminated */
-int clientsCronHandleTimeout(redisClient *c) {
-    time_t now = server.unixtime;
+/* Check for timeouts. Returns non-zero if the client was terminated.
+ * The function gets the current time in milliseconds as argument since
+ * it gets called multiple times in a loop, so calling gettimeofday() for
+ * each iteration would be costly without any actual gain. */
+int clientsCronHandleTimeout(redisClient *c, mstime_t now_ms) {
+    time_t now = now_ms/1000;
 
     if (server.maxidletime &&
         !(c->flags & REDIS_SLAVE) &&    /* no timeout for slaves */
@@ -922,11 +932,16 @@ int clientsCronHandleTimeout(redisClient *c) {
         /* Blocked OPS timeout is handled with milliseconds resolution.
          * However note that the actual resolution is limited by
          * server.hz. */
-        mstime_t now_ms = mstime();
 
         if (c->bpop.timeout != 0 && c->bpop.timeout < now_ms) {
+            /* Handle blocking operation specific timeout. */
             replyToBlockedClientTimedOut(c);
             unblockClient(c);
+        } else if (server.cluster_enabled) {
+            /* Cluster: handle unblock & redirect of clients blocked
+             * into keys no longer served by this server. */
+            if (clusterRedirectBlockedClientIfNeeded(c))
+                unblockClient(c);
         }
     }
     return 0;
@@ -958,17 +973,23 @@ int clientsCronResizeQueryBuffer(redisClient *c) {
     return 0;
 }
 
+#define CLIENTS_CRON_MIN_ITERATIONS 5
 void clientsCron(void) {
-    /* Make sure to process at least 1/(server.hz*10) of clients per call.
-     * Since this function is called server.hz times per second we are sure that
-     * in the worst case we process all the clients in 10 seconds.
-     * In normal conditions (a reasonable number of clients) we process
-     * all the clients in a shorter time. */
+    /* Make sure to process at least numclients/server.hz of clients
+     * per call. Since this function is called server.hz times per second
+     * we are sure that in the worst case we process all the clients in 1
+     * second. */
     int numclients = listLength(server.clients);
-    int iterations = numclients/(server.hz*10);
+    int iterations = numclients/server.hz;
+    mstime_t now = mstime();
+
+    /* Process at least a few clients while we are at it, even if we need
+     * to process less than CLIENTS_CRON_MIN_ITERATIONS to meet our contract
+     * of processing each client once per second. */
+    if (iterations < CLIENTS_CRON_MIN_ITERATIONS)
+        iterations = (numclients < CLIENTS_CRON_MIN_ITERATIONS) ?
+                     numclients : CLIENTS_CRON_MIN_ITERATIONS;
 
-    if (iterations < 50)
-        iterations = (numclients < 50) ? numclients : 50;
     while(listLength(server.clients) && iterations--) {
         redisClient *c;
         listNode *head;
@@ -982,7 +1003,7 @@ void clientsCron(void) {
         /* The following functions do different service checks on the client.
          * The protocol is that they return non-zero if the client was
          * terminated. */
-        if (clientsCronHandleTimeout(c)) continue;
+        if (clientsCronHandleTimeout(c,now)) continue;
         if (clientsCronResizeQueryBuffer(c)) continue;
     }
 }
@@ -1259,6 +1280,12 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
 void beforeSleep(struct aeEventLoop *eventLoop) {
     REDIS_NOTUSED(eventLoop);
 
+    /* Call the Redis Cluster before sleep function. Note that this function
+     * may change the state of Redis Cluster (from ok to fail or vice versa),
+     * so it's a good idea to call it before serving the unblocked clients
+     * later in this function. */
+    if (server.cluster_enabled) clusterBeforeSleep();
+
     /* Run a fast expire cycle (the called function will return
      * ASAP if a fast cycle is not needed). */
     if (server.active_expire_enabled && server.masterhost == NULL)
@@ -1290,9 +1317,6 @@ void beforeSleep(struct aeEventLoop *eventLoop) {
 
     /* Write the AOF buffer on disk */
     flushAppendOnlyFile(0);
-
-    /* Call the Redis Cluster before sleep function. */
-    if (server.cluster_enabled) clusterBeforeSleep();
 }
 
 /* =========================== Server initialization ======================== */
@@ -1489,12 +1513,16 @@ void initServerConfig(void) {
     server.cached_master = NULL;
     server.repl_master_initial_offset = -1;
     server.repl_state = REDIS_REPL_NONE;
+    server.repl_transfer_tmpfile = NULL;
+    server.repl_transfer_fd = -1;
+    server.repl_transfer_s = -1;
     server.repl_syncio_timeout = REDIS_REPL_SYNCIO_TIMEOUT;
     server.repl_serve_stale_data = REDIS_DEFAULT_SLAVE_SERVE_STALE_DATA;
     server.repl_slave_ro = REDIS_DEFAULT_SLAVE_READ_ONLY;
     server.repl_down_since = 0; /* Never connected, repl is down since EVER. */
     server.repl_disable_tcp_nodelay = REDIS_DEFAULT_REPL_DISABLE_TCP_NODELAY;
     server.repl_diskless_sync = REDIS_DEFAULT_REPL_DISKLESS_SYNC;
+    server.repl_diskless_load = REDIS_DEFAULT_REPL_DISKLESS_LOAD;
     server.repl_diskless_sync_delay = REDIS_DEFAULT_REPL_DISKLESS_SYNC_DELAY;
     server.slave_priority = REDIS_DEFAULT_SLAVE_PRIORITY;
     server.master_repl_offset = 0;
@@ -1529,6 +1557,7 @@ void initServerConfig(void) {
     server.lpushCommand = lookupCommandByCString("lpush");
     server.lpopCommand = lookupCommandByCString("lpop");
     server.rpopCommand = lookupCommandByCString("rpop");
+    server.sremCommand = lookupCommandByCString("srem");
 
     /* Slow log */
     server.slowlog_log_slower_than = REDIS_SLOWLOG_LOG_SLOWER_THAN;
@@ -1735,6 +1764,7 @@ void resetServerStats(void) {
     }
     server.stat_net_input_bytes = 0;
     server.stat_net_output_bytes = 0;
+    server.aof_delayed_fsync = 0;
 }
 
 void initServer(void) {
@@ -1779,7 +1809,7 @@ void initServer(void) {
         server.sofd = anetUnixServer(server.neterr,server.unixsocket,
             server.unixsocketperm, server.tcp_backlog);
         if (server.sofd == ANET_ERR) {
-            redisLog(REDIS_WARNING, "Opening socket: %s", server.neterr);
+            redisLog(REDIS_WARNING, "Opening Unix socket: %s", server.neterr);
             exit(1);
         }
         anetNonBlock(NULL,server.sofd);
@@ -2001,6 +2031,9 @@ struct redisCommand *lookupCommandOrOriginal(sds name) {
  * + REDIS_PROPAGATE_NONE (no propagation of command at all)
  * + REDIS_PROPAGATE_AOF (propagate into the AOF file if is enabled)
  * + REDIS_PROPAGATE_REPL (propagate into the replication link)
+ *
+ * This should not be used inside commands implementation. Use instead
+ * alsoPropagate(), preventCommandPropagation(), forceCommandPropagation().
  */
 void propagate(struct redisCommand *cmd, int dbid, robj **argv, int argc,
                int flags)
@@ -2012,11 +2045,31 @@ void propagate(struct redisCommand *cmd, int dbid, robj **argv, int argc,
 }
 
 /* Used inside commands to schedule the propagation of additional commands
- * after the current command is propagated to AOF / Replication. */
+ * after the current command is propagated to AOF / Replication.
+ *
+ * 'cmd' must be a pointer to the Redis command to replicate, dbid is the
+ * database ID the command should be propagated into.
+ * Arguments of the command to propagte are passed as an array of redis
+ * objects pointers of len 'argc', using the 'argv' vector.
+ *
+ * The function does not take a reference to the passed 'argv' vector,
+ * so it is up to the caller to release the passed argv (but it is usually
+ * stack allocated).  The function autoamtically increments ref count of
+ * passed objects, so the caller does not need to. */
 void alsoPropagate(struct redisCommand *cmd, int dbid, robj **argv, int argc,
                    int target)
 {
-    redisOpArrayAppend(&server.also_propagate,cmd,dbid,argv,argc,target);
+    robj **argvcopy;
+    int j;
+
+    if (server.loading) return; /* No propagation during loading. */
+
+    argvcopy = zmalloc(sizeof(robj*)*argc);
+    for (j = 0; j < argc; j++) {
+        argvcopy[j] = argv[j];
+        incrRefCount(argv[j]);
+    }
+    redisOpArrayAppend(&server.also_propagate,cmd,dbid,argvcopy,argc,target);
 }
 
 /* It is possible to call the function forceCommandPropagation() inside a
@@ -2027,6 +2080,13 @@ void forceCommandPropagation(redisClient *c, int flags) {
     if (flags & REDIS_PROPAGATE_AOF) c->flags |= REDIS_FORCE_AOF;
 }
 
+/* Avoid that the executed command is propagated at all. This way we
+ * are free to just propagate what we want using the alsoPropagate()
+ * API. */
+void preventCommandPropagation(redisClient *c) {
+    c->flags |= REDIS_PREVENT_PROP;
+}
+
 /* Call() is the core of Redis execution of a command */
 void call(redisClient *c, int flags) {
     long long dirty, start, duration;
@@ -2080,7 +2140,7 @@ void call(redisClient *c, int flags) {
     }
 
     /* Propagate the command into the AOF and replication link */
-    if (flags & REDIS_CALL_PROPAGATE) {
+    if (flags & REDIS_CALL_PROPAGATE && (c->flags & REDIS_PREVENT_PROP) == 0) {
         int flags = REDIS_PROPAGATE_NONE;
 
         if (c->flags & REDIS_FORCE_REPL) flags |= REDIS_PROPAGATE_REPL;
@@ -2091,20 +2151,24 @@ void call(redisClient *c, int flags) {
             propagate(c->cmd,c->db->id,c->argv,c->argc,flags);
     }
 
-    /* Restore the old FORCE_AOF/REPL flags, since call can be executed
+    /* Restore the old replication flags, since call can be executed
      * recursively. */
-    c->flags &= ~(REDIS_FORCE_AOF|REDIS_FORCE_REPL);
-    c->flags |= client_old_flags & (REDIS_FORCE_AOF|REDIS_FORCE_REPL);
+    c->flags &= ~(REDIS_FORCE_AOF|REDIS_FORCE_REPL|REDIS_PREVENT_PROP);
+    c->flags |= client_old_flags &
+        (REDIS_FORCE_AOF|REDIS_FORCE_REPL|REDIS_PREVENT_PROP);
 
     /* Handle the alsoPropagate() API to handle commands that want to propagate
-     * multiple separated commands. */
+     * multiple separated commands. Note that alsoPropagate() is not affected
+     * by REDIS_PREVENT_PROP flag. */
     if (server.also_propagate.numops) {
         int j;
         redisOp *rop;
 
-        for (j = 0; j < server.also_propagate.numops; j++) {
-            rop = &server.also_propagate.ops[j];
-            propagate(rop->cmd, rop->dbid, rop->argv, rop->argc, rop->target);
+        if (flags & REDIS_CALL_PROPAGATE) {
+            for (j = 0; j < server.also_propagate.numops; j++) {
+                rop = &server.also_propagate.ops[j];
+                propagate(rop->cmd,rop->dbid,rop->argv,rop->argc,rop->target);
+            }
         }
         redisOpArrayFree(&server.also_propagate);
     }
@@ -2160,36 +2224,22 @@ int processCommand(redisClient *c) {
      * 2) The command has no key arguments. */
     if (server.cluster_enabled &&
         !(c->flags & REDIS_MASTER) &&
+        !(c->flags & REDIS_LUA_CLIENT &&
+          server.lua_caller->flags & REDIS_MASTER) &&
         !(c->cmd->getkeys_proc == NULL && c->cmd->firstkey == 0))
     {
         int hashslot;
 
         if (server.cluster->state != REDIS_CLUSTER_OK) {
             flagTransaction(c);
-            addReplySds(c,sdsnew("-CLUSTERDOWN The cluster is down. Use CLUSTER INFO for more information\r\n"));
+            clusterRedirectClient(c,NULL,0,REDIS_CLUSTER_REDIR_DOWN_STATE);
             return REDIS_OK;
         } else {
             int error_code;
             clusterNode *n = getNodeByQuery(c,c->cmd,c->argv,c->argc,&hashslot,&error_code);
-            if (n == NULL) {
+            if (n == NULL || n != server.cluster->myself) {
                 flagTransaction(c);
-                if (error_code == REDIS_CLUSTER_REDIR_CROSS_SLOT) {
-                    addReplySds(c,sdsnew("-CROSSSLOT Keys in request don't hash to the same slot\r\n"));
-                } else if (error_code == REDIS_CLUSTER_REDIR_UNSTABLE) {
-                    /* The request spawns mutliple keys in the same slot,
-                     * but the slot is not "stable" currently as there is
-                     * a migration or import in progress. */
-                    addReplySds(c,sdsnew("-TRYAGAIN Multiple keys request during rehashing of slot\r\n"));
-                } else {
-                    redisPanic("getNodeByQuery() unknown error.");
-                }
-                return REDIS_OK;
-            } else if (n != server.cluster->myself) {
-                flagTransaction(c);
-                addReplySds(c,sdscatprintf(sdsempty(),
-                    "-%s %d %s:%d\r\n",
-                    (error_code == REDIS_CLUSTER_REDIR_ASK) ? "ASK" : "MOVED",
-                    hashslot,n->ip,n->port));
+                clusterRedirectClient(c,n,hashslot,error_code);
                 return REDIS_OK;
             }
         }
@@ -2701,9 +2751,13 @@ sds genRedisInfoString(char *section) {
         char hmem[64];
         char peak_hmem[64];
         char total_system_hmem[64];
+        char used_memory_lua_hmem[64];
+        char used_memory_rss_hmem[64];
+        char maxmemory_hmem[64];
         size_t zmalloc_used = zmalloc_used_memory();
         size_t total_system_mem = server.system_memory_size;
-        char *evict_policy = maxmemoryToString();
+        const char *evict_policy = maxmemoryToString();
+        long long memory_lua = (long long)lua_gc(server.lua,LUA_GCCOUNT,0)*1024;
 
         /* Peak memory is updated from time to time by serverCron() so it
          * may happen that the instantaneous value is slightly bigger than
@@ -2715,6 +2769,9 @@ sds genRedisInfoString(char *section) {
         bytesToHuman(hmem,zmalloc_used);
         bytesToHuman(peak_hmem,server.stat_peak_memory);
         bytesToHuman(total_system_hmem,total_system_mem);
+        bytesToHuman(used_memory_lua_hmem,memory_lua);
+        bytesToHuman(used_memory_rss_hmem,server.resident_set_size);
+        bytesToHuman(maxmemory_hmem,server.maxmemory);
 
         if (sections++) info = sdscat(info,"\r\n");
         info = sdscatprintf(info,
@@ -2722,25 +2779,33 @@ sds genRedisInfoString(char *section) {
             "used_memory:%zu\r\n"
             "used_memory_human:%s\r\n"
             "used_memory_rss:%zu\r\n"
+            "used_memory_rss_human:%s\r\n"
             "used_memory_peak:%zu\r\n"
             "used_memory_peak_human:%s\r\n"
             "total_system_memory:%lu\r\n"
             "total_system_memory_human:%s\r\n"
             "used_memory_lua:%lld\r\n"
+            "used_memory_lua_human:%s\r\n"
+            "maxmemory:%lld\r\n"
+            "maxmemory_human:%s\r\n"
+            "maxmemory_policy:%s\r\n"
             "mem_fragmentation_ratio:%.2f\r\n"
-            "mem_allocator:%s\r\n"
-            "maxmemory_policy:%s\r\n",
+            "mem_allocator:%s\r\n",
             zmalloc_used,
             hmem,
             server.resident_set_size,
+            used_memory_rss_hmem,
             server.stat_peak_memory,
             peak_hmem,
             (unsigned long)total_system_mem,
             total_system_hmem,
-            ((long long)lua_gc(server.lua,LUA_GCCOUNT,0))*1024LL,
+            memory_lua,
+            used_memory_lua_hmem,
+            server.maxmemory,
+            maxmemory_hmem,
+            evict_policy,
             zmalloc_get_fragmentation_ratio(server.resident_set_size),
-            ZMALLOC_LIB,
-            evict_policy
+            ZMALLOC_LIB
             );
     }
 
@@ -3147,13 +3212,7 @@ void evictionPoolPopulate(dict *sampledict, dict *keydict, struct evictionPoolEn
         samples = zmalloc(sizeof(samples[0])*server.maxmemory_samples);
     }
 
-#if 1 /* Use bulk get by default. */
-    count = dictGetRandomKeys(sampledict,samples,server.maxmemory_samples);
-#else
-    count = server.maxmemory_samples;
-    for (j = 0; j < count; j++) samples[j] = dictGetRandomKey(sampledict);
-#endif
-
+    count = dictGetSomeKeys(sampledict,samples,server.maxmemory_samples);
     for (j = 0; j < count; j++) {
         unsigned long long idle;
         sds key;
@@ -3208,7 +3267,7 @@ void evictionPoolPopulate(dict *sampledict, dict *keydict, struct evictionPoolEn
 int freeMemoryIfNeeded(void) {
     size_t mem_used, mem_tofree, mem_freed;
     int slaves = listLength(server.slaves);
-    mstime_t latency;
+    mstime_t latency, eviction_latency;
 
     /* Remove the size of slaves output buffers and AOF buffer from the
      * count of used memory. */
@@ -3339,7 +3398,11 @@ int freeMemoryIfNeeded(void) {
                  * AOF and Output buffer memory will be freed eventually so
                  * we only care about memory used by the key space. */
                 delta = (long long) zmalloc_used_memory();
+                latencyStartMonitor(eviction_latency);
                 dbDelete(db,keyobj);
+                latencyEndMonitor(eviction_latency);
+                latencyAddSampleIfNeeded("eviction-del",eviction_latency);
+                latencyRemoveNestedEvent(latency,eviction_latency);
                 delta -= (long long) zmalloc_used_memory();
                 mem_freed += delta;
                 server.stat_evictedkeys++;

src/redis.h

@@ -118,6 +118,7 @@ typedef long long mstime_t; /* millisecond time type. */
 #define REDIS_DEFAULT_RDB_CHECKSUM 1
 #define REDIS_DEFAULT_RDB_FILENAME "dump.rdb"
 #define REDIS_DEFAULT_REPL_DISKLESS_SYNC 0
+#define REDIS_DEFAULT_REPL_DISKLESS_LOAD 0
 #define REDIS_DEFAULT_REPL_DISKLESS_SYNC_DELAY 5
 #define REDIS_DEFAULT_SLAVE_SERVE_STALE_DATA 1
 #define REDIS_DEFAULT_SLAVE_READ_ONLY 1
@@ -257,6 +258,7 @@ typedef long long mstime_t; /* millisecond time type. */
 #define REDIS_PRE_PSYNC (1<<16)   /* Instance don't understand PSYNC. */
 #define REDIS_READONLY (1<<17)    /* Cluster client is in read-only state. */
 #define REDIS_PUBSUB (1<<18)      /* Client is in Pub/Sub mode. */
+#define REDIS_PREVENT_PROP (1<<19)  /* Don't propagate to AOF / Slaves. */
 
 /* Client block type (btype field in client structure)
  * if REDIS_BLOCKED flag is set. */
@@ -542,8 +544,8 @@ typedef struct redisClient {
     int multibulklen;       /* number of multi bulk arguments left to read */
     long bulklen;           /* length of bulk argument in multi bulk request */
     list *reply;
-    unsigned long reply_bytes; /* Tot bytes of objects in reply list */
-    int sentlen;            /* Amount of bytes already sent in the current
+    unsigned long long reply_bytes; /* Tot bytes of objects in reply list */
+    size_t sentlen;         /* Amount of bytes already sent in the current
                                buffer or object being sent. */
     time_t ctime;           /* Client creation time */
     time_t lastinteraction; /* time of the last interaction, used for timeout */
@@ -553,8 +555,8 @@ typedef struct redisClient {
     int replstate;          /* replication state if this is a slave */
     int repl_put_online_on_ack; /* Install slave write handler on ACK. */
     int repldbfd;           /* replication DB file descriptor */
-    off_t repldboff;        /* replication DB file offset */
-    off_t repldbsize;       /* replication DB file size */
+    off_t repldboff;       /* replication DB file offset */
+    off_t repldbsize;      /* replication DB file size */
     sds replpreamble;       /* replication DB preamble. */
     long long reploff;      /* replication offset if this is our master */
     long long repl_ack_off; /* replication ack offset, if this is a slave */
@@ -569,6 +571,7 @@ typedef struct redisClient {
     dict *pubsub_channels;  /* channels a client is interested in (SUBSCRIBE) */
     list *pubsub_patterns;  /* patterns a client is interested in (SUBSCRIBE) */
     sds peerid;             /* Cached peer ID. */
+    int repl_eof_supported; /* slave supports EOF based, diskless replication */
 
     /* Response buffer */
     int bufpos;
@@ -708,7 +711,7 @@ struct redisServer {
     off_t loading_process_events_interval_bytes;
     /* Fast pointers to often looked up command */
     struct redisCommand *delCommand, *multiCommand, *lpushCommand, *lpopCommand,
-                        *rpopCommand;
+                        *rpopCommand, *sremCommand;
     /* Fields used only for stats */
     time_t stat_starttime;          /* Server start time */
     long long stat_numcommands;     /* Number of processed commands */
@@ -827,7 +830,8 @@ struct redisServer {
     int repl_min_slaves_to_write;   /* Min number of slaves to write. */
     int repl_min_slaves_max_lag;    /* Max lag of <count> slaves to write. */
     int repl_good_slaves_count;     /* Number of slaves with lag <= max_lag. */
-    int repl_diskless_sync;         /* Send RDB to slaves sockets directly. */
+    int repl_diskless_load;         /* Slave parse RDB directly from the socket. */
+    int repl_diskless_sync;         /* Master send RDB to slaves sockets directly. */
     int repl_diskless_sync_delay;   /* Delay to start a diskless repl BGSAVE. */
     /* Replication (slave) */
     char *masterauth;               /* AUTH with this password with master */
@@ -1052,14 +1056,14 @@ void acceptTcpHandler(aeEventLoop *el, int fd, void *privdata, int mask);
 void acceptUnixHandler(aeEventLoop *el, int fd, void *privdata, int mask);
 void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask);
 void addReplyBulk(redisClient *c, robj *obj);
-void addReplyBulkCString(redisClient *c, char *s);
-void addReplyBulkCBuffer(redisClient *c, void *p, size_t len);
+void addReplyBulkCString(redisClient *c, const char *s);
+void addReplyBulkCBuffer(redisClient *c, const void *p, size_t len);
 void addReplyBulkLongLong(redisClient *c, long long ll);
 void addReply(redisClient *c, robj *obj);
 void addReplySds(redisClient *c, sds s);
 void addReplyBulkSds(redisClient *c, sds s);
-void addReplyError(redisClient *c, char *err);
-void addReplyStatus(redisClient *c, char *status);
+void addReplyError(redisClient *c, const char *err);
+void addReplyStatus(redisClient *c, const char *status);
 void addReplyDouble(redisClient *c, double d);
 void addReplyLongLong(redisClient *c, long long ll);
 void addReplyMultiBulkLen(redisClient *c, long length);
@@ -1073,6 +1077,7 @@ sds catClientInfoString(sds s, redisClient *client);
 sds getAllClientsInfoString(void);
 void rewriteClientCommandVector(redisClient *c, int argc, ...);
 void rewriteClientCommandArgument(redisClient *c, int i, robj *newval);
+void replaceClientCommandVector(redisClient *c, int argc, robj **argv);
 unsigned long getClientOutputBufferMemoryUsage(redisClient *c);
 void freeClientsInAsyncFreeQueue(void);
 void asyncCloseClientOnOutputBufferLimitReached(redisClient *c);
@@ -1135,9 +1140,9 @@ void freeSetObject(robj *o);
 void freeZsetObject(robj *o);
 void freeHashObject(robj *o);
 robj *createObject(int type, void *ptr);
-robj *createStringObject(char *ptr, size_t len);
-robj *createRawStringObject(char *ptr, size_t len);
-robj *createEmbeddedStringObject(char *ptr, size_t len);
+robj *createStringObject(const char *ptr, size_t len);
+robj *createRawStringObject(const char *ptr, size_t len);
+robj *createEmbeddedStringObject(const char *ptr, size_t len);
 robj *dupStringObject(robj *o);
 int isObjectRepresentableAsLongLong(robj *o, long long *llongval);
 robj *tryObjectEncoding(robj *o);
@@ -1194,7 +1199,8 @@ long long replicationGetSlaveOffset(void);
 char *replicationGetSlaveName(redisClient *c);
 
 /* Generic persistence functions */
-void startLoading(FILE *fp);
+void startLoading(size_t size);
+void startLoadingFile(FILE *fp);
 void loadingProgress(off_t pos);
 void stopLoading(void);
 
@@ -1239,6 +1245,7 @@ void zzlNext(unsigned char *zl, unsigned char **eptr, unsigned char **sptr);
 void zzlPrev(unsigned char *zl, unsigned char **eptr, unsigned char **sptr);
 unsigned int zsetLength(robj *zobj);
 void zsetConvert(robj *zobj, int encoding);
+int zsetScore(robj *zobj, robj *member, double *score);
 unsigned long zslGetRank(zskiplist *zsl, double score, robj *o);
 
 /* Core functions */
@@ -1252,6 +1259,7 @@ void call(redisClient *c, int flags);
 void propagate(struct redisCommand *cmd, int dbid, robj **argv, int argc, int flags);
 void alsoPropagate(struct redisCommand *cmd, int dbid, robj **argv, int argc, int target);
 void forceCommandPropagation(redisClient *c, int flags);
+void preventCommandPropagation(redisClient *c);
 int prepareForShutdown();
 #ifdef __GNUC__
 void redisLog(int level, const char *fmt, ...)
@@ -1272,7 +1280,7 @@ void closeListeningSockets(int unlink_unix_socket);
 void updateCachedTime(void);
 void resetServerStats(void);
 unsigned int getLRUClock(void);
-char *maxmemoryToString(void);
+const char *maxmemoryToString(void);
 
 /* Set data type */
 robj *setTypeCreate(robj *value);
@@ -1325,7 +1333,7 @@ void loadServerConfig(char *filename, char *options);
 void appendServerSaveParams(time_t seconds, int changes);
 void resetServerSaveParams(void);
 struct rewriteConfigState; /* Forward declaration to export API. */
-void rewriteConfigRewriteLine(struct rewriteConfigState *state, char *option, sds line, int force);
+void rewriteConfigRewriteLine(struct rewriteConfigState *state, const char *option, sds line, int force);
 int rewriteConfig(char *path);
 
 /* db.c -- Keyspace access API */
@@ -1393,6 +1401,7 @@ void blockClient(redisClient *c, int btype);
 void unblockClient(redisClient *c);
 void replyToBlockedClientTimedOut(redisClient *c);
 int getTimeoutFromObjectOrReply(redisClient *c, robj *object, mstime_t *timeout, int unit);
+void disconnectAllBlockedClients(void);
 
 /* Git SHA1 */
 char *redisGitSHA1(void);
@@ -1514,6 +1523,7 @@ void hmsetCommand(redisClient *c);
 void hmgetCommand(redisClient *c);
 void hdelCommand(redisClient *c);
 void hlenCommand(redisClient *c);
+void hstrlenCommand(redisClient *c);
 void zremrangebyrankCommand(redisClient *c);
 void zunionstoreCommand(redisClient *c);
 void zinterstoreCommand(redisClient *c);
@@ -1552,6 +1562,14 @@ void bitcountCommand(redisClient *c);
 void bitposCommand(redisClient *c);
 void replconfCommand(redisClient *c);
 void waitCommand(redisClient *c);
+void geoencodeCommand(redisClient *c);
+void geodecodeCommand(redisClient *c);
+void georadiusByMemberCommand(redisClient *c);
+void georadiusCommand(redisClient *c);
+void geoaddCommand(redisClient *c);
+void geohashCommand(redisClient *c);
+void geoposCommand(redisClient *c);
+void geodistCommand(redisClient *c);
 void pfselftestCommand(redisClient *c);
 void pfaddCommand(redisClient *c);
 void pfcountCommand(redisClient *c);

src/replication.c

@@ -441,14 +441,19 @@ need_full_resync:
  * socket target depending on the configuration, and making sure that
  * the script cache is flushed before to start.
  *
- * Returns REDIS_OK on success or REDIS_ERR otherwise. */
-int startBgsaveForReplication(void) {
+ * Returns REDIS_OK on success or REDIS_ERR otherwise.
+ *
+ * The caller should pass '1' as the function argument if all the slaves
+ * currently waiting for a BGSAVE all claimed to support the EOF-style
+ * streaming format for RDB transfer. Otherwise it should be '0'. */
+int startBgsaveForReplication(int all_slaves_supprot_eof) {
     int retval;
+    int use_eof = all_slaves_support_eof && server.repl_diskless_sync;
 
     redisLog(REDIS_NOTICE,"Starting BGSAVE for SYNC with target: %s",
-        server.repl_diskless_sync ? "slaves sockets" : "disk");
+        use_eof ? "slaves sockets" : "disk");
 
-    if (server.repl_diskless_sync)
+    if (use_eof)
         retval = rdbSaveToSlavesSockets();
     else
         retval = rdbSaveBackground(server.rdb_filename);
@@ -553,7 +558,7 @@ void syncCommand(redisClient *c) {
         c->replstate = REDIS_REPL_WAIT_BGSAVE_START;
         redisLog(REDIS_NOTICE,"Waiting for next BGSAVE for SYNC");
     } else {
-        if (server.repl_diskless_sync) {
+        if (server.repl_diskless_sync && c->repl_eof_supported) {
             /* Diskless replication RDB child is created inside
              * replicationCron() since we want to delay its start a
              * few seconds to wait for more slaves to arrive. */
@@ -562,7 +567,7 @@ void syncCommand(redisClient *c) {
                 redisLog(REDIS_NOTICE,"Delay next BGSAVE for SYNC");
         } else {
             /* Ok we don't have a BGSAVE in progress, let's start one. */
-            if (startBgsaveForReplication() != REDIS_OK) {
+            if (startBgsaveForReplication(0) != REDIS_OK) {
                 redisLog(REDIS_NOTICE,"Replication failed, can't BGSAVE");
                 addReplyError(c,"Unable to perform background save");
                 return;
@@ -613,6 +618,8 @@ void replconfCommand(redisClient *c) {
                     &port,NULL) != REDIS_OK))
                 return;
             c->slave_listening_port = port;
+        } else if (!strcasecmp(c->argv[j]->ptr,"eof-supported")) {
+            c->repl_eof_supported = 1;
         } else if (!strcasecmp(c->argv[j]->ptr,"ack")) {
             /* REPLCONF ACK is used by slave to inform the master the amount
              * of replication stream that it processed so far. It is an
@@ -652,7 +659,8 @@ void replconfCommand(redisClient *c) {
  *
  * It does a few things:
  *
- * 1) Put the slave in ONLINE state.
+ * 1) Put the slave in ONLINE state (useless when the function is called
+ *    because state is already ONLINE but repl_put_online_on_ack is true).
  * 2) Make sure the writable event is re-installed, since calling the SYNC
  *    command disables it, so that we can accumulate output buffer without
  *    sending it to the slave.
@@ -660,7 +668,7 @@ void replconfCommand(redisClient *c) {
 void putSlaveOnline(redisClient *slave) {
     slave->replstate = REDIS_REPL_ONLINE;
     slave->repl_put_online_on_ack = 0;
-    slave->repl_ack_time = server.unixtime;
+    slave->repl_ack_time = server.unixtime; /* Prevent false timeout. */
     if (aeCreateFileEvent(server.el, slave->fd, AE_WRITABLE,
         sendReplyToClient, slave) == AE_ERR) {
         redisLog(REDIS_WARNING,"Unable to register writable event for slave bulk transfer: %s", strerror(errno));
@@ -744,7 +752,8 @@ void sendBulkToSlave(aeEventLoop *el, int fd, void *privdata, int mask) {
  * (if it had a disk or socket target). */
 void updateSlavesWaitingBgsave(int bgsaveerr, int type) {
     listNode *ln;
-    int startbgsave = 0;
+    int slaves_waiting_eof = 0;
+    int slaves_waiting_noneof = 0;
     listIter li;
 
     listRewind(server.slaves,&li);
@@ -752,7 +761,10 @@ void updateSlavesWaitingBgsave(int bgsaveerr, int type) {
         redisClient *slave = ln->value;
 
         if (slave->replstate == REDIS_REPL_WAIT_BGSAVE_START) {
-            startbgsave = 1;
+            if (slave->repl_eof_supported)
+                slaves_waiting_eof++;
+            else
+                slaves_waiting_noneof++;
             slave->replstate = REDIS_REPL_WAIT_BGSAVE_END;
         } else if (slave->replstate == REDIS_REPL_WAIT_BGSAVE_END) {
             struct redis_stat buf;
@@ -773,6 +785,7 @@ void updateSlavesWaitingBgsave(int bgsaveerr, int type) {
                  * is technically online now. */
                 slave->replstate = REDIS_REPL_ONLINE;
                 slave->repl_put_online_on_ack = 1;
+                slave->repl_ack_time = server.unixtime; /* Timeout otherwise. */
             } else {
                 if (bgsaveerr != REDIS_OK) {
                     freeClient(slave);
@@ -799,8 +812,10 @@ void updateSlavesWaitingBgsave(int bgsaveerr, int type) {
             }
         }
     }
-    if (startbgsave) {
-        if (startBgsaveForReplication() != REDIS_OK) {
+    if (slaves_waiting_eof || slaves_waiting_noneof) {
+        /* if there is at least one slave that doesn't support EOF, we'll
+         * start an non-eof replication */
+        if (startBgsaveForReplication(slaves_waiting_noneof==0) != REDIS_OK) {
             listIter li;
 
             listRewind(server.slaves,&li);
@@ -823,9 +838,13 @@ void replicationAbortSyncTransfer(void) {
 
     aeDeleteFileEvent(server.el,server.repl_transfer_s,AE_READABLE);
     close(server.repl_transfer_s);
-    close(server.repl_transfer_fd);
-    unlink(server.repl_transfer_tmpfile);
-    zfree(server.repl_transfer_tmpfile);
+    if (server.repl_transfer_fd!=-1) {
+        close(server.repl_transfer_fd);
+        unlink(server.repl_transfer_tmpfile);
+        zfree(server.repl_transfer_tmpfile);
+        server.repl_transfer_tmpfile = NULL;
+        server.repl_transfer_fd = -1;
+    }
     server.repl_state = REDIS_REPL_CONNECT;
 }
 
@@ -877,6 +896,7 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
     char buf[4096];
     ssize_t nread, readlen;
     off_t left;
+    sds initialCommandStream = NULL;
     REDIS_NOTUSED(el);
     REDIS_NOTUSED(privdata);
     REDIS_NOTUSED(mask);
@@ -931,129 +951,189 @@ void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
              * at the next call. */
             server.repl_transfer_size = 0;
             redisLog(REDIS_NOTICE,
-                "MASTER <-> SLAVE sync: receiving streamed RDB from master");
+                "MASTER <-> SLAVE sync: receiving streamed RDB from master with EOF %s",
+                server.repl_diskless_load? "to parser":"to disk");
         } else {
             usemark = 0;
             server.repl_transfer_size = strtol(buf+1,NULL,10);
             redisLog(REDIS_NOTICE,
-                "MASTER <-> SLAVE sync: receiving %lld bytes from master",
-                (long long) server.repl_transfer_size);
+                "MASTER <-> SLAVE sync: receiving %lld bytes from master %s",
+                (long long) server.repl_transfer_size,
+                server.repl_diskless_load? "to parser":"to disk");
         }
         return;
     }
 
-    /* Read bulk data */
-    if (usemark) {
-        readlen = sizeof(buf);
-    } else {
-        left = server.repl_transfer_size - server.repl_transfer_read;
-        readlen = (left < (signed)sizeof(buf)) ? left : (signed)sizeof(buf);
-    }
-
-    nread = read(fd,buf,readlen);
-    if (nread <= 0) {
-        redisLog(REDIS_WARNING,"I/O error trying to sync with MASTER: %s",
-            (nread == -1) ? strerror(errno) : "connection lost");
-        replicationAbortSyncTransfer();
-        return;
-    }
-    server.stat_net_input_bytes += nread;
-
-    /* When a mark is used, we want to detect EOF asap in order to avoid
-     * writing the EOF mark into the file... */
-    int eof_reached = 0;
-
-    if (usemark) {
-        /* Update the last bytes array, and check if it matches our delimiter.*/
-        if (nread >= REDIS_RUN_ID_SIZE) {
-            memcpy(lastbytes,buf+nread-REDIS_RUN_ID_SIZE,REDIS_RUN_ID_SIZE);
+    if (!server.repl_diskless_load) {
+        /* read the data from the socket, store it to a file and search for the EOF */
+        if (usemark) {
+            readlen = sizeof(buf);
         } else {
-            int rem = REDIS_RUN_ID_SIZE-nread;
-            memmove(lastbytes,lastbytes+nread,rem);
-            memcpy(lastbytes+rem,buf,nread);
+            left = server.repl_transfer_size - server.repl_transfer_read;
+            readlen = (left < (signed)sizeof(buf)) ? left : (signed)sizeof(buf);
         }
-        if (memcmp(lastbytes,eofmark,REDIS_RUN_ID_SIZE) == 0) eof_reached = 1;
-    }
 
-    server.repl_transfer_lastio = server.unixtime;
-    if (write(server.repl_transfer_fd,buf,nread) != nread) {
-        redisLog(REDIS_WARNING,"Write error or short write writing to the DB dump file needed for MASTER <-> SLAVE synchronization: %s", strerror(errno));
-        goto error;
-    }
-    server.repl_transfer_read += nread;
+        nread = read(fd,buf,readlen);
+        if (nread <= 0) {
+            redisLog(REDIS_WARNING,"I/O error trying to sync with MASTER: %s",
+                (nread == -1) ? strerror(errno) : "connection lost");
+            replicationAbortSyncTransfer();
+            return;
+        }
+        server.stat_net_input_bytes += nread;
 
-    /* Delete the last 40 bytes from the file if we reached EOF. */
-    if (usemark && eof_reached) {
-        if (ftruncate(server.repl_transfer_fd,
-            server.repl_transfer_read - REDIS_RUN_ID_SIZE) == -1)
-        {
-            redisLog(REDIS_WARNING,"Error truncating the RDB file received from the master for SYNC: %s", strerror(errno));
+        /* When a mark is used, we want to detect EOF asap in order to avoid
+         * writing the EOF mark into the file... */
+        int eof_reached = 0;
+
+        if (usemark) {
+            /* Update the last bytes array, and check if it matches our delimiter.*/
+            if (nread >= REDIS_RUN_ID_SIZE) {
+                memcpy(lastbytes,buf+nread-REDIS_RUN_ID_SIZE,REDIS_RUN_ID_SIZE);
+            } else {
+                int rem = REDIS_RUN_ID_SIZE-nread;
+                memmove(lastbytes,lastbytes+nread,rem);
+                memcpy(lastbytes+rem,buf,nread);
+            }
+            if (memcmp(lastbytes,eofmark,REDIS_RUN_ID_SIZE) == 0) eof_reached = 1;
+        }
+
+        server.repl_transfer_lastio = server.unixtime;
+        if (write(server.repl_transfer_fd,buf,nread) != nread) {
+            redisLog(REDIS_WARNING,"Write error or short write writing to the DB dump file needed for MASTER <-> SLAVE synchronization: %s", strerror(errno));
             goto error;
         }
+        server.repl_transfer_read += nread;
+
+        /* Delete the last 40 bytes from the file if we reached EOF. */
+        if (usemark && eof_reached) {
+            if (ftruncate(server.repl_transfer_fd,
+                server.repl_transfer_read - REDIS_RUN_ID_SIZE) == -1)
+            {
+                redisLog(REDIS_WARNING,"Error truncating the RDB file received from the master for SYNC: %s", strerror(errno));
+                goto error;
+            }
+        }
+
+        /* Sync data on disk from time to time, otherwise at the end of the transfer
+         * we may suffer a big delay as the memory buffers are copied into the
+         * actual disk. */
+        if (server.repl_transfer_read >=
+            server.repl_transfer_last_fsync_off + REPL_MAX_WRITTEN_BEFORE_FSYNC)
+        {
+            off_t sync_size = server.repl_transfer_read -
+                              server.repl_transfer_last_fsync_off;
+            rdb_fsync_range(server.repl_transfer_fd,
+                server.repl_transfer_last_fsync_off, sync_size);
+            server.repl_transfer_last_fsync_off += sync_size;
+        }
+
+        /* Check if the transfer is now complete */
+        if (!usemark) {
+            if (server.repl_transfer_read == server.repl_transfer_size)
+                eof_reached = 1;
+        }
+        if (!eof_reached)
+            return;
     }
 
-    /* Sync data on disk from time to time, otherwise at the end of the transfer
-     * we may suffer a big delay as the memory buffers are copied into the
-     * actual disk. */
-    if (server.repl_transfer_read >=
-        server.repl_transfer_last_fsync_off + REPL_MAX_WRITTEN_BEFORE_FSYNC)
-    {
-        off_t sync_size = server.repl_transfer_read -
-                          server.repl_transfer_last_fsync_off;
-        rdb_fsync_range(server.repl_transfer_fd,
-            server.repl_transfer_last_fsync_off, sync_size);
-        server.repl_transfer_last_fsync_off += sync_size;
-    }
+    /* We reach here when the slave is using diskless replication,
+     * or when we are done reading from the socket to the rdb file. */
+    redisLog(REDIS_NOTICE, "MASTER <-> SLAVE sync: Flushing old data");
+    signalFlushedDb(-1);
+    emptyDb(replicationEmptyDbCallback);
+    /* Before loading the DB into memory we need to delete the readable
+     * handler, otherwise it will get called recursively since
+     * rdbLoad() will call the event loop to process events from time to
+     * time for non blocking loading. */
+    aeDeleteFileEvent(server.el,server.repl_transfer_s,AE_READABLE);
+    redisLog(REDIS_NOTICE, "MASTER <-> SLAVE sync: Loading DB in memory");
+    if (server.repl_diskless_load) {
+        rio rdb;
+        rioInitWithFd(&rdb,fd);
+        /* Put the socket in blocking mode to simplify RDB transfer.
+         * We'll restore it when the RDB is received. */
+        anetBlock(NULL,fd);
+        anetRecvTimeout(NULL,fd,server.repl_timeout*1000);
 
-    /* Check if the transfer is now complete */
-    if (!usemark) {
-        if (server.repl_transfer_read == server.repl_transfer_size)
-            eof_reached = 1;
-    }
-
-    if (eof_reached) {
+        if (rdbLoadRio(&rdb, server.repl_transfer_size) != REDIS_OK) {
+            redisLog(REDIS_WARNING,"Failed trying to load the MASTER synchronization DB from disk");
+            replicationAbortSyncTransfer();
+            rioFreeFd(&rdb, NULL);
+            /* Remove the half-loaded data, and load back the old dataset
+             * if we have persistence turned on.
+             *
+             * TODO:
+             * 1) Actually allow rdbLoadRio() to don't fail with exit().
+             * 2) Load RDB / AOF.
+             *
+             * Right now this code path is not entered when the connection
+             * breaks between master and slave AFAIK.
+             */
+            emptyDb(NULL);
+            return;
+        }
+        if (usemark) {
+            if (!rioRead(&rdb,buf,REDIS_RUN_ID_SIZE) || memcmp(buf,eofmark,REDIS_RUN_ID_SIZE) != 0) {
+                redisLog(REDIS_WARNING,"replication stream EOF marker is broken");
+                replicationAbortSyncTransfer();
+                rioFreeFd(&rdb, NULL);
+                return;
+            }
+        }
+        /* get the unread command stream from the rio buffer */
+        rioFreeFd(&rdb, &initialCommandStream);
+        /* Restore the socket as non-blocking. */
+        anetNonBlock(NULL,fd);
+        anetRecvTimeout(NULL,fd,0);
+    } else {
         if (rename(server.repl_transfer_tmpfile,server.rdb_filename) == -1) {
             redisLog(REDIS_WARNING,"Failed trying to rename the temp DB into dump.rdb in MASTER <-> SLAVE synchronization: %s", strerror(errno));
             replicationAbortSyncTransfer();
             return;
         }
-        redisLog(REDIS_NOTICE, "MASTER <-> SLAVE sync: Flushing old data");
-        signalFlushedDb(-1);
-        emptyDb(replicationEmptyDbCallback);
-        /* Before loading the DB into memory we need to delete the readable
-         * handler, otherwise it will get called recursively since
-         * rdbLoad() will call the event loop to process events from time to
-         * time for non blocking loading. */
-        aeDeleteFileEvent(server.el,server.repl_transfer_s,AE_READABLE);
-        redisLog(REDIS_NOTICE, "MASTER <-> SLAVE sync: Loading DB in memory");
         if (rdbLoad(server.rdb_filename) != REDIS_OK) {
             redisLog(REDIS_WARNING,"Failed trying to load the MASTER synchronization DB from disk");
             replicationAbortSyncTransfer();
             return;
         }
-        /* Final setup of the connected slave <- master link */
         zfree(server.repl_transfer_tmpfile);
         close(server.repl_transfer_fd);
-        replicationCreateMasterClient(server.repl_transfer_s);
-        redisLog(REDIS_NOTICE, "MASTER <-> SLAVE sync: Finished with success");
-        /* Restart the AOF subsystem now that we finished the sync. This
-         * will trigger an AOF rewrite, and when done will start appending
-         * to the new file. */
-        if (server.aof_state != REDIS_AOF_OFF) {
-            int retry = 10;
-
-            stopAppendOnly();
-            while (retry-- && startAppendOnly() == REDIS_ERR) {
-                redisLog(REDIS_WARNING,"Failed enabling the AOF after successful master synchronization! Trying it again in one second.");
-                sleep(1);
-            }
-            if (!retry) {
-                redisLog(REDIS_WARNING,"FATAL: this slave instance finished the synchronization with its master, but the AOF can't be turned on. Exiting now.");
-                exit(1);
-            }
-        }
+        server.repl_transfer_fd = -1;
+        server.repl_transfer_tmpfile = NULL;
+    }
+    /* Final setup of the connected slave <- master link */
+    replicationCreateMasterClient(server.repl_transfer_s);
+    /* feed the initial command stream into the client input buffer */
+    if (initialCommandStream) {
+        int nread = sdslen(initialCommandStream);
+        server.current_client = server.master;
+        server.master->querybuf = sdscatsds(server.master->querybuf, initialCommandStream);
+        server.master->reploff += nread;
+        server.master->lastinteraction = server.unixtime;
+        server.stat_net_input_bytes += nread;
+        processInputBuffer(server.master);
+        server.current_client = NULL;
+        sdsfree(initialCommandStream);
     }
 
+    redisLog(REDIS_NOTICE, "MASTER <-> SLAVE sync: Finished with success");
+    /* Restart the AOF subsystem now that we finished the sync. This
+     * will trigger an AOF rewrite, and when done will start appending
+     * to the new file. */
+    if (server.aof_state != REDIS_AOF_OFF) {
+        int retry = 10;
+
+        stopAppendOnly();
+        while (retry-- && startAppendOnly() == REDIS_ERR) {
+            redisLog(REDIS_WARNING,"Failed enabling the AOF after successful master synchronization! Trying it again in one second.");
+            sleep(1);
+        }
+        if (!retry) {
+            redisLog(REDIS_WARNING,"FATAL: this slave instance finished the synchronization with its master, but the AOF can't be turned on. Exiting now.");
+            exit(1);
+        }
+    }
     return;
 
 error:
@@ -1317,6 +1397,18 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
         sdsfree(err);
     }
 
+    /* Inform the master that this slave supports EOF marker of diskless-sync */
+    {
+        err = sendSynchronousCommand(fd,"REPLCONF","eof-supported","yes",
+                                         NULL);
+        /* Ignore the error if any, not all the Redis versions support
+         * REPLCONF eof-supported. */
+        if (err[0] == '-') {
+            redisLog(REDIS_NOTICE,"(Non critical) Master does not understand REPLCONF eof-supported: %s", err);
+        }
+        sdsfree(err);
+    }
+
     /* Try a partial resynchonization. If we don't have a cached master
      * slaveTryPartialResynchronization() will at least try to use PSYNC
      * to start a full resynchronization so that we get the master run id
@@ -1341,16 +1433,20 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
     }
 
     /* Prepare a suitable temp file for bulk transfer */
-    while(maxtries--) {
-        snprintf(tmpfile,256,
-            "temp-%d.%ld.rdb",(int)server.unixtime,(long int)getpid());
-        dfd = open(tmpfile,O_CREAT|O_WRONLY|O_EXCL,0644);
-        if (dfd != -1) break;
-        sleep(1);
-    }
-    if (dfd == -1) {
-        redisLog(REDIS_WARNING,"Opening the temp file needed for MASTER <-> SLAVE synchronization: %s",strerror(errno));
-        goto error;
+    if (!server.repl_diskless_load) {
+        while(maxtries--) {
+            snprintf(tmpfile,256,
+                "temp-%d.%ld.rdb",(int)server.unixtime,(long int)getpid());
+            dfd = open(tmpfile,O_CREAT|O_WRONLY|O_EXCL,0644);
+            if (dfd != -1) break;
+            sleep(1);
+        }
+        if (dfd == -1) {
+            redisLog(REDIS_WARNING,"Opening the temp file needed for MASTER <-> SLAVE synchronization: %s",strerror(errno));
+            goto error;
+        }
+        server.repl_transfer_tmpfile = zstrdup(tmpfile);
+        server.repl_transfer_fd = dfd;
     }
 
     /* Setup the non blocking download of the bulk file. */
@@ -1367,13 +1463,17 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
     server.repl_transfer_size = -1;
     server.repl_transfer_read = 0;
     server.repl_transfer_last_fsync_off = 0;
-    server.repl_transfer_fd = dfd;
     server.repl_transfer_lastio = server.unixtime;
-    server.repl_transfer_tmpfile = zstrdup(tmpfile);
     return;
 
 error:
     close(fd);
+    if (server.repl_transfer_fd != -1)
+        close(server.repl_transfer_fd);
+    if (server.repl_transfer_tmpfile)
+        zfree(server.repl_transfer_tmpfile);
+    server.repl_transfer_tmpfile = NULL;
+    server.repl_transfer_fd = -1;
     server.repl_transfer_s = -1;
     server.repl_state = REDIS_REPL_CONNECT;
     return;
@@ -1382,7 +1482,7 @@ error:
 int connectWithMaster(void) {
     int fd;
 
-    fd = anetTcpNonBlockBindConnect(NULL,
+    fd = anetTcpNonBlockBestEffortBindConnect(NULL,
         server.masterhost,server.masterport,REDIS_BIND_ADDR);
     if (fd == -1) {
         redisLog(REDIS_WARNING,"Unable to connect to MASTER: %s",
@@ -1444,6 +1544,7 @@ void replicationSetMaster(char *ip, int port) {
     server.masterhost = sdsnew(ip);
     server.masterport = port;
     if (server.master) freeClient(server.master);
+    disconnectAllBlockedClients(); /* Clients blocked in master, now slave. */
     disconnectSlaves(); /* Force our slaves to resync with us as well. */
     replicationDiscardCachedMaster(); /* Don't try a PSYNC. */
     freeReplicationBacklog(); /* Don't allow our chained slaves to PSYNC. */
@@ -2070,7 +2171,8 @@ void replicationCron(void) {
      * slaves in WAIT_BGSAVE_START state. */
     if (server.rdb_child_pid == -1 && server.aof_child_pid == -1) {
         time_t idle, max_idle = 0;
-        int slaves_waiting = 0;
+        int slaves_waiting_eof = 0;
+        int slaves_waiting_noneof = 0;
         listNode *ln;
         listIter li;
 
@@ -2080,14 +2182,19 @@ void replicationCron(void) {
             if (slave->replstate == REDIS_REPL_WAIT_BGSAVE_START) {
                 idle = server.unixtime - slave->lastinteraction;
                 if (idle > max_idle) max_idle = idle;
-                slaves_waiting++;
+                if (slave->repl_eof_supported)
+                    slaves_waiting_eof++;
+                else
+                    slaves_waiting_noneof++;
             }
         }
 
-        if (slaves_waiting && max_idle > server.repl_diskless_sync_delay) {
+        if ((slaves_waiting_eof || slaves_waiting_noneof) && max_idle > server.repl_diskless_sync_delay) {
             /* Start a BGSAVE. Usually with socket target, or with disk target
-             * if there was a recent socket -> disk config change. */
-            if (startBgsaveForReplication() == REDIS_OK) {
+             * if there was a recent socket -> disk config change.
+             * if there is at least one slave that doesn't support EOF, we'll
+             * start an non-eof replication */
+            if (startBgsaveForReplication(slaves_waiting_noneof==0) == REDIS_OK){
                 /* It started! We need to change the state of slaves
                  * from WAIT_BGSAVE_START to WAIT_BGSAVE_END in case
                  * the current target is disk. Otherwise it was already done

src/rio.c

@@ -157,13 +157,106 @@ void rioInitWithFile(rio *r, FILE *fp) {
     r->io.file.autosync = 0;
 }
 
+/* ------------------- File descriptor implementation ------------------- */
+
+static size_t rioFdWrite(rio *r, const void *buf, size_t len) {
+    REDIS_NOTUSED(r);
+    REDIS_NOTUSED(buf);
+    REDIS_NOTUSED(len);
+    return 0; /* Error, this target does not yet support writing. */
+}
+
+/* Returns 1 or 0 for success/failure. */
+static size_t rioFdRead(rio *r, void *buf, size_t len) {
+    size_t avail = sdslen(r->io.fd.buf)-r->io.fd.pos;
+
+    /* if the buffer is too small for the entire request: realloc */
+    if (sdslen(r->io.fd.buf) + sdsavail(r->io.fd.buf) < len)
+        r->io.fd.buf = sdsMakeRoomFor(r->io.fd.buf, len - sdslen(r->io.fd.buf));
+
+    /* if the remaining unused buffer is not large enough: memmove so that we can read the rest */
+    if (len > avail && sdsavail(r->io.fd.buf) < len - avail) {
+        sdsrange(r->io.fd.buf, r->io.fd.pos, -1);
+        r->io.fd.pos = 0;
+    }
+
+    /* if we don't already have all the data in the sds, read more */
+    while (len > sdslen(r->io.fd.buf) - r->io.fd.pos) {
+        size_t toread = len - (sdslen(r->io.fd.buf) - r->io.fd.pos);
+        /* read either what's missing, or REDIS_IOBUF_LEN, the bigger of the two */
+        if (toread < REDIS_IOBUF_LEN)
+            toread = REDIS_IOBUF_LEN;
+        if (toread > sdsavail(r->io.fd.buf))
+            toread = sdsavail(r->io.fd.buf);
+        int retval = read(r->io.fd.fd, (char*)r->io.fd.buf + sdslen(r->io.fd.buf), toread);
+        if (retval <= 0) {
+            if (errno == EWOULDBLOCK) errno = ETIMEDOUT;
+            return 0;
+        }
+        sdsIncrLen(r->io.fd.buf, retval);
+    }
+
+    memcpy(buf, (char*)r->io.fd.buf + r->io.fd.pos, len);
+    r->io.fd.pos += len;
+    return len;
+}
+
+/* Returns read/write position in file. */
+static off_t rioFdTell(rio *r) {
+    off_t pos = lseek(r->io.fd.fd, 0, SEEK_CUR);
+    return pos - sdslen(r->io.fd.buf) + r->io.fd.pos;
+}
+
+/* Flushes any buffer to target device if applicable. Returns 1 on success
+ * and 0 on failures. */
+static int rioFdFlush(rio *r) {
+    /* Our flush is implemented by the write method, that recognizes a
+     * buffer set to NULL with a count of zero as a flush request. */
+    return rioFdWrite(r,NULL,0);
+}
+
+static const rio rioFdIO = {
+    rioFdRead,
+    rioFdWrite,
+    rioFdTell,
+    rioFdFlush,
+    NULL,           /* update_checksum */
+    0,              /* current checksum */
+    0,              /* bytes read or written */
+    0,              /* read/write chunk size */
+    { { NULL, 0 } } /* union for io-specific vars */
+};
+
+void rioInitWithFd(rio *r, int fd) {
+    *r = rioFdIO;
+    r->io.fd.fd = fd;
+    r->io.fd.pos = 0;
+    r->io.fd.buf = sdsnewlen(NULL, REDIS_IOBUF_LEN);
+    sdsclear(r->io.fd.buf);
+}
+
+/* release the rio stream.
+ * optionally returns the unread buffered data. */
+void rioFreeFd(rio *r, sds* out_remainingBufferedData) {
+    if(out_remainingBufferedData && (size_t)r->io.fd.pos < sdslen(r->io.fd.buf)) {
+        if (r->io.fd.pos > 0)
+            sdsrange(r->io.fd.buf, r->io.fd.pos, -1);
+        *out_remainingBufferedData = r->io.fd.buf;
+    } else {
+        sdsfree(r->io.fd.buf);
+        if (out_remainingBufferedData)
+            *out_remainingBufferedData = NULL;
+    }
+    r->io.fd.buf = NULL;
+}
+
 /* ------------------- File descriptors set implementation ------------------- */
 
 /* Returns 1 or 0 for success/failure.
  * The function returns success as long as we are able to correctly write
  * to at least one file descriptor.
  *
- * When buf is NULL adn len is 0, the function performs a flush operation
+ * When buf is NULL and len is 0, the function performs a flush operation
  * if there is some pending buffer, so this function is also used in order
  * to implement rioFdsetFlush(). */
 static size_t rioFdsetWrite(rio *r, const void *buf, size_t len) {
@@ -176,7 +269,7 @@ static size_t rioFdsetWrite(rio *r, const void *buf, size_t len) {
      * a given size, we actually write to the sockets. */
     if (len) {
         r->io.fdset.buf = sdscatlen(r->io.fdset.buf,buf,len);
-        len = 0; /* Prevent entering the while belove if we don't flush. */
+        len = 0; /* Prevent entering the while below if we don't flush. */
         if (sdslen(r->io.fdset.buf) > REDIS_IOBUF_LEN) doflush = 1;
     }
 
@@ -276,6 +369,7 @@ void rioInitWithFdset(rio *r, int *fds, int numfds) {
     r->io.fdset.buf = sdsempty();
 }
 
+/* release the rio stream. */
 void rioFreeFdset(rio *r) {
     zfree(r->io.fdset.fds);
     zfree(r->io.fdset.state);

src/rio.h

@@ -73,6 +73,12 @@ struct _rio {
             off_t buffered; /* Bytes written since last fsync. */
             off_t autosync; /* fsync after 'autosync' bytes written. */
         } file;
+        /* file descriptor */
+        struct {
+            int fd;       /* File descriptor. */
+            off_t pos;
+            sds buf;
+        } fd;
         /* Multiple FDs target (used to write to N sockets). */
         struct {
             int *fds;       /* File descriptors. */
@@ -126,8 +132,12 @@ static inline int rioFlush(rio *r) {
 
 void rioInitWithFile(rio *r, FILE *fp);
 void rioInitWithBuffer(rio *r, sds s);
+void rioInitWithFd(rio *r, int fd);
 void rioInitWithFdset(rio *r, int *fds, int numfds);
 
+void rioFreeFdset(rio *r);
+void rioFreeFd(rio *r, sds* out_remainingBufferedData);
+
 size_t rioWriteBulkCount(rio *r, char prefix, int count);
 size_t rioWriteBulkString(rio *r, const char *buf, size_t len);
 size_t rioWriteBulkLongLong(rio *r, long long l);

src/scripting.c

@@ -357,8 +357,9 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
     if (cmd->flags & REDIS_CMD_WRITE) server.lua_write_dirty = 1;
 
     /* If this is a Redis Cluster node, we need to make sure Lua is not
-     * trying to access non-local keys. */
-    if (server.cluster_enabled) {
+     * trying to access non-local keys, with the exception of commands
+     * received from our master. */
+    if (server.cluster_enabled && !(server.lua_caller->flags & REDIS_MASTER)) {
         /* Duplicate relevant flags in the lua client. */
         c->flags &= ~(REDIS_READONLY|REDIS_ASKING);
         c->flags |= server.lua_caller->flags & (REDIS_READONLY|REDIS_ASKING);
@@ -611,11 +612,12 @@ void scriptingEnableGlobalsProtection(lua_State *lua) {
 
     /* strict.lua from: http://metalua.luaforge.net/src/lib/strict.lua.html.
      * Modified to be adapted to Redis. */
+    s[j++]="local dbg=debug\n";
     s[j++]="local mt = {}\n";
     s[j++]="setmetatable(_G, mt)\n";
     s[j++]="mt.__newindex = function (t, n, v)\n";
-    s[j++]="  if debug.getinfo(2) then\n";
-    s[j++]="    local w = debug.getinfo(2, \"S\").what\n";
+    s[j++]="  if dbg.getinfo(2) then\n";
+    s[j++]="    local w = dbg.getinfo(2, \"S\").what\n";
     s[j++]="    if w ~= \"main\" and w ~= \"C\" then\n";
     s[j++]="      error(\"Script attempted to create global variable '\"..tostring(n)..\"'\", 2)\n";
     s[j++]="    end\n";
@@ -623,11 +625,12 @@ void scriptingEnableGlobalsProtection(lua_State *lua) {
     s[j++]="  rawset(t, n, v)\n";
     s[j++]="end\n";
     s[j++]="mt.__index = function (t, n)\n";
-    s[j++]="  if debug.getinfo(2) and debug.getinfo(2, \"S\").what ~= \"C\" then\n";
+    s[j++]="  if dbg.getinfo(2) and dbg.getinfo(2, \"S\").what ~= \"C\" then\n";
     s[j++]="    error(\"Script attempted to access unexisting global variable '\"..tostring(n)..\"'\", 2)\n";
     s[j++]="  end\n";
     s[j++]="  return rawget(t, n)\n";
     s[j++]="end\n";
+    s[j++]="debug = nil\n";
     s[j++]=NULL;
 
     for (j = 0; s[j] != NULL; j++) code = sdscatlen(code,s[j],strlen(s[j]));
@@ -731,10 +734,11 @@ void scriptingInit(void) {
      * information about the caller, that's what makes sense from the point
      * of view of the user debugging a script. */
     {
-        char *errh_func =       "function __redis__err__handler(err)\n"
-                                "  local i = debug.getinfo(2,'nSl')\n"
+        char *errh_func =       "local dbg = debug\n"
+                                "function __redis__err__handler(err)\n"
+                                "  local i = dbg.getinfo(2,'nSl')\n"
                                 "  if i and i.what == 'C' then\n"
-                                "    i = debug.getinfo(3,'nSl')\n"
+                                "    i = dbg.getinfo(3,'nSl')\n"
                                 "  end\n"
                                 "  if i then\n"
                                 "    return i.source .. ':' .. i.currentline .. ': ' .. err\n"
@@ -876,7 +880,7 @@ void luaSetGlobalArray(lua_State *lua, char *var, robj **elev, int elec) {
 }
 
 /* Define a lua function with the specified function name and body.
- * The function name musts be a 2 characters long string, since all the
+ * The function name musts be a 42 characters long string, since all the
  * functions we defined in the Lua context are in the form:
  *
  *   f_<hex sha1 sum>

src/sds.c

@@ -71,7 +71,7 @@ sds sdsempty(void) {
     return sdsnewlen("",0);
 }
 
-/* Create a new sds string starting from a null termined C string. */
+/* Create a new sds string starting from a null terminated C string. */
 sds sdsnew(const char *init) {
     size_t initlen = (init == NULL) ? 0 : strlen(init);
     return sdsnewlen(init, initlen);
@@ -557,7 +557,7 @@ sds sdscatfmt(sds s, char const *fmt, ...) {
  * Example:
  *
  * s = sdsnew("AA...AA.a.aa.aHelloWorld     :::");
- * s = sdstrim(s,"A. :");
+ * s = sdstrim(s,"Aa. :");
  * printf("%s\n", s);
  *
  * Output will be just "Hello World".
@@ -1098,6 +1098,7 @@ int sdsTest(int argc, char *argv[]) {
             unsigned int oldfree;
 
             sdsfree(x);
+            sdsfree(y);
             x = sdsnew("0");
             sh = (void*) (x-(sizeof(struct sdshdr)));
             test_cond("sdsnew() free/len buffers", sh->len == 1 && sh->free == 0);
@@ -1110,6 +1111,8 @@ int sdsTest(int argc, char *argv[]) {
             test_cond("sdsIncrLen() -- content", x[0] == '0' && x[1] == '1');
             test_cond("sdsIncrLen() -- len", sh->len == 2);
             test_cond("sdsIncrLen() -- free", sh->free == oldfree-1);
+
+            sdsfree(x);
         }
     }
     test_report()

src/sha1.c

@@ -23,7 +23,7 @@ A million repetitions of "a"
 
 #include <stdio.h>
 #include <string.h>
-#include <sys/types.h>	/* for u_int*_t */
+#include <stdint.h>
 #include "solarisfixes.h"
 #include "sha1.h"
 #include "config.h"
@@ -53,12 +53,12 @@ A million repetitions of "a"
 
 /* Hash a single 512-bit block. This is the core of the algorithm. */
 
-void SHA1Transform(u_int32_t state[5], const unsigned char buffer[64])
+void SHA1Transform(uint32_t state[5], const unsigned char buffer[64])
 {
-    u_int32_t a, b, c, d, e;
+    uint32_t a, b, c, d, e;
     typedef union {
         unsigned char c[64];
-        u_int32_t l[16];
+        uint32_t l[16];
     } CHAR64LONG16;
 #ifdef SHA1HANDSOFF
     CHAR64LONG16 block[1];  /* use array to appear as a pointer */
@@ -128,9 +128,9 @@ void SHA1Init(SHA1_CTX* context)
 
 /* Run your data through this. */
 
-void SHA1Update(SHA1_CTX* context, const unsigned char* data, u_int32_t len)
+void SHA1Update(SHA1_CTX* context, const unsigned char* data, uint32_t len)
 {
-    u_int32_t i, j;
+    uint32_t i, j;
 
     j = context->count[0];
     if ((context->count[0] += len << 3) < j)
@@ -168,7 +168,7 @@ void SHA1Final(unsigned char digest[20], SHA1_CTX* context)
 
     for (i = 0; i < 2; i++)
        {
-        u_int32_t t = context->count[i];
+        uint32_t t = context->count[i];
         int j;
 
         for (j = 0; j < 4; t >>= 8, j++)

src/sha1.h

@@ -8,14 +8,14 @@ By Steve Reid <steve@edmweb.com>
 */
 
 typedef struct {
-    u_int32_t state[5];
-    u_int32_t count[2];
+    uint32_t state[5];
+    uint32_t count[2];
     unsigned char buffer[64];
 } SHA1_CTX;
 
-void SHA1Transform(u_int32_t state[5], const unsigned char buffer[64]);
+void SHA1Transform(uint32_t state[5], const unsigned char buffer[64]);
 void SHA1Init(SHA1_CTX* context);
-void SHA1Update(SHA1_CTX* context, const unsigned char* data, u_int32_t len);
+void SHA1Update(SHA1_CTX* context, const unsigned char* data, uint32_t len);
 void SHA1Final(unsigned char digest[20], SHA1_CTX* context);
 
 #ifdef REDIS_TEST

src/sort.c

@@ -209,7 +209,7 @@ void sortCommand(redisClient *c) {
     }
 
     /* Create a list of operations to perform for every sorted element.
-     * Operations can be GET/DEL/INCR/DECR */
+     * Operations can be GET */
     operations = listCreate();
     listSetFreeMethod(operations,zfree);
     j = 2; /* options start at argv[2] */

src/t_hash.c

@@ -130,7 +130,6 @@ robj *hashTypeGetObject(robj *o, robj *field) {
                 value = createStringObjectFromLongLong(vll);
             }
         }
-
     } else if (o->encoding == REDIS_ENCODING_HT) {
         robj *aux;
 
@@ -144,6 +143,29 @@ robj *hashTypeGetObject(robj *o, robj *field) {
     return value;
 }
 
+/* Higher level function using hashTypeGet*() to return the length of the
+ * object associated with the requested field, or 0 if the field does not
+ * exist. */
+size_t hashTypeGetValueLength(robj *o, robj *field) {
+    size_t len = 0;
+    if (o->encoding == REDIS_ENCODING_ZIPLIST) {
+        unsigned char *vstr = NULL;
+        unsigned int vlen = UINT_MAX;
+        long long vll = LLONG_MAX;
+
+        if (hashTypeGetFromZiplist(o, field, &vstr, &vlen, &vll) == 0)
+            len = vstr ? vlen : sdigits10(vll);
+    } else if (o->encoding == REDIS_ENCODING_HT) {
+        robj *aux;
+
+        if (hashTypeGetFromHashTable(o, field, &aux) == 0)
+            len = stringObjectLen(aux);
+    } else {
+        redisPanic("Unknown hash encoding");
+    }
+    return len;
+}
+
 /* Test if the specified field exists in the given hash. Returns 1 if the field
  * exists, and 0 when it doesn't. */
 int hashTypeExists(robj *o, robj *field) {
@@ -679,12 +701,21 @@ void hdelCommand(redisClient *c) {
 
 void hlenCommand(redisClient *c) {
     robj *o;
+
     if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
         checkType(c,o,REDIS_HASH)) return;
 
     addReplyLongLong(c,hashTypeLength(o));
 }
 
+void hstrlenCommand(redisClient *c) {
+    robj *o;
+
+    if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
+        checkType(c,o,REDIS_HASH)) return;
+    addReplyLongLong(c,hashTypeGetValueLength(o,c->argv[2]));
+}
+
 static void addHashIteratorCursorToReply(redisClient *c, hashTypeIterator *hi, int what) {
     if (hi->encoding == REDIS_ENCODING_ZIPLIST) {
         unsigned char *vstr = NULL;

src/t_set.c

@@ -45,6 +45,11 @@ robj *setTypeCreate(robj *value) {
     return createSetObject();
 }
 
+/* Add the specified value into a set. The function takes care of incrementing
+ * the reference count of the object if needed in order to retain a copy.
+ *
+ * If the value was already member of the set, nothing is done and 0 is
+ * returned, otherwise the new element is added and 1 is returned. */
 int setTypeAdd(robj *subject, robj *value) {
     long long llval;
     if (subject->encoding == REDIS_ENCODING_HT) {
@@ -139,7 +144,11 @@ void setTypeReleaseIterator(setTypeIterator *si) {
  * Since set elements can be internally be stored as redis objects or
  * simple arrays of integers, setTypeNext returns the encoding of the
  * set object you are iterating, and will populate the appropriate pointer
- * (eobj) or (llobj) accordingly.
+ * (objele) or (llele) accordingly.
+ *
+ * Note that both the objele and llele pointers should be passed and cannot
+ * be NULL since the function will try to defensively populate the non
+ * used field with values which are easy to trap if misused.
  *
  * When there are no longer elements -1 is returned.
  * Returned objects ref count is not incremented, so this function is
@@ -149,9 +158,13 @@ int setTypeNext(setTypeIterator *si, robj **objele, int64_t *llele) {
         dictEntry *de = dictNext(si->di);
         if (de == NULL) return -1;
         *objele = dictGetKey(de);
+        *llele = -123456789; /* Not needed. Defensive. */
     } else if (si->encoding == REDIS_ENCODING_INTSET) {
         if (!intsetGet(si->subject->ptr,si->ii++,llele))
             return -1;
+        *objele = NULL; /* Not needed. Defensive. */
+    } else {
+        redisPanic("Wrong set encoding in setTypeNext");
     }
     return si->encoding;
 }
@@ -192,6 +205,10 @@ robj *setTypeNextObject(setTypeIterator *si) {
  * field of the object and is used by the caller to check if the
  * int64_t pointer or the redis object pointer was populated.
  *
+ * Note that both the objele and llele pointers should be passed and cannot
+ * be NULL since the function will try to defensively populate the non
+ * used field with values which are easy to trap if misused.
+ *
  * When an object is returned (the set was a real set) the ref count
  * of the object is not incremented so this function can be considered
  * copy on write friendly. */
@@ -199,114 +216,16 @@ int setTypeRandomElement(robj *setobj, robj **objele, int64_t *llele) {
     if (setobj->encoding == REDIS_ENCODING_HT) {
         dictEntry *de = dictGetRandomKey(setobj->ptr);
         *objele = dictGetKey(de);
+        *llele = -123456789; /* Not needed. Defensive. */
     } else if (setobj->encoding == REDIS_ENCODING_INTSET) {
         *llele = intsetRandom(setobj->ptr);
+        *objele = NULL; /* Not needed. Defensive. */
     } else {
         redisPanic("Unknown set encoding");
     }
     return setobj->encoding;
 }
 
-/* Return a number of random elements from a non empty set.
- *
- * This is a version of setTypeRandomElement() that is modified in order to
- * return multiple entries, using dictGetRandomKeys() and intsetRandomMembers().
- *
- * The elements are stored into 'aux_set' which should be of a set type.
- *
- * The function returns the number of items stored into 'aux_set', that may
- * be less than 'count' if the hash table has less than 'count' elements
- * inside.
- *
- * Note that this function is not suitable when you need a good distribution
- * of the returned items, but only when you need to "sample" a given number
- * of continuous elements to run some kind of algorithm or to produce
- * statistics. However the function is much faster than setTypeRandomElement()
- * at producing N elements, and the elements are guaranteed to be non
- * repeating.
- */
-unsigned long setTypeRandomElements(robj *set, unsigned long count,
-                                    robj *aux_set) {
-    unsigned long set_size;
-    unsigned long elements_to_return = count;
-    unsigned long elements_copied = 0;
-    unsigned long current_element = 0;
-
-    /* Like all setType* functions, we assume good behavior on part of the
-     * caller, so no extra parameter checks are made. */
-
-    /* If the number of elements in the the set is less than the count
-     * requested, just return all of them. */
-    set_size = setTypeSize(set);
-    if (set_size < count) {
-        elements_to_return = set_size;
-    }
-
-    /* TODO: It is definitely faster adding items to the set by directly
-     * handling the Dict or intset inside it, avoiding the constant encoding
-     * checks inside setTypeAdd(). However, We don't want to touch the set
-     * internals in non setType* functions. So, we just call setTypeAdd()
-     * multiple times, but this isn't an optimal solution.
-     * Another option would be to create a bulk-add function:
-     * setTypeAddBulk(). */
-    if (set->encoding == REDIS_ENCODING_HT) {
-        /* Allocate result array */
-        dictEntry **random_elements =
-            zmalloc(sizeof(dictEntry*) * elements_to_return);
-
-        /* Get the random elements */
-        elements_copied =
-            dictGetRandomKeys(set->ptr, random_elements, elements_to_return);
-        redisAssert(elements_copied == elements_to_return);
-
-        /* Put them into the set */
-        for (current_element = 0; current_element < elements_copied;
-             current_element++) {
-
-            /* We get the key and duplicate it, as we know it is a string */
-            setTypeAdd(aux_set,
-                dictGetKey(random_elements[current_element]));
-        }
-
-        zfree(random_elements);
-
-    } else if (set->encoding == REDIS_ENCODING_INTSET) {
-        /* Allocate result array */
-        int64_t *random_elements =
-            zmalloc(sizeof(int64_t) * elements_to_return);
-        robj* element_as_str = NULL;
-
-        elements_copied =
-            intsetRandomMembers((intset*) set->ptr,
-                                random_elements,
-                                elements_to_return);
-
-        redisAssert(elements_copied == elements_to_return);
-
-        /* Put them into the set */
-        for (current_element = 0; current_element < elements_copied;
-             current_element++) {
-
-            element_as_str = createStringObjectFromLongLong(
-                    random_elements[current_element]);
-
-            /* Put the values in the set */
-            setTypeAdd(aux_set,
-                       element_as_str);
-
-            decrRefCount(element_as_str);
-        }
-
-        zfree(random_elements);
-    } else {
-        redisPanic("Unknown set encoding");
-    }
-
-    /* We have a set with random elements. Return the actual elements in
-       the aux_set. */
-    return elements_copied;
-}
-
 unsigned long setTypeSize(robj *subject) {
     if (subject->encoding == REDIS_ENCODING_HT) {
         return dictSize((dict*)subject->ptr);
@@ -335,7 +254,7 @@ void setTypeConvert(robj *setobj, int enc) {
 
         /* To add the elements we extract integers and create redis objects */
         si = setTypeInitIterator(setobj);
-        while (setTypeNext(si,NULL,&intele) != -1) {
+        while (setTypeNext(si,&element,&intele) != -1) {
             element = createStringObjectFromLongLong(intele);
             redisAssertWithInfo(NULL,element,
                                 dictAdd(d,element,NULL) == DICT_OK);
@@ -424,7 +343,7 @@ void smoveCommand(redisClient *c) {
 
     /* If srcset and dstset are equal, SMOVE is a no-op */
     if (srcset == dstset) {
-        addReply(c,shared.cone);
+        addReply(c,setTypeIsMember(srcset,ele) ? shared.cone : shared.czero);
         return;
     }
 
@@ -480,15 +399,18 @@ void scardCommand(redisClient *c) {
     addReplyLongLong(c,setTypeSize(o));
 }
 
-/* handle the "SPOP key <count>" variant. The normal version of the
+/* Handle the "SPOP key <count>" variant. The normal version of the
  * command is handled by the spopCommand() function itself. */
 
+/* How many times bigger should be the set compared to the remaining size
+ * for us to use the "create new set" strategy? Read later in the
+ * implementation for more info. */
+#define SPOP_MOVE_STRATEGY_MUL 5
+
 void spopWithCountCommand(redisClient *c) {
     long l;
     unsigned long count, size;
-    unsigned long elements_returned;
-    robj *set, *aux, *aux_set;
-    int64_t llele;
+    robj *set;
 
     /* Get the count argument */
     if (getLongFromObjectOrReply(c,c->argv[2],&l,NULL) != REDIS_OK) return;
@@ -511,18 +433,16 @@ void spopWithCountCommand(redisClient *c) {
         return;
     }
 
-    /* Get the size of the set. It is always > 0, as empty sets get
-     * deleted.  */
     size = setTypeSize(set);
 
     /* Generate an SPOP keyspace notification */
     notifyKeyspaceEvent(REDIS_NOTIFY_SET,"spop",c->argv[1],c->db->id);
+    server.dirty += count;
 
     /* CASE 1:
      * The number of requested elements is greater than or equal to
      * the number of elements inside the set: simply return the whole set. */
     if (count >= size) {
-
         /* We just return the entire set */
         sunionDiffGenericCommand(c,c->argv+1,1,NULL,REDIS_OP_UNION);
 
@@ -530,73 +450,110 @@ void spopWithCountCommand(redisClient *c) {
         dbDelete(c->db,c->argv[1]);
         notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,"del",c->argv[1],c->db->id);
 
-        /* Replicate/AOF this command as an SREM operation */
-        aux = createStringObject("DEL",3);
-        rewriteClientCommandVector(c,2,aux,c->argv[1]);
-        decrRefCount(aux);
-
+        /* Propagate this command as an DEL operation */
+        rewriteClientCommandVector(c,2,shared.del,c->argv[1]);
+        signalModifiedKey(c->db,c->argv[1]);
+        server.dirty++;
         return;
     }
 
-    /* CASE 2:
-     * The number of requested elements is less than the number
-     * of elements inside the set. */
+    /* Case 2 and 3 require to replicate SPOP as a set of SERM commands.
+     * Prepare our replication argument vector. Also send the array length
+     * which is common to both the code paths. */
+    robj *propargv[3];
+    propargv[0] = createStringObject("SREM",4);
+    propargv[1] = c->argv[1];
+    addReplyMultiBulkLen(c,count);
 
-    /* We need an auxiliary set. Optimistically, we create a set using an
-     * Intset internally. */
-    aux = createStringObjectFromLongLong(0);
-    aux_set = setTypeCreate(aux);
-    decrRefCount(aux);
+    /* Common iteration vars. */
+    robj *objele;
+    int encoding;
+    int64_t llele;
+    unsigned long remaining = size-count; /* Elements left after SPOP. */
 
-    /* Get the count requested of random elements from the set into our
-     * auxiliary set. */
-    elements_returned = setTypeRandomElements(set, count, aux_set);
-    redisAssert(elements_returned == count);
-
-    {
-        setTypeIterator *si;
-        robj *objele;
-        int element_encoding;
-
-        addReplyMultiBulkLen(c, elements_returned);
-
-        /* Replicate/AOF this command as an SREM operation */
-        aux = createStringObject("SREM",4);
-
-        si = setTypeInitIterator(aux_set);
-        while ((element_encoding = setTypeNext(si, &objele, &llele)) != -1) {
-            if (element_encoding == REDIS_ENCODING_HT) {
-
-                addReplyBulk(c, objele);
-
-                /* Replicate/AOF this command as an SREM commands */
-                rewriteClientCommandVector(c, 3, aux, c->argv[1], objele);
-                setTypeRemove(set, objele);
-            }
-            else if (element_encoding == REDIS_ENCODING_INTSET) {
-                /* TODO: setTypeRemove() forces us to convert all of the ints
-                 *       to string... isn't there a nicer way to do this? */
+    /* If we are here, the number of requested elements is less than the
+     * number of elements inside the set. Also we are sure that count < size.
+     * Use two different strategies.
+     *
+     * CASE 2: The number of elements to return is small compared to the
+     * set size. We can just extract random elements and return them to
+     * the set. */
+    if (remaining*SPOP_MOVE_STRATEGY_MUL > count) {
+        while(count--) {
+            encoding = setTypeRandomElement(set,&objele,&llele);
+            if (encoding == REDIS_ENCODING_INTSET) {
                 objele = createStringObjectFromLongLong(llele);
-                addReplyBulk(c, objele);
-
-                /* Replicate/AOF this command as an SREM commands */
-                rewriteClientCommandVector(c, 3, aux, c->argv[1], objele);
-                setTypeRemove(set, objele);
-
-                /* We created it, we kill it. */
-                decrRefCount(objele);
+            } else {
+                incrRefCount(objele);
             }
-            else {
-                redisPanic("Unknown set encoding");
+
+            /* Return the element to the client and remove from the set. */
+            addReplyBulk(c,objele);
+            setTypeRemove(set,objele);
+
+            /* Replicate/AOF this command as an SREM operation */
+            propargv[2] = objele;
+            alsoPropagate(server.sremCommand,c->db->id,propargv,3,
+                REDIS_PROPAGATE_AOF|REDIS_PROPAGATE_REPL);
+            decrRefCount(objele);
+        }
+    } else {
+    /* CASE 3: The number of elements to return is very big, approaching
+     * the size of the set itself. After some time extracting random elements
+     * from such a set becomes computationally expensive, so we use
+     * a different strategy, we extract random elements that we don't
+     * want to return (the elements that will remain part of the set),
+     * creating a new set as we do this (that will be stored as the original
+     * set). Then we return the elements left in the original set and
+     * release it. */
+        robj *newset = NULL;
+
+        /* Create a new set with just the remaining elements. */
+        while(remaining--) {
+            encoding = setTypeRandomElement(set,&objele,&llele);
+            if (encoding == REDIS_ENCODING_INTSET) {
+                objele = createStringObjectFromLongLong(llele);
+            } else {
+                incrRefCount(objele);
             }
+            if (!newset) newset = setTypeCreate(objele);
+            setTypeAdd(newset,objele);
+            setTypeRemove(set,objele);
+            decrRefCount(objele);
+        }
+
+        /* Assign the new set as the key value. */
+        incrRefCount(set); /* Protect the old set value. */
+        dbOverwrite(c->db,c->argv[1],newset);
+
+        /* Tranfer the old set to the client and release it. */
+        setTypeIterator *si;
+        si = setTypeInitIterator(set);
+        while((encoding = setTypeNext(si,&objele,&llele)) != -1) {
+            if (encoding == REDIS_ENCODING_INTSET) {
+                objele = createStringObjectFromLongLong(llele);
+            } else {
+                incrRefCount(objele);
+            }
+            addReplyBulk(c,objele);
+
+            /* Replicate/AOF this command as an SREM operation */
+            propargv[2] = objele;
+            alsoPropagate(server.sremCommand,c->db->id,propargv,3,
+                REDIS_PROPAGATE_AOF|REDIS_PROPAGATE_REPL);
+
+            decrRefCount(objele);
         }
         setTypeReleaseIterator(si);
-
-        decrRefCount(aux);
+        decrRefCount(set);
     }
 
-    /* Free the auxiliary set - we need it no more. */
-    decrRefCount(aux_set);
+    /* Don't propagate the command itself even if we incremented the
+     * dirty counter. We don't want to propagate an SPOP command since
+     * we propagated the command as a set of SREMs operations using
+     * the alsoPropagate() API. */
+    decrRefCount(propargv[0]);
+    preventCommandPropagation(c);
 }
 
 void spopCommand(redisClient *c) {

src/t_string.c

@@ -206,7 +206,7 @@ void setrangeCommand(redisClient *c) {
         if (checkStringLength(c,offset+sdslen(value)) != REDIS_OK)
             return;
 
-        o = createObject(REDIS_STRING,sdsempty());
+        o = createObject(REDIS_STRING,sdsnewlen(NULL, offset+sdslen(value)));
         dbAdd(c->db,c->argv[1],o);
     } else {
         size_t olen;

src/t_zset.c

@@ -213,7 +213,7 @@ static int zslValueGteMin(double value, zrangespec *spec) {
     return spec->minex ? (value > spec->min) : (value >= spec->min);
 }
 
-static int zslValueLteMax(double value, zrangespec *spec) {
+int zslValueLteMax(double value, zrangespec *spec) {
     return spec->maxex ? (value < spec->max) : (value <= spec->max);
 }
 
@@ -1166,40 +1166,109 @@ void zsetConvert(robj *zobj, int encoding) {
     }
 }
 
+/* Return (by reference) the score of the specified member of the sorted set
+ * storing it into *score. If the element does not exist REDIS_ERR is returned
+ * otherwise REDIS_OK is returned and *score is correctly populated.
+ * If 'zobj' or 'member' is NULL, REDIS_ERR is returned. */
+int zsetScore(robj *zobj, robj *member, double *score) {
+    if (!zobj || !member) return REDIS_ERR;
+
+    if (zobj->encoding == REDIS_ENCODING_ZIPLIST) {
+        if (zzlFind(zobj->ptr, member, score) == NULL) return REDIS_ERR;
+    } else if (zobj->encoding == REDIS_ENCODING_SKIPLIST) {
+        zset *zs = zobj->ptr;
+        dictEntry *de = dictFind(zs->dict, member);
+        if (de == NULL) return REDIS_ERR;
+        *score = *(double*)dictGetVal(de);
+    } else {
+        redisPanic("Unknown sorted set encoding");
+    }
+    return REDIS_OK;
+}
+
 /*-----------------------------------------------------------------------------
  * Sorted set commands
  *----------------------------------------------------------------------------*/
 
 /* This generic command implements both ZADD and ZINCRBY. */
-void zaddGenericCommand(redisClient *c, int incr) {
+#define ZADD_NONE 0
+#define ZADD_INCR (1<<0)    /* Increment the score instead of setting it. */
+#define ZADD_NX (1<<1)      /* Don't touch elements not already existing. */
+#define ZADD_XX (1<<2)      /* Only touch elements already exisitng. */
+#define ZADD_CH (1<<3)      /* Return num of elements added or updated. */
+void zaddGenericCommand(redisClient *c, int flags) {
     static char *nanerr = "resulting score is not a number (NaN)";
     robj *key = c->argv[1];
     robj *ele;
     robj *zobj;
     robj *curobj;
     double score = 0, *scores = NULL, curscore = 0.0;
-    int j, elements = (c->argc-2)/2;
-    int added = 0, updated = 0;
+    int j, elements;
+    int scoreidx = 0;
+    /* The following vars are used in order to track what the command actually
+     * did during the execution, to reply to the client and to trigger the
+     * notification of keyspace change. */
+    int added = 0;      /* Number of new elements added. */
+    int updated = 0;    /* Number of elements with updated score. */
+    int processed = 0;  /* Number of elements processed, may remain zero with
+                           options like XX. */
 
-    if (c->argc % 2) {
+    /* Parse options. At the end 'scoreidx' is set to the argument position
+     * of the score of the first score-element pair. */
+    scoreidx = 2;
+    while(scoreidx < c->argc) {
+        char *opt = c->argv[scoreidx]->ptr;
+        if (!strcasecmp(opt,"nx")) flags |= ZADD_NX;
+        else if (!strcasecmp(opt,"xx")) flags |= ZADD_XX;
+        else if (!strcasecmp(opt,"ch")) flags |= ZADD_CH;
+        else if (!strcasecmp(opt,"incr")) flags |= ZADD_INCR;
+        else break;
+        scoreidx++;
+    }
+
+    /* Turn options into simple to check vars. */
+    int incr = (flags & ZADD_INCR) != 0;
+    int nx = (flags & ZADD_NX) != 0;
+    int xx = (flags & ZADD_XX) != 0;
+    int ch = (flags & ZADD_CH) != 0;
+
+    /* After the options, we expect to have an even number of args, since
+     * we expect any number of score-element pairs. */
+    elements = c->argc-scoreidx;
+    if (elements % 2) {
         addReply(c,shared.syntaxerr);
         return;
     }
+    elements /= 2; /* Now this holds the number of score-element pairs. */
+
+    /* Check for incompatible options. */
+    if (nx && xx) {
+        addReplyError(c,
+            "XX and NX options at the same time are not compatible");
+        return;
+    }
+
+    if (incr && elements > 1) {
+        addReplyError(c,
+            "INCR option supports a single increment-element pair");
+        return;
+    }
 
     /* Start parsing all the scores, we need to emit any syntax error
      * before executing additions to the sorted set, as the command should
      * either execute fully or nothing at all. */
     scores = zmalloc(sizeof(double)*elements);
     for (j = 0; j < elements; j++) {
-        if (getDoubleFromObjectOrReply(c,c->argv[2+j*2],&scores[j],NULL)
+        if (getDoubleFromObjectOrReply(c,c->argv[scoreidx+j*2],&scores[j],NULL)
             != REDIS_OK) goto cleanup;
     }
 
     /* Lookup the key and create the sorted set if does not exist. */
     zobj = lookupKeyWrite(c->db,key);
     if (zobj == NULL) {
+        if (xx) goto reply_to_client; /* No key + XX option: nothing to do. */
         if (server.zset_max_ziplist_entries == 0 ||
-            server.zset_max_ziplist_value < sdslen(c->argv[3]->ptr))
+            server.zset_max_ziplist_value < sdslen(c->argv[scoreidx+1]->ptr))
         {
             zobj = createZsetObject();
         } else {
@@ -1220,8 +1289,9 @@ void zaddGenericCommand(redisClient *c, int incr) {
             unsigned char *eptr;
 
             /* Prefer non-encoded element when dealing with ziplists. */
-            ele = c->argv[3+j*2];
+            ele = c->argv[scoreidx+1+j*2];
             if ((eptr = zzlFind(zobj->ptr,ele,&curscore)) != NULL) {
+                if (nx) continue;
                 if (incr) {
                     score += curscore;
                     if (isnan(score)) {
@@ -1237,7 +1307,8 @@ void zaddGenericCommand(redisClient *c, int incr) {
                     server.dirty++;
                     updated++;
                 }
-            } else {
+                processed++;
+            } else if (!xx) {
                 /* Optimize: check if the element is too large or the list
                  * becomes too long *before* executing zzlInsert. */
                 zobj->ptr = zzlInsert(zobj->ptr,ele,score);
@@ -1247,15 +1318,18 @@ void zaddGenericCommand(redisClient *c, int incr) {
                     zsetConvert(zobj,REDIS_ENCODING_SKIPLIST);
                 server.dirty++;
                 added++;
+                processed++;
             }
         } else if (zobj->encoding == REDIS_ENCODING_SKIPLIST) {
             zset *zs = zobj->ptr;
             zskiplistNode *znode;
             dictEntry *de;
 
-            ele = c->argv[3+j*2] = tryObjectEncoding(c->argv[3+j*2]);
+            ele = c->argv[scoreidx+1+j*2] =
+                tryObjectEncoding(c->argv[scoreidx+1+j*2]);
             de = dictFind(zs->dict,ele);
             if (de != NULL) {
+                if (nx) continue;
                 curobj = dictGetKey(de);
                 curscore = *(double*)dictGetVal(de);
 
@@ -1280,22 +1354,30 @@ void zaddGenericCommand(redisClient *c, int incr) {
                     server.dirty++;
                     updated++;
                 }
-            } else {
+                processed++;
+            } else if (!xx) {
                 znode = zslInsert(zs->zsl,score,ele);
                 incrRefCount(ele); /* Inserted in skiplist. */
                 redisAssertWithInfo(c,NULL,dictAdd(zs->dict,ele,&znode->score) == DICT_OK);
                 incrRefCount(ele); /* Added to dictionary. */
                 server.dirty++;
                 added++;
+                processed++;
             }
         } else {
             redisPanic("Unknown sorted set encoding");
         }
     }
-    if (incr) /* ZINCRBY */
-        addReplyDouble(c,score);
-    else /* ZADD */
-        addReplyLongLong(c,added);
+
+reply_to_client:
+    if (incr) { /* ZINCRBY or INCR option. */
+        if (processed)
+            addReplyDouble(c,score);
+        else
+            addReply(c,shared.nullbulk);
+    } else { /* ZADD. */
+        addReplyLongLong(c,ch ? added+updated : added);
+    }
 
 cleanup:
     zfree(scores);
@@ -1307,11 +1389,11 @@ cleanup:
 }
 
 void zaddCommand(redisClient *c) {
-    zaddGenericCommand(c,0);
+    zaddGenericCommand(c,ZADD_NONE);
 }
 
 void zincrbyCommand(redisClient *c) {
-    zaddGenericCommand(c,1);
+    zaddGenericCommand(c,ZADD_INCR);
 }
 
 void zremCommand(redisClient *c) {
@@ -2753,25 +2835,10 @@ void zscoreCommand(redisClient *c) {
     if ((zobj = lookupKeyReadOrReply(c,key,shared.nullbulk)) == NULL ||
         checkType(c,zobj,REDIS_ZSET)) return;
 
-    if (zobj->encoding == REDIS_ENCODING_ZIPLIST) {
-        if (zzlFind(zobj->ptr,c->argv[2],&score) != NULL)
-            addReplyDouble(c,score);
-        else
-            addReply(c,shared.nullbulk);
-    } else if (zobj->encoding == REDIS_ENCODING_SKIPLIST) {
-        zset *zs = zobj->ptr;
-        dictEntry *de;
-
-        c->argv[2] = tryObjectEncoding(c->argv[2]);
-        de = dictFind(zs->dict,c->argv[2]);
-        if (de != NULL) {
-            score = *(double*)dictGetVal(de);
-            addReplyDouble(c,score);
-        } else {
-            addReply(c,shared.nullbulk);
-        }
+    if (zsetScore(zobj,c->argv[2],&score) == REDIS_ERR) {
+        addReply(c,shared.nullbulk);
     } else {
-        redisPanic("Unknown sorted set encoding");
+        addReplyDouble(c,score);
     }
 }
 
@@ -2819,7 +2886,7 @@ void zrankGenericCommand(redisClient *c, int reverse) {
         dictEntry *de;
         double score;
 
-        ele = c->argv[2] = tryObjectEncoding(c->argv[2]);
+        ele = c->argv[2];
         de = dictFind(zs->dict,ele);
         if (de != NULL) {
             score = *(double*)dictGetVal(de);

src/util.c

@@ -38,6 +38,7 @@
 #include <sys/time.h>
 #include <float.h>
 #include <stdint.h>
+#include <errno.h>
 
 #include "util.h"
 #include "sha1.h"
@@ -170,11 +171,12 @@ int stringmatch(const char *pattern, const char *string, int nocase) {
 }
 
 /* Convert a string representing an amount of memory into the number of
- * bytes, so for instance memtoll("1Gi") will return 1073741824 that is
+ * bytes, so for instance memtoll("1Gb") will return 1073741824 that is
  * (1024*1024*1024).
  *
  * On parsing error, if *err is not NULL, it's set to 1, otherwise it's
- * set to 0 */
+ * set to 0. On error the function return value is 0, regardless of the
+ * fact 'err' is NULL or not. */
 long long memtoll(const char *p, int *err) {
     const char *u;
     char buf[128];
@@ -183,6 +185,7 @@ long long memtoll(const char *p, int *err) {
     unsigned int digits;
 
     if (err) *err = 0;
+
     /* Search the first non digit character. */
     u = p;
     if (*u == '-') u++;
@@ -203,16 +206,26 @@ long long memtoll(const char *p, int *err) {
         mul = 1024L*1024*1024;
     } else {
         if (err) *err = 1;
-        mul = 1;
+        return 0;
     }
+
+    /* Copy the digits into a buffer, we'll use strtoll() to convert
+     * the digit (without the unit) into a number. */
     digits = u-p;
     if (digits >= sizeof(buf)) {
         if (err) *err = 1;
-        return LLONG_MAX;
+        return 0;
     }
     memcpy(buf,p,digits);
     buf[digits] = '\0';
-    val = strtoll(buf,NULL,10);
+
+    char *endptr;
+    errno = 0;
+    val = strtoll(buf,&endptr,10);
+    if ((val == 0 && errno == EINVAL) || *endptr != '\0') {
+        if (err) *err = 1;
+        return 0;
+    }
     return val*mul;
 }
 
@@ -238,6 +251,18 @@ uint32_t digits10(uint64_t v) {
     return 12 + digits10(v / 1000000000000UL);
 }
 
+/* Like digits10() but for signed values. */
+uint32_t sdigits10(int64_t v) {
+    if (v < 0) {
+        /* Abs value of LLONG_MIN requires special handling. */
+        uint64_t uv = (v != LLONG_MIN) ?
+                      (uint64_t)-v : ((uint64_t) LLONG_MAX)+1;
+        return digits10(uv)+1; /* +1 for the minus. */
+    } else {
+        return digits10(v);
+    }
+}
+
 /* Convert a long long into a string. Returns the number of
  * characters needed to represent the number.
  * If the buffer is not big enough to store the string, 0 is returned.

src/util.h

@@ -30,11 +30,14 @@
 #ifndef __REDIS_UTIL_H
 #define __REDIS_UTIL_H
 
+#include <stdint.h>
 #include "sds.h"
 
 int stringmatchlen(const char *p, int plen, const char *s, int slen, int nocase);
 int stringmatch(const char *p, const char *s, int nocase);
 long long memtoll(const char *p, int *err);
+uint32_t digits10(uint64_t v);
+uint32_t sdigits10(int64_t v);
 int ll2string(char *s, size_t len, long long value);
 int string2ll(const char *s, size_t slen, long long *value);
 int string2l(const char *s, size_t slen, long *value);

src/version.h

@@ -1 +1 @@
-#define REDIS_VERSION "2.9.999"
+#define REDIS_VERSION "3.1.999"

src/ziplist.h

@@ -28,6 +28,9 @@
  * POSSIBILITY OF SUCH DAMAGE.
  */
 
+#ifndef _ZIPLIST_H
+#define _ZIPLIST_H
+
 #define ZIPLIST_HEAD 0
 #define ZIPLIST_TAIL 1
 
@@ -49,3 +52,5 @@ size_t ziplistBlobLen(unsigned char *zl);
 #ifdef REDIS_TEST
 int ziplistTest(int argc, char *argv[]);
 #endif
+
+#endif /* _ZIPLIST_H */

tests/cluster/run.tcl

@@ -17,6 +17,7 @@ proc main {} {
     }
     run_tests
     cleanup
+    end_tests
 }
 
 if {[catch main e]} {

tests/cluster/tests/10-manual-failover.tcl

@@ -0,0 +1,192 @@
+# Check the manual failover
+
+source "../tests/includes/init-tests.tcl"
+
+test "Create a 5 nodes cluster" {
+    create_cluster 5 5
+}
+
+test "Cluster is up" {
+    assert_cluster_state ok
+}
+
+test "Cluster is writable" {
+    cluster_write_test 0
+}
+
+test "Instance #5 is a slave" {
+    assert {[RI 5 role] eq {slave}}
+}
+
+test "Instance #5 synced with the master" {
+    wait_for_condition 1000 50 {
+        [RI 5 master_link_status] eq {up}
+    } else {
+        fail "Instance #5 master link status is not up"
+    }
+}
+
+set current_epoch [CI 1 cluster_current_epoch]
+
+set numkeys 50000
+set numops 10000
+set cluster [redis_cluster 127.0.0.1:[get_instance_attrib redis 0 port]]
+catch {unset content}
+array set content {}
+
+test "Send CLUSTER FAILOVER to #5, during load" {
+    for {set j 0} {$j < $numops} {incr j} {
+        # Write random data to random list.
+        set listid [randomInt $numkeys]
+        set key "key:$listid"
+        set ele [randomValue]
+        # We write both with Lua scripts and with plain commands.
+        # This way we are able to stress Lua -> Redis command invocation
+        # as well, that has tests to prevent Lua to write into wrong
+        # hash slots.
+        if {$listid % 2} {
+            $cluster rpush $key $ele
+        } else {
+           $cluster eval {redis.call("rpush",KEYS[1],ARGV[1])} 1 $key $ele
+        }
+        lappend content($key) $ele
+
+        if {($j % 1000) == 0} {
+            puts -nonewline W; flush stdout
+        }
+
+        if {$j == $numops/2} {R 5 cluster failover}
+    }
+}
+
+test "Wait for failover" {
+    wait_for_condition 1000 50 {
+        [CI 1 cluster_current_epoch] > $current_epoch
+    } else {
+        fail "No failover detected"
+    }
+}
+
+test "Cluster should eventually be up again" {
+    assert_cluster_state ok
+}
+
+test "Cluster is writable" {
+    cluster_write_test 1
+}
+
+test "Instance #5 is now a master" {
+    assert {[RI 5 role] eq {master}}
+}
+
+test "Verify $numkeys keys for consistency with logical content" {
+    # Check that the Redis Cluster content matches our logical content.
+    foreach {key value} [array get content] {
+        assert {[$cluster lrange $key 0 -1] eq $value}
+    }
+}
+
+test "Instance #0 gets converted into a slave" {
+    wait_for_condition 1000 50 {
+        [RI 0 role] eq {slave}
+    } else {
+        fail "Old master was not converted into slave"
+    }
+}
+
+## Check that manual failover does not happen if we can't talk with the master.
+
+source "../tests/includes/init-tests.tcl"
+
+test "Create a 5 nodes cluster" {
+    create_cluster 5 5
+}
+
+test "Cluster is up" {
+    assert_cluster_state ok
+}
+
+test "Cluster is writable" {
+    cluster_write_test 0
+}
+
+test "Instance #5 is a slave" {
+    assert {[RI 5 role] eq {slave}}
+}
+
+test "Instance #5 synced with the master" {
+    wait_for_condition 1000 50 {
+        [RI 5 master_link_status] eq {up}
+    } else {
+        fail "Instance #5 master link status is not up"
+    }
+}
+
+test "Make instance #0 unreachable without killing it" {
+    R 0 deferred 1
+    R 0 DEBUG SLEEP 10
+}
+
+test "Send CLUSTER FAILOVER to instance #5" {
+    R 5 cluster failover
+}
+
+test "Instance #5 is still a slave after some time (no failover)" {
+    after 5000
+    assert {[RI 5 role] eq {master}}
+}
+
+test "Wait for instance #0 to return back alive" {
+    R 0 deferred 0
+    assert {[R 0 read] eq {OK}}
+}
+
+## Check with "force" failover happens anyway.
+
+source "../tests/includes/init-tests.tcl"
+
+test "Create a 5 nodes cluster" {
+    create_cluster 5 5
+}
+
+test "Cluster is up" {
+    assert_cluster_state ok
+}
+
+test "Cluster is writable" {
+    cluster_write_test 0
+}
+
+test "Instance #5 is a slave" {
+    assert {[RI 5 role] eq {slave}}
+}
+
+test "Instance #5 synced with the master" {
+    wait_for_condition 1000 50 {
+        [RI 5 master_link_status] eq {up}
+    } else {
+        fail "Instance #5 master link status is not up"
+    }
+}
+
+test "Make instance #0 unreachable without killing it" {
+    R 0 deferred 1
+    R 0 DEBUG SLEEP 10
+}
+
+test "Send CLUSTER FAILOVER to instance #5" {
+    R 5 cluster failover force
+}
+
+test "Instance #5 is a master after some time" {
+    wait_for_condition 1000 50 {
+        [RI 5 role] eq {master}
+    } else {
+        fail "Instance #5 is not a master after some time regardless of FORCE"
+    }
+}
+
+test "Wait for instance #0 to return back alive" {
+    R 0 deferred 0
+    assert {[R 0 read] eq {OK}}
+}

tests/cluster/tests/11-manual-takeover.tcl

@@ -0,0 +1,59 @@
+# Manual takeover test
+
+source "../tests/includes/init-tests.tcl"
+
+test "Create a 5 nodes cluster" {
+    create_cluster 5 5
+}
+
+test "Cluster is up" {
+    assert_cluster_state ok
+}
+
+test "Cluster is writable" {
+    cluster_write_test 0
+}
+
+test "Killing majority of master nodes" {
+    kill_instance redis 0
+    kill_instance redis 1
+    kill_instance redis 2
+}
+
+test "Cluster should eventually be down" {
+    assert_cluster_state fail
+}
+
+test "Use takeover to bring slaves back" {
+    R 5 cluster failover takeover
+    R 6 cluster failover takeover
+    R 7 cluster failover takeover
+}
+
+test "Cluster should eventually be up again" {
+    assert_cluster_state ok
+}
+
+test "Cluster is writable" {
+    cluster_write_test 4
+}
+
+test "Instance #5, #6, #7 are now masters" {
+    assert {[RI 5 role] eq {master}}
+    assert {[RI 6 role] eq {master}}
+    assert {[RI 7 role] eq {master}}
+}
+
+test "Restarting the previously killed master nodes" {
+    restart_instance redis 0
+    restart_instance redis 1
+    restart_instance redis 2
+}
+
+test "Instance #0, #1, #2 gets converted into a slaves" {
+    wait_for_condition 1000 50 {
+        [RI 0 role] eq {slave} && [RI 1 role] eq {slave} && [RI 2 role] eq {slave}
+    } else {
+        fail "Old masters not converted into slaves"
+    }
+}

tests/instances.tcl

@@ -19,6 +19,7 @@ set ::verbose 0
 set ::valgrind 0
 set ::pause_on_error 0
 set ::simulate_error 0
+set ::failed 0
 set ::sentinel_instances {}
 set ::redis_instances {}
 set ::sentinel_base_port 20000
@@ -231,6 +232,7 @@ proc test {descr code} {
     flush stdout
 
     if {[catch {set retval [uplevel 1 $code]} error]} {
+        incr ::failed
         if {[string match "assertion:*" $error]} {
             set msg [string range $error 10 end]
             puts [colorstr red $msg]
@@ -246,6 +248,7 @@ proc test {descr code} {
     }
 }
 
+# Execute all the units inside the 'tests' directory.
 proc run_tests {} {
     set tests [lsort [glob ../tests/*]]
     foreach test $tests {
@@ -258,6 +261,17 @@ proc run_tests {} {
     }
 }
 
+# Print a message and exists with 0 / 1 according to zero or more failures.
+proc end_tests {} {
+    if {$::failed == 0} {
+        puts "GOOD! No errors."
+        exit 0
+    } else {
+        puts "WARNING $::failed tests faield."
+        exit 1
+    }
+}
+
 # The "S" command is used to interact with the N-th Sentinel.
 # The general form is:
 #

tests/integration/aof.tcl

@@ -204,7 +204,7 @@ tags {"aof"} {
         }
     }
 
-    ## Test that SPOP with <count> (that modifies the client's argc/argv) is correctly free'd
+    ## Uses the alsoPropagate() API.
     create_aof {
         append_to_aof [formatCommand sadd set foo]
         append_to_aof [formatCommand sadd set bar]

tests/integration/logging.tcl

@@ -0,0 +1,24 @@
+set server_path [tmpdir server.log]
+set system_name [string tolower [exec uname -s]]
+
+if {$system_name eq {linux} || $system_name eq {darwin}} {
+    start_server [list overrides [list dir $server_path]] {
+        test "Server is able to generate a stack trace on selected systems" {
+            r config set watchdog-period 200
+            r debug sleep 1
+            set pattern "*debugCommand*"
+            set retry 10
+            while {$retry} {
+                set result [exec tail -100 < [srv 0 stdout]]
+                if {[string match $pattern $result]} {
+                    break
+                }
+                incr retry -1
+                after 1000
+            }
+            if {$retry == 0} {
+                error "assertion:expected stack trace not found into log file"
+            }
+        }
+    }
+}

tests/integration/replication-4.tcl

@@ -1,12 +1,3 @@
-proc start_bg_complex_data {host port db ops} {
-    set tclsh [info nameofexecutable]
-    exec $tclsh tests/helpers/bg_complex_data.tcl $host $port $db $ops &
-}
-
-proc stop_bg_complex_data {handle} {
-    catch {exec /bin/kill -9 $handle}
-}
-
 start_server {tags {"repl"}} {
     start_server {} {
 
@@ -132,5 +123,24 @@ start_server {tags {"repl"}} {
             }
             assert {[$master dbsize] > 0}
         }
+
+        test {Replication of SPOP command -- alsoPropagate() API} {
+            $master del myset
+            set size [expr 1+[randomInt 100]]
+            set content {}
+            for {set j 0} {$j < $size} {incr j} {
+                lappend content [randomValue]
+            }
+            $master sadd myset {*}$content
+
+            set count [randomInt 100]
+            set result [$master spop myset $count]
+
+            wait_for_condition 50 100 {
+                [$master debug digest] eq [$slave debug digest]
+            } else {
+                fail "SPOP replication inconsistency"
+            }
+        }
     }
 }

tests/integration/replication-psync.tcl

@@ -1,12 +1,3 @@
-proc start_bg_complex_data {host port db ops} {
-    set tclsh [info nameofexecutable]
-    exec $tclsh tests/helpers/bg_complex_data.tcl $host $port $db $ops &
-}
-
-proc stop_bg_complex_data {handle} {
-    catch {exec /bin/kill -9 $handle}
-}
-
 # Creates a master-slave pair and breaks the link continuously to force
 # partial resyncs attempts, all this while flooding the master with
 # write queries.

tests/integration/replication.tcl

@@ -1,10 +1,17 @@
 start_server {tags {"repl"}} {
+    set A [srv 0 client]
+    set A_host [srv 0 host]
+    set A_port [srv 0 port]
     start_server {} {
-        test {First server should have role slave after SLAVEOF} {
-            r -1 slaveof [srv 0 host] [srv 0 port]
+        set B [srv 0 client]
+        set B_host [srv 0 host]
+        set B_port [srv 0 port]
+
+        test {Set instance A as slave of B} {
+            $A slaveof $B_host $B_port
             wait_for_condition 50 100 {
-                [s -1 role] eq {slave} &&
-                [string match {*master_link_status:up*} [r -1 info replication]]
+                [lindex [$A role] 0] eq {slave} &&
+                [string match {*master_link_status:up*} [$A info replication]]
             } else {
                 fail "Can't turn the instance into a slave"
             }
@@ -15,9 +22,9 @@ start_server {tags {"repl"}} {
             $rd brpoplpush a b 5
             r lpush a foo
             wait_for_condition 50 100 {
-                [r debug digest] eq [r -1 debug digest]
+                [$A debug digest] eq [$B debug digest]
             } else {
-                fail "Master and slave have different digest: [r debug digest] VS [r -1 debug digest]"
+                fail "Master and slave have different digest: [$A debug digest] VS [$B debug digest]"
             }
         }
 
@@ -28,7 +35,36 @@ start_server {tags {"repl"}} {
             r lpush c 3
             $rd brpoplpush c d 5
             after 1000
-            assert_equal [r debug digest] [r -1 debug digest]
+            assert_equal [$A debug digest] [$B debug digest]
+        }
+
+        test {BLPOP followed by role change, issue #2473} {
+            set rd [redis_deferring_client]
+            $rd blpop foo 0 ; # Block while B is a master
+
+            # Turn B into master of A
+            $A slaveof no one
+            $B slaveof $A_host $A_port
+            wait_for_condition 50 100 {
+                [lindex [$B role] 0] eq {slave} &&
+                [string match {*master_link_status:up*} [$B info replication]]
+            } else {
+                fail "Can't turn the instance into a slave"
+            }
+
+            # Push elements into the "foo" list of the new slave.
+            # If the client is still attached to the instance, we'll get
+            # a desync between the two instances.
+            $A rpush foo a b c
+            after 100
+
+            wait_for_condition 50 100 {
+                [$A debug digest] eq [$B debug digest] &&
+                [$A lrange foo 0 -1] eq {a b c} &&
+                [$B lrange foo 0 -1] eq {a b c}
+            } else {
+                fail "Master and slave have different digest: [$A debug digest] VS [$B debug digest]"
+            }
         }
     }
 }
@@ -94,85 +130,91 @@ start_server {tags {"repl"}} {
     }
 }
 
-foreach dl {no yes} {
-    start_server {tags {"repl"}} {
-        set master [srv 0 client]
-        $master config set repl-diskless-sync $dl
-        set master_host [srv 0 host]
-        set master_port [srv 0 port]
-        set slaves {}
-        set load_handle0 [start_write_load $master_host $master_port 3]
-        set load_handle1 [start_write_load $master_host $master_port 5]
-        set load_handle2 [start_write_load $master_host $master_port 20]
-        set load_handle3 [start_write_load $master_host $master_port 8]
-        set load_handle4 [start_write_load $master_host $master_port 4]
-        start_server {} {
-            lappend slaves [srv 0 client]
+foreach mdl {no yes} {
+    foreach sdl {no yes} {
+        start_server {tags {"repl"}} {
+            set master [srv 0 client]
+            $master config set repl-diskless-sync $mdl
+            $master config set repl-diskless-sync-delay 1
+            set master_host [srv 0 host]
+            set master_port [srv 0 port]
+            set slaves {}
+            set load_handle0 [start_bg_complex_data $master_host $master_port 9 100000000]
+            set load_handle1 [start_bg_complex_data $master_host $master_port 11 100000000]
+            set load_handle2 [start_bg_complex_data $master_host $master_port 12 100000000]
+            set load_handle3 [start_write_load $master_host $master_port 8]
+            set load_handle4 [start_write_load $master_host $master_port 4]
             start_server {} {
                 lappend slaves [srv 0 client]
                 start_server {} {
                     lappend slaves [srv 0 client]
-                    test "Connect multiple slaves at the same time (issue #141), diskless=$dl" {
-                        # Send SALVEOF commands to slaves
-                        [lindex $slaves 0] slaveof $master_host $master_port
-                        [lindex $slaves 1] slaveof $master_host $master_port
-                        [lindex $slaves 2] slaveof $master_host $master_port
+                    start_server {} {
+                        lappend slaves [srv 0 client]
+                        test "Connect multiple slaves at the same time (issue #141), master diskless=$mdl, slave diskless=$sdl" {
+                            # Send SLAVEOF commands to slaves
+                            [lindex $slaves 0] config set repl-diskless-load $sdl
+                            [lindex $slaves 1] config set repl-diskless-load $sdl
+                            [lindex $slaves 2] config set repl-diskless-load $sdl
+                            [lindex $slaves 0] slaveof $master_host $master_port
+                            [lindex $slaves 1] slaveof $master_host $master_port
+                            [lindex $slaves 2] slaveof $master_host $master_port
 
-                        # Wait for all the three slaves to reach the "online"
-                        # state from the POV of the master.
-                        set retry 500
-                        while {$retry} {
-                            set info [r -3 info]
-                            if {[string match {*slave0:*state=online*slave1:*state=online*slave2:*state=online*} $info]} {
-                                break
-                            } else {
-                                incr retry -1
-                                after 100
+                            # Wait for all the three slaves to reach the "online"
+                            # state from the POV of the master.
+                            set retry 500
+                            while {$retry} {
+                                set info [r -3 info]
+                                if {[string match {*slave0:*state=online*slave1:*state=online*slave2:*state=online*} $info]} {
+                                    break
+                                } else {
+                                    incr retry -1
+                                    after 100
+                                }
+                            }
+                            if {$retry == 0} {
+                                error "assertion:Slaves not correctly synchronized"
                             }
-                        }
-                        if {$retry == 0} {
-                            error "assertion:Slaves not correctly synchronized"
-                        }
 
-                        # Wait that slaves acknowledge they are online so
-                        # we are sure that DBSIZE and DEBUG DIGEST will not
-                        # fail because of timing issues.
-                        wait_for_condition 500 100 {
-                            [lindex [[lindex $slaves 0] role] 3] eq {connected} &&
-                            [lindex [[lindex $slaves 1] role] 3] eq {connected} &&
-                            [lindex [[lindex $slaves 2] role] 3] eq {connected}
-                        } else {
-                            fail "Slaves still not connected after some time"
+                            # Wait that slaves acknowledge they are online so
+                            # we are sure that DBSIZE and DEBUG DIGEST will not
+                            # fail because of timing issues.
+                            wait_for_condition 500 100 {
+                                [lindex [[lindex $slaves 0] role] 3] eq {connected} &&
+                                [lindex [[lindex $slaves 1] role] 3] eq {connected} &&
+                                [lindex [[lindex $slaves 2] role] 3] eq {connected}
+                            } else {
+                                fail "Slaves still not connected after some time"
+                            }
+
+                            # Stop the write load
+                            stop_bg_complex_data $load_handle0
+                            stop_bg_complex_data $load_handle1
+                            stop_bg_complex_data $load_handle2
+                            stop_write_load $load_handle3
+                            stop_write_load $load_handle4
+
+                            # Make sure that slaves and master have same
+                            # number of keys
+                            wait_for_condition 500 100 {
+                                [$master dbsize] == [[lindex $slaves 0] dbsize] &&
+                                [$master dbsize] == [[lindex $slaves 1] dbsize] &&
+                                [$master dbsize] == [[lindex $slaves 2] dbsize]
+                            } else {
+                                fail "Different number of keys between masted and slave after too long time."
+                            }
+
+                            # Check digests
+                            set digest [$master debug digest]
+                            set digest0 [[lindex $slaves 0] debug digest]
+                            set digest1 [[lindex $slaves 1] debug digest]
+                            set digest2 [[lindex $slaves 2] debug digest]
+                            assert {$digest ne 0000000000000000000000000000000000000000}
+                            assert {$digest eq $digest0}
+                            assert {$digest eq $digest1}
+                            assert {$digest eq $digest2}
                         }
-
-                        # Stop the write load
-                        stop_write_load $load_handle0
-                        stop_write_load $load_handle1
-                        stop_write_load $load_handle2
-                        stop_write_load $load_handle3
-                        stop_write_load $load_handle4
-
-                        # Make sure that slaves and master have same
-                        # number of keys
-                        wait_for_condition 500 100 {
-                            [$master dbsize] == [[lindex $slaves 0] dbsize] &&
-                            [$master dbsize] == [[lindex $slaves 1] dbsize] &&
-                            [$master dbsize] == [[lindex $slaves 2] dbsize]
-                        } else {
-                            fail "Different number of keys between masted and slave after too long time."
-                        }
-
-                        # Check digests
-                        set digest [$master debug digest]
-                        set digest0 [[lindex $slaves 0] debug digest]
-                        set digest1 [[lindex $slaves 1] debug digest]
-                        set digest2 [[lindex $slaves 2] debug digest]
-                        assert {$digest ne 0000000000000000000000000000000000000000}
-                        assert {$digest eq $digest0}
-                        assert {$digest eq $digest1}
-                        assert {$digest eq $digest2}
-                    }
-               }
+                   }
+                }
             }
         }
     }

tests/sentinel/run.tcl

@@ -13,6 +13,7 @@ proc main {} {
     spawn_instance redis $::redis_base_port $::instances_count
     run_tests
     cleanup
+    end_tests
 }
 
 if {[catch main e]} {

tests/sentinel/tests/06-ckquorum.tcl

@@ -0,0 +1,34 @@
+# Test for the SENTINEL CKQUORUM command
+
+source "../tests/includes/init-tests.tcl"
+set num_sentinels [llength $::sentinel_instances]
+
+test "CKQUORUM reports OK and the right amount of Sentinels" {
+    foreach_sentinel_id id {
+        assert_match "*OK $num_sentinels usable*" [S $id SENTINEL CKQUORUM mymaster]
+    }
+}
+
+test "CKQUORUM detects quorum cannot be reached" {
+    set orig_quorum [expr {$num_sentinels/2+1}]
+    S 0 SENTINEL SET mymaster quorum [expr {$num_sentinels+1}]
+    catch {[S 0 SENTINEL CKQUORUM mymaster]} err
+    assert_match "*NOQUORUM*" $err
+    S 0 SENTINEL SET mymaster quorum $orig_quorum
+}
+
+test "CKQUORUM detects failover authorization cannot be reached" {
+    set orig_quorum [expr {$num_sentinels/2+1}]
+    S 0 SENTINEL SET mymaster quorum 1
+    kill_instance sentinel 1
+    kill_instance sentinel 2
+    kill_instance sentinel 3
+    after 5000
+    catch {[S 0 SENTINEL CKQUORUM mymaster]} err
+    assert_match "*NOQUORUM*" $err
+    S 0 SENTINEL SET mymaster quorum $orig_quorum
+    restart_instance sentinel 1
+    restart_instance sentinel 2
+    restart_instance sentinel 3
+}
+

tests/support/server.tcl

@@ -54,10 +54,15 @@ proc kill_server config {
 
     # kill server and wait for the process to be totally exited
     catch {exec kill $pid}
+    if {$::valgrind} {
+        set max_wait 60000
+    } else {
+        set max_wait 10000
+    }
     while {[is_alive $config]} {
         incr wait 10
 
-        if {$wait >= 5000} {
+        if {$wait >= $max_wait} {
             puts "Forcing process $pid to exit..."
             catch {exec kill -KILL $pid}
         } elseif {$wait % 1000 == 0} {

tests/support/util.tcl

@@ -371,3 +371,15 @@ proc start_write_load {host port seconds} {
 proc stop_write_load {handle} {
     catch {exec /bin/kill -9 $handle}
 }
+
+# Execute a background process writing complex data for the specified number
+# of ops to the specified Redis instance.
+proc start_bg_complex_data {host port db ops} {
+    set tclsh [info nameofexecutable]
+    exec $tclsh tests/helpers/bg_complex_data.tcl $host $port $db $ops &
+}
+
+# Stop a process generating write load executed with start_bg_complex_data.
+proc stop_bg_complex_data {handle} {
+    catch {exec /bin/kill -9 $handle}
+}

tests/test_helper.tcl

@@ -16,8 +16,10 @@ set ::all_tests {
     unit/dump
     unit/auth
     unit/protocol
-    unit/basic
+    unit/keyspace
     unit/scan
+    unit/type/string
+    unit/type/incr
     unit/type/list
     unit/type/list-2
     unit/type/list-3
@@ -38,6 +40,7 @@ set ::all_tests {
     integration/aof
     integration/rdb
     integration/convert-zipmap-hash-on-load
+    integration/logging
     unit/pubsub
     unit/slowlog
     unit/scripting

tests/unit/geo.tcl

@@ -0,0 +1,158 @@
+# Helper functins to simulate search-in-radius in the Tcl side in order to
+# verify the Redis implementation with a fuzzy test.
+proc geo_degrad deg {expr {$deg*atan(1)*8/360}}
+
+proc geo_distance {lon1d lat1d lon2d lat2d} {
+    set lon1r [geo_degrad $lon1d]
+    set lat1r [geo_degrad $lat1d]
+    set lon2r [geo_degrad $lon2d]
+    set lat2r [geo_degrad $lat2d]
+    set v [expr {sin(($lon2r - $lon1r) / 2)}]
+    set u [expr {sin(($lat2r - $lat1r) / 2)}]
+    expr {2.0 * 6372797.560856 * \
+            asin(sqrt($u * $u + cos($lat1r) * cos($lat2r) * $v * $v))}
+}
+
+proc geo_random_point {lonvar latvar} {
+    upvar 1 $lonvar lon
+    upvar 1 $latvar lat
+    # Note that the actual latitude limit should be -85 to +85, we restrict
+    # the test to -70 to +70 since in this range the algorithm is more precise
+    # while outside this range occasionally some element may be missing.
+    set lon [expr {-180 + rand()*360}]
+    set lat [expr {-70 + rand()*140}]
+}
+
+start_server {tags {"geo"}} {
+    test {GEOADD create} {
+        r geoadd nyc -73.9454966 40.747533 "lic market"
+    } {1}
+
+    test {GEOADD update} {
+        r geoadd nyc -73.9454966 40.747533 "lic market"
+    } {0}
+
+    test {GEOADD invalid coordinates} {
+        catch {
+            r geoadd nyc -73.9454966 40.747533 "lic market" \
+                foo bar "luck market"
+        } err
+        set err
+    } {*valid*}
+
+    test {GEOADD multi add} {
+        r geoadd nyc -73.9733487 40.7648057 "central park n/q/r" -73.9903085 40.7362513 "union square" -74.0131604 40.7126674 "wtc one" -73.7858139 40.6428986 "jfk" -73.9375699 40.7498929 "q4" -73.9564142 40.7480973 4545
+    } {6}
+
+    test {Check geoset values} {
+        r zrange nyc 0 -1 withscores
+    } {{wtc one} 1791873972053020 {union square} 1791875485187452 {central park n/q/r} 1791875761332224 4545 1791875796750882 {lic market} 1791875804419201 q4 1791875830079666 jfk 1791895905559723}
+
+    test {GEORADIUS simple (sorted)} {
+        r georadius nyc -73.9798091 40.7598464 3 km asc
+    } {{central park n/q/r} 4545 {union square}}
+
+    test {GEORADIUS withdist (sorted)} {
+        r georadius nyc -73.9798091 40.7598464 3 km withdist asc
+    } {{{central park n/q/r} 0.7750} {4545 2.3651} {{union square} 2.7697}}
+
+    test {GEORADIUS with COUNT} {
+        r georadius nyc -73.9798091 40.7598464 10 km COUNT 3
+    } {{central park n/q/r} 4545 {union square}}
+
+    test {GEORADIUS with COUNT DESC} {
+        r georadius nyc -73.9798091 40.7598464 10 km COUNT 2 DESC
+    } {{wtc one} q4}
+
+    test {GEORADIUSBYMEMBER simple (sorted)} {
+        r georadiusbymember nyc "wtc one" 7 km
+    } {{wtc one} {union square} {central park n/q/r} 4545 {lic market}}
+
+    test {GEORADIUSBYMEMBER withdist (sorted)} {
+        r georadiusbymember nyc "wtc one" 7 km withdist
+    } {{{wtc one} 0.0000} {{union square} 3.2544} {{central park n/q/r} 6.7000} {4545 6.1975} {{lic market} 6.8969}}
+
+    test {GEOHASH is able to return geohash strings} {
+        # Example from Wikipedia.
+        r del points
+        r geoadd points -5.6 42.6 test
+        lindex [r geohash points test] 0
+    } {ezs42e44yx0}
+
+    test {GEOPOS simple} {
+        r del points
+        r geoadd points 10 20 a 30 40 b
+        lassign [lindex [r geopos points a b] 0] x1 y1
+        lassign [lindex [r geopos points a b] 1] x2 y2
+        assert {abs($x1 - 10) < 0.001}
+        assert {abs($y1 - 20) < 0.001}
+        assert {abs($x2 - 30) < 0.001}
+        assert {abs($y2 - 40) < 0.001}
+    }
+
+    test {GEOPOS missing element} {
+        r del points
+        r geoadd points 10 20 a 30 40 b
+        lindex [r geopos points a x b] 1
+    } {}
+
+    test {GEODIST simple & unit} {
+        r del points
+        r geoadd points 13.361389 38.115556 "Palermo" \
+                        15.087269 37.502669 "Catania"
+        set m [r geodist points Palermo Catania]
+        assert {$m > 166274 && $m < 166275}
+        set km [r geodist points Palermo Catania km]
+        assert {$km > 166.2 && $km < 166.3}
+    }
+
+    test {GEODIST missing elements} {
+        r del points
+        r geoadd points 13.361389 38.115556 "Palermo" \
+                        15.087269 37.502669 "Catania"
+        set m [r geodist points Palermo Agrigento]
+        assert {$m eq {}}
+        set m [r geodist points Ragusa Agrigento]
+        assert {$m eq {}}
+        set m [r geodist empty_key Palermo Catania]
+        assert {$m eq {}}
+    }
+
+    test {GEOADD + GEORANGE randomized test} {
+        set attempt 10
+        while {[incr attempt -1]} {
+            unset -nocomplain debuginfo
+            set srand_seed [randomInt 1000000]
+            lappend debuginfo "srand_seed is $srand_seed"
+            expr {srand($srand_seed)} ; # If you need a reproducible run
+            r del mypoints
+            set radius_km [expr {[randomInt 200]+10}]
+            set radius_m [expr {$radius_km*1000}]
+            geo_random_point search_lon search_lat
+            lappend debuginfo "Search area: $search_lon,$search_lat $radius_km km"
+            set tcl_result {}
+            set argv {}
+            for {set j 0} {$j < 20000} {incr j} {
+                geo_random_point lon lat
+                lappend argv $lon $lat "place:$j"
+                if {[geo_distance $lon $lat $search_lon $search_lat] < $radius_m} {
+                    lappend tcl_result "place:$j"
+                    lappend debuginfo "place:$j $lon $lat [expr {[geo_distance $lon $lat $search_lon $search_lat]/1000}] km"
+                }
+            }
+            r geoadd mypoints {*}$argv
+            set res [lsort [r georadius mypoints $search_lon $search_lat $radius_km km]]
+            set res2 [lsort $tcl_result]
+            set test_result OK
+            if {$res != $res2} {
+                puts "Redis: $res"
+                puts "Tcl  : $res2"
+                puts [join $debuginfo "\n"]
+                set test_result FAIL
+            }
+            unset -nocomplain debuginfo
+            if {$test_result ne {OK}} break
+        }
+        set test_result
+    } {OK}
+}

tests/unit/keyspace.tcl

@@ -0,0 +1,249 @@
+start_server {tags {"keyspace"}} {
+    test {DEL against a single item} {
+        r set x foo
+        assert {[r get x] eq "foo"}
+        r del x
+        r get x
+    } {}
+
+    test {Vararg DEL} {
+        r set foo1 a
+        r set foo2 b
+        r set foo3 c
+        list [r del foo1 foo2 foo3 foo4] [r mget foo1 foo2 foo3]
+    } {3 {{} {} {}}}
+
+    test {KEYS with pattern} {
+        foreach key {key_x key_y key_z foo_a foo_b foo_c} {
+            r set $key hello
+        }
+        lsort [r keys foo*]
+    } {foo_a foo_b foo_c}
+
+    test {KEYS to get all keys} {
+        lsort [r keys *]
+    } {foo_a foo_b foo_c key_x key_y key_z}
+
+    test {DBSIZE} {
+        r dbsize
+    } {6}
+
+    test {DEL all keys} {
+        foreach key [r keys *] {r del $key}
+        r dbsize
+    } {0}
+
+    test "DEL against expired key" {
+        r debug set-active-expire 0
+        r setex keyExpire 1 valExpire
+        after 1100
+        assert_equal 0 [r del keyExpire]
+        r debug set-active-expire 1
+    }
+
+    test {EXISTS} {
+        set res {}
+        r set newkey test
+        append res [r exists newkey]
+        r del newkey
+        append res [r exists newkey]
+    } {10}
+
+    test {Zero length value in key. SET/GET/EXISTS} {
+        r set emptykey {}
+        set res [r get emptykey]
+        append res [r exists emptykey]
+        r del emptykey
+        append res [r exists emptykey]
+    } {10}
+
+    test {Commands pipelining} {
+        set fd [r channel]
+        puts -nonewline $fd "SET k1 xyzk\r\nGET k1\r\nPING\r\n"
+        flush $fd
+        set res {}
+        append res [string match OK* [r read]]
+        append res [r read]
+        append res [string match PONG* [r read]]
+        format $res
+    } {1xyzk1}
+
+    test {Non existing command} {
+        catch {r foobaredcommand} err
+        string match ERR* $err
+    } {1}
+
+    test {RENAME basic usage} {
+        r set mykey hello
+        r rename mykey mykey1
+        r rename mykey1 mykey2
+        r get mykey2
+    } {hello}
+
+    test {RENAME source key should no longer exist} {
+        r exists mykey
+    } {0}
+
+    test {RENAME against already existing key} {
+        r set mykey a
+        r set mykey2 b
+        r rename mykey2 mykey
+        set res [r get mykey]
+        append res [r exists mykey2]
+    } {b0}
+
+    test {RENAMENX basic usage} {
+        r del mykey
+        r del mykey2
+        r set mykey foobar
+        r renamenx mykey mykey2
+        set res [r get mykey2]
+        append res [r exists mykey]
+    } {foobar0}
+
+    test {RENAMENX against already existing key} {
+        r set mykey foo
+        r set mykey2 bar
+        r renamenx mykey mykey2
+    } {0}
+
+    test {RENAMENX against already existing key (2)} {
+        set res [r get mykey]
+        append res [r get mykey2]
+    } {foobar}
+
+    test {RENAME against non existing source key} {
+        catch {r rename nokey foobar} err
+        format $err
+    } {ERR*}
+
+    test {RENAME where source and dest key are the same (existing)} {
+        r set mykey foo
+        r rename mykey mykey
+    } {OK}
+
+    test {RENAMENX where source and dest key are the same (existing)} {
+        r set mykey foo
+        r renamenx mykey mykey
+    } {0}
+
+    test {RENAME where source and dest key are the same (non existing)} {
+        r del mykey
+        catch {r rename mykey mykey} err
+        format $err
+    } {ERR*}
+
+    test {RENAME with volatile key, should move the TTL as well} {
+        r del mykey mykey2
+        r set mykey foo
+        r expire mykey 100
+        assert {[r ttl mykey] > 95 && [r ttl mykey] <= 100}
+        r rename mykey mykey2
+        assert {[r ttl mykey2] > 95 && [r ttl mykey2] <= 100}
+    }
+
+    test {RENAME with volatile key, should not inherit TTL of target key} {
+        r del mykey mykey2
+        r set mykey foo
+        r set mykey2 bar
+        r expire mykey2 100
+        assert {[r ttl mykey] == -1 && [r ttl mykey2] > 0}
+        r rename mykey mykey2
+        r ttl mykey2
+    } {-1}
+
+    test {DEL all keys again (DB 0)} {
+        foreach key [r keys *] {
+            r del $key
+        }
+        r dbsize
+    } {0}
+
+    test {DEL all keys again (DB 1)} {
+        r select 10
+        foreach key [r keys *] {
+            r del $key
+        }
+        set res [r dbsize]
+        r select 9
+        format $res
+    } {0}
+
+    test {MOVE basic usage} {
+        r set mykey foobar
+        r move mykey 10
+        set res {}
+        lappend res [r exists mykey]
+        lappend res [r dbsize]
+        r select 10
+        lappend res [r get mykey]
+        lappend res [r dbsize]
+        r select 9
+        format $res
+    } [list 0 0 foobar 1]
+
+    test {MOVE against key existing in the target DB} {
+        r set mykey hello
+        r move mykey 10
+    } {0}
+
+    test {MOVE against non-integer DB (#1428)} {
+        r set mykey hello
+        catch {r move mykey notanumber} e
+        set e
+    } {*ERR*index out of range}
+
+    test {SET/GET keys in different DBs} {
+        r set a hello
+        r set b world
+        r select 10
+        r set a foo
+        r set b bared
+        r select 9
+        set res {}
+        lappend res [r get a]
+        lappend res [r get b]
+        r select 10
+        lappend res [r get a]
+        lappend res [r get b]
+        r select 9
+        format $res
+    } {hello world foo bared}
+
+    test {RANDOMKEY} {
+        r flushdb
+        r set foo x
+        r set bar y
+        set foo_seen 0
+        set bar_seen 0
+        for {set i 0} {$i < 100} {incr i} {
+            set rkey [r randomkey]
+            if {$rkey eq {foo}} {
+                set foo_seen 1
+            }
+            if {$rkey eq {bar}} {
+                set bar_seen 1
+            }
+        }
+        list $foo_seen $bar_seen
+    } {1 1}
+
+    test {RANDOMKEY against empty DB} {
+        r flushdb
+        r randomkey
+    } {}
+
+    test {RANDOMKEY regression 1} {
+        r flushdb
+        r set x 10
+        r del x
+        r randomkey
+    } {}
+
+    test {KEYS * two times with long key, Github issue #1208} {
+        r flushdb
+        r set dlskeriewrioeuwqoirueioqwrueoqwrueqw test
+        r keys *
+        r keys *
+    } {dlskeriewrioeuwqoirueioqwrueoqwrueqw}
+}

tests/unit/memefficiency.tcl

@@ -1,15 +1,20 @@
 proc test_memory_efficiency {range} {
     r flushall
+    set rd [redis_deferring_client]
     set base_mem [s used_memory]
     set written 0
     for {set j 0} {$j < 10000} {incr j} {
         set key key:$j
         set val [string repeat A [expr {int(rand()*$range)}]]
-        r set $key $val
+        $rd set $key $val
         incr written [string length $key]
         incr written [string length $val]
         incr written 2 ;# A separator is the minimum to store key-value data.
     }
+    for {set j 0} {$j < 10000} {incr j} {
+        $rd read ; # Discard replies
+    }
+
     set current_mem [s used_memory]
     set used [expr {$current_mem-$base_mem}]
     set efficiency [expr {double($written)/$used}]

tests/unit/type/hash.tcl

@@ -390,6 +390,51 @@ start_server {tags {"hash"}} {
         lappend rv [string match "ERR*not*float*" $bigerr]
     } {1 1}
 
+    test {HSTRLEN against the small hash} {
+        set err {}
+        foreach k [array names smallhash *] {
+            if {[string length $smallhash($k)] ne [r hstrlen smallhash $k]} {
+                set err "[string length $smallhash($k)] != [r hstrlen smallhash $k]"
+                break
+            }
+        }
+        set _ $err
+    } {}
+
+    test {HSTRLEN against the big hash} {
+        set err {}
+        foreach k [array names bighash *] {
+            if {[string length $bighash($k)] ne [r hstrlen bighash $k]} {
+                set err "[string length $bighash($k)] != [r hstrlen bighash $k]"
+                break
+            }
+        }
+        set _ $err
+    } {}
+
+    test {HSTRLEN against non existing field} {
+        set rv {}
+        lappend rv [r hstrlen smallhash __123123123__]
+        lappend rv [r hstrlen bighash __123123123__]
+        set _ $rv
+    } {0 0}
+
+    test {HSTRLEN corner cases} {
+        set vals {
+            -9223372036854775808 9223372036854775807 9223372036854775808
+            {} 0 -1 x
+        }
+        foreach v $vals {
+            r hmset smallhash field $v
+            r hmset bighash field $v
+            set len1 [string length $v]
+            set len2 [r hstrlen smallhash field]
+            set len3 [r hstrlen bighash field]
+            assert {$len1 == $len2}
+            assert {$len2 == $len3}
+        }
+    }
+
     test {Hash ziplist regression test for large keys} {
         r hset hash kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk a
         r hset hash kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk b

tests/unit/type/incr.tcl

@@ -0,0 +1,147 @@
+start_server {tags {"incr"}} {
+    test {INCR against non existing key} {
+        set res {}
+        append res [r incr novar]
+        append res [r get novar]
+    } {11}
+
+    test {INCR against key created by incr itself} {
+        r incr novar
+    } {2}
+
+    test {INCR against key originally set with SET} {
+        r set novar 100
+        r incr novar
+    } {101}
+
+    test {INCR over 32bit value} {
+        r set novar 17179869184
+        r incr novar
+    } {17179869185}
+
+    test {INCRBY over 32bit value with over 32bit increment} {
+        r set novar 17179869184
+        r incrby novar 17179869184
+    } {34359738368}
+
+    test {INCR fails against key with spaces (left)} {
+        r set novar "    11"
+        catch {r incr novar} err
+        format $err
+    } {ERR*}
+
+    test {INCR fails against key with spaces (right)} {
+        r set novar "11    "
+        catch {r incr novar} err
+        format $err
+    } {ERR*}
+
+    test {INCR fails against key with spaces (both)} {
+        r set novar "    11    "
+        catch {r incr novar} err
+        format $err
+    } {ERR*}
+
+    test {INCR fails against a key holding a list} {
+        r rpush mylist 1
+        catch {r incr mylist} err
+        r rpop mylist
+        format $err
+    } {WRONGTYPE*}
+
+    test {DECRBY over 32bit value with over 32bit increment, negative res} {
+        r set novar 17179869184
+        r decrby novar 17179869185
+    } {-1}
+
+    test {INCR uses shared objects in the 0-9999 range} {
+        r set foo -1
+        r incr foo
+        assert {[r object refcount foo] > 1}
+        r set foo 9998
+        r incr foo
+        assert {[r object refcount foo] > 1}
+        r incr foo
+        assert {[r object refcount foo] == 1}
+    }
+
+    test {INCR can modify objects in-place} {
+        r set foo 20000
+        r incr foo
+        assert {[r object refcount foo] == 1}
+        set old [lindex [split [r debug object foo]] 1]
+        r incr foo
+        set new [lindex [split [r debug object foo]] 1]
+        assert {[string range $old 0 2] eq "at:"}
+        assert {[string range $new 0 2] eq "at:"}
+        assert {$old eq $new}
+    }
+
+    test {INCRBYFLOAT against non existing key} {
+        r del novar
+        list    [roundFloat [r incrbyfloat novar 1]] \
+                [roundFloat [r get novar]] \
+                [roundFloat [r incrbyfloat novar 0.25]] \
+                [roundFloat [r get novar]]
+    } {1 1 1.25 1.25}
+
+    test {INCRBYFLOAT against key originally set with SET} {
+        r set novar 1.5
+        roundFloat [r incrbyfloat novar 1.5]
+    } {3}
+
+    test {INCRBYFLOAT over 32bit value} {
+        r set novar 17179869184
+        r incrbyfloat novar 1.5
+    } {17179869185.5}
+
+    test {INCRBYFLOAT over 32bit value with over 32bit increment} {
+        r set novar 17179869184
+        r incrbyfloat novar 17179869184
+    } {34359738368}
+
+    test {INCRBYFLOAT fails against key with spaces (left)} {
+        set err {}
+        r set novar "    11"
+        catch {r incrbyfloat novar 1.0} err
+        format $err
+    } {ERR*valid*}
+
+    test {INCRBYFLOAT fails against key with spaces (right)} {
+        set err {}
+        r set novar "11    "
+        catch {r incrbyfloat novar 1.0} err
+        format $err
+    } {ERR*valid*}
+
+    test {INCRBYFLOAT fails against key with spaces (both)} {
+        set err {}
+        r set novar " 11 "
+        catch {r incrbyfloat novar 1.0} err
+        format $err
+    } {ERR*valid*}
+
+    test {INCRBYFLOAT fails against a key holding a list} {
+        r del mylist
+        set err {}
+        r rpush mylist 1
+        catch {r incrbyfloat mylist 1.0} err
+        r del mylist
+        format $err
+    } {WRONGTYPE*}
+
+    test {INCRBYFLOAT does not allow NaN or Infinity} {
+        r set foo 0
+        set err {}
+        catch {r incrbyfloat foo +inf} err
+        set err
+        # p.s. no way I can force NaN to test it from the API because
+        # there is no way to increment / decrement by infinity nor to
+        # perform divisions.
+    } {ERR*would produce*}
+
+    test {INCRBYFLOAT decrement} {
+        r set foo 1
+        roundFloat [r incrbyfloat foo -1.1]
+    } {-0.1}
+}

tests/unit/type/set.tcl

@@ -346,6 +346,33 @@ start_server {
         r spop nonexisting_key 100
     } {}
 
+    test "SPOP new implementation: code path #1" {
+        set content {1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20}
+        create_set myset $content
+        set res [r spop myset 30]
+        assert {[lsort $content] eq [lsort $res]}
+    }
+
+    test "SPOP new implementation: code path #2" {
+        set content {1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20}
+        create_set myset $content
+        set res [r spop myset 2]
+        assert {[llength $res] == 2}
+        assert {[r scard myset] == 18}
+        set union [concat [r smembers myset] $res]
+        assert {[lsort $union] eq [lsort $content]}
+    }
+
+    test "SPOP new implementation: code path #3" {
+        set content {1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20}
+        create_set myset $content
+        set res [r spop myset 18]
+        assert {[llength $res] == 18}
+        assert {[r scard myset] == 2}
+        set union [concat [r smembers myset] $res]
+        assert {[lsort $union] eq [lsort $content]}
+    }
+
     test "SRANDMEMBER with <count> against non existing key" {
         r srandmember nonexisting_key 100
     } {}
@@ -492,6 +519,7 @@ start_server {
     test "SMOVE non existing key" {
         setup_move
         assert_equal 0 [r smove myset1 myset2 foo]
+        assert_equal 0 [r smove myset1 myset1 foo]
         assert_equal {1 a b} [lsort [r smembers myset1]]
         assert_equal {2 3 4} [lsort [r smembers myset2]]
     }

tests/unit/type/string.tcl

@@ -1,9 +1,4 @@
-start_server {tags {"basic"}} {
-    test {DEL all keys to start with a clean DB} {
-        foreach key [r keys *] {r del $key}
-        r dbsize
-    } {0}
-
+start_server {tags {"string"}} {
     test {SET and GET an item} {
         r set x foobar
         r get x
@@ -14,38 +9,6 @@ start_server {tags {"basic"}} {
         r get x
     } {}
 
-    test {DEL against a single item} {
-        r del x
-        r get x
-    } {}
-
-    test {Vararg DEL} {
-        r set foo1 a
-        r set foo2 b
-        r set foo3 c
-        list [r del foo1 foo2 foo3 foo4] [r mget foo1 foo2 foo3]
-    } {3 {{} {} {}}}
-
-    test {KEYS with pattern} {
-        foreach key {key_x key_y key_z foo_a foo_b foo_c} {
-            r set $key hello
-        }
-        lsort [r keys foo*]
-    } {foo_a foo_b foo_c}
-
-    test {KEYS to get all keys} {
-        lsort [r keys *]
-    } {foo_a foo_b foo_c key_x key_y key_z}
-
-    test {DBSIZE} {
-        r dbsize
-    } {6}
-
-    test {DEL all keys} {
-        foreach key [r keys *] {r del $key}
-        r dbsize
-    } {0}
-
     test {Very big payload in GET/SET} {
         set buf [string repeat "abcd" 1000000]
         r set foo $buf
@@ -75,6 +38,7 @@ start_server {tags {"basic"}} {
         } {}
 
         test {SET 10000 numeric keys and access all them in reverse order} {
+            r flushdb
             set err {}
             for {set x 0} {$x < 10000} {incr x} {
                 r set $x $x
@@ -90,157 +54,11 @@ start_server {tags {"basic"}} {
             set _ $err
         } {}
 
-        test {DBSIZE should be 10101 now} {
+        test {DBSIZE should be 10000 now} {
             r dbsize
-        } {10101}
+        } {10000}
     }
 
-    test {INCR against non existing key} {
-        set res {}
-        append res [r incr novar]
-        append res [r get novar]
-    } {11}
-
-    test {INCR against key created by incr itself} {
-        r incr novar
-    } {2}
-
-    test {INCR against key originally set with SET} {
-        r set novar 100
-        r incr novar
-    } {101}
-
-    test {INCR over 32bit value} {
-        r set novar 17179869184
-        r incr novar
-    } {17179869185}
-
-    test {INCRBY over 32bit value with over 32bit increment} {
-        r set novar 17179869184
-        r incrby novar 17179869184
-    } {34359738368}
-
-    test {INCR fails against key with spaces (left)} {
-        r set novar "    11"
-        catch {r incr novar} err
-        format $err
-    } {ERR*}
-
-    test {INCR fails against key with spaces (right)} {
-        r set novar "11    "
-        catch {r incr novar} err
-        format $err
-    } {ERR*}
-
-    test {INCR fails against key with spaces (both)} {
-        r set novar "    11    "
-        catch {r incr novar} err
-        format $err
-    } {ERR*}
-
-    test {INCR fails against a key holding a list} {
-        r rpush mylist 1
-        catch {r incr mylist} err
-        r rpop mylist
-        format $err
-    } {WRONGTYPE*}
-
-    test {DECRBY over 32bit value with over 32bit increment, negative res} {
-        r set novar 17179869184
-        r decrby novar 17179869185
-    } {-1}
-
-    test {INCR uses shared objects in the 0-9999 range} {
-        r set foo -1
-        r incr foo
-        assert {[r object refcount foo] > 1}
-        r set foo 9998
-        r incr foo
-        assert {[r object refcount foo] > 1}
-        r incr foo
-        assert {[r object refcount foo] == 1}
-    }
-
-    test {INCR can modify objects in-place} {
-        r set foo 20000
-        r incr foo
-        assert {[r object refcount foo] == 1}
-        set old [lindex [split [r debug object foo]] 1]
-        r incr foo
-        set new [lindex [split [r debug object foo]] 1]
-        assert {[string range $old 0 2] eq "at:"}
-        assert {[string range $new 0 2] eq "at:"}
-        assert {$old eq $new}
-    }
-
-    test {INCRBYFLOAT against non existing key} {
-        r del novar
-        list    [roundFloat [r incrbyfloat novar 1]] \
-                [roundFloat [r get novar]] \
-                [roundFloat [r incrbyfloat novar 0.25]] \
-                [roundFloat [r get novar]]
-    } {1 1 1.25 1.25}
-
-    test {INCRBYFLOAT against key originally set with SET} {
-        r set novar 1.5
-        roundFloat [r incrbyfloat novar 1.5]
-    } {3}
-
-    test {INCRBYFLOAT over 32bit value} {
-        r set novar 17179869184
-        r incrbyfloat novar 1.5
-    } {17179869185.5}
-
-    test {INCRBYFLOAT over 32bit value with over 32bit increment} {
-        r set novar 17179869184
-        r incrbyfloat novar 17179869184
-    } {34359738368}
-
-    test {INCRBYFLOAT fails against key with spaces (left)} {
-        set err {}
-        r set novar "    11"
-        catch {r incrbyfloat novar 1.0} err
-        format $err
-    } {ERR*valid*}
-
-    test {INCRBYFLOAT fails against key with spaces (right)} {
-        set err {}
-        r set novar "11    "
-        catch {r incrbyfloat novar 1.0} err
-        format $err
-    } {ERR*valid*}
-
-    test {INCRBYFLOAT fails against key with spaces (both)} {
-        set err {}
-        r set novar " 11 "
-        catch {r incrbyfloat novar 1.0} err
-        format $err
-    } {ERR*valid*}
-
-    test {INCRBYFLOAT fails against a key holding a list} {
-        r del mylist
-        set err {}
-        r rpush mylist 1
-        catch {r incrbyfloat mylist 1.0} err
-        r del mylist
-        format $err
-    } {WRONGTYPE*}
-
-    test {INCRBYFLOAT does not allow NaN or Infinity} {
-        r set foo 0
-        set err {}
-        catch {r incrbyfloat foo +inf} err
-        set err
-        # p.s. no way I can force NaN to test it from the API because
-        # there is no way to increment / decrement by infinity nor to
-        # perform divisions.
-    } {ERR*would produce*}
-
-    test {INCRBYFLOAT decrement} {
-        r set foo 1
-        roundFloat [r incrbyfloat foo -1.1]
-    } {-0.1}
-
     test "SETNX target key missing" {
         r del novar
         assert_equal 1 [r setnx novar foobared]
@@ -284,172 +102,6 @@ start_server {tags {"basic"}} {
         assert_equal 20 [r get x]
     }
 
-    test "DEL against expired key" {
-        r debug set-active-expire 0
-        r setex keyExpire 1 valExpire
-        after 1100
-        assert_equal 0 [r del keyExpire]
-        r debug set-active-expire 1
-    }
-
-    test {EXISTS} {
-        set res {}
-        r set newkey test
-        append res [r exists newkey]
-        r del newkey
-        append res [r exists newkey]
-    } {10}
-
-    test {Zero length value in key. SET/GET/EXISTS} {
-        r set emptykey {}
-        set res [r get emptykey]
-        append res [r exists emptykey]
-        r del emptykey
-        append res [r exists emptykey]
-    } {10}
-
-    test {Commands pipelining} {
-        set fd [r channel]
-        puts -nonewline $fd "SET k1 xyzk\r\nGET k1\r\nPING\r\n"
-        flush $fd
-        set res {}
-        append res [string match OK* [r read]]
-        append res [r read]
-        append res [string match PONG* [r read]]
-        format $res
-    } {1xyzk1}
-
-    test {Non existing command} {
-        catch {r foobaredcommand} err
-        string match ERR* $err
-    } {1}
-
-    test {RENAME basic usage} {
-        r set mykey hello
-        r rename mykey mykey1
-        r rename mykey1 mykey2
-        r get mykey2
-    } {hello}
-
-    test {RENAME source key should no longer exist} {
-        r exists mykey
-    } {0}
-
-    test {RENAME against already existing key} {
-        r set mykey a
-        r set mykey2 b
-        r rename mykey2 mykey
-        set res [r get mykey]
-        append res [r exists mykey2]
-    } {b0}
-
-    test {RENAMENX basic usage} {
-        r del mykey
-        r del mykey2
-        r set mykey foobar
-        r renamenx mykey mykey2
-        set res [r get mykey2]
-        append res [r exists mykey]
-    } {foobar0}
-
-    test {RENAMENX against already existing key} {
-        r set mykey foo
-        r set mykey2 bar
-        r renamenx mykey mykey2
-    } {0}
-
-    test {RENAMENX against already existing key (2)} {
-        set res [r get mykey]
-        append res [r get mykey2]
-    } {foobar}
-
-    test {RENAME against non existing source key} {
-        catch {r rename nokey foobar} err
-        format $err
-    } {ERR*}
-
-    test {RENAME where source and dest key is the same} {
-        catch {r rename mykey mykey} err
-        format $err
-    } {ERR*}
-
-    test {RENAME with volatile key, should move the TTL as well} {
-        r del mykey mykey2
-        r set mykey foo
-        r expire mykey 100
-        assert {[r ttl mykey] > 95 && [r ttl mykey] <= 100}
-        r rename mykey mykey2
-        assert {[r ttl mykey2] > 95 && [r ttl mykey2] <= 100}
-    }
-
-    test {RENAME with volatile key, should not inherit TTL of target key} {
-        r del mykey mykey2
-        r set mykey foo
-        r set mykey2 bar
-        r expire mykey2 100
-        assert {[r ttl mykey] == -1 && [r ttl mykey2] > 0}
-        r rename mykey mykey2
-        r ttl mykey2
-    } {-1}
-
-    test {DEL all keys again (DB 0)} {
-        foreach key [r keys *] {
-            r del $key
-        }
-        r dbsize
-    } {0}
-
-    test {DEL all keys again (DB 1)} {
-        r select 10
-        foreach key [r keys *] {
-            r del $key
-        }
-        set res [r dbsize]
-        r select 9
-        format $res
-    } {0}
-
-    test {MOVE basic usage} {
-        r set mykey foobar
-        r move mykey 10
-        set res {}
-        lappend res [r exists mykey]
-        lappend res [r dbsize]
-        r select 10
-        lappend res [r get mykey]
-        lappend res [r dbsize]
-        r select 9
-        format $res
-    } [list 0 0 foobar 1]
-
-    test {MOVE against key existing in the target DB} {
-        r set mykey hello
-        r move mykey 10
-    } {0}
-
-    test {MOVE against non-integer DB (#1428)} {
-        r set mykey hello
-        catch {r move mykey notanumber} e
-        set e
-    } {*ERR*index out of range}
-
-    test {SET/GET keys in different DBs} {
-        r set a hello
-        r set b world
-        r select 10
-        r set a foo
-        r set b bared
-        r select 9
-        set res {}
-        lappend res [r get a]
-        lappend res [r get b]
-        r select 10
-        lappend res [r get a]
-        lappend res [r get b]
-        r select 9
-        format $res
-    } {hello world foo bared}
-
     test {MGET} {
         r flushdb
         r set foo BAR
@@ -467,37 +119,8 @@ start_server {tags {"basic"}} {
         r mget foo baazz bar myset
     } {BAR {} FOO {}}
 
-    test {RANDOMKEY} {
-        r flushdb
-        r set foo x
-        r set bar y
-        set foo_seen 0
-        set bar_seen 0
-        for {set i 0} {$i < 100} {incr i} {
-            set rkey [r randomkey]
-            if {$rkey eq {foo}} {
-                set foo_seen 1
-            }
-            if {$rkey eq {bar}} {
-                set bar_seen 1
-            }
-        }
-        list $foo_seen $bar_seen
-    } {1 1}
-
-    test {RANDOMKEY against empty DB} {
-        r flushdb
-        r randomkey
-    } {}
-
-    test {RANDOMKEY regression 1} {
-        r flushdb
-        r set x 10
-        r del x
-        r randomkey
-    } {}
-
     test {GETSET (set new value)} {
+        r del foo
         list [r getset foo xyz] [r get foo]
     } {{} xyz}
 
@@ -792,13 +415,6 @@ start_server {tags {"basic"}} {
         assert {$ttl <= 10 && $ttl > 5}
     }
 
-    test {KEYS * two times with long key, Github issue #1208} {
-        r flushdb
-        r set dlskeriewrioeuwqoirueioqwrueoqwrueqw test
-        r keys *
-        r keys *
-    } {dlskeriewrioeuwqoirueioqwrueoqwrueqw}
-
     test {GETRANGE with huge ranges, Github issue #1844} {
         r set foo bar
         r getrange foo 0 4294967297

tests/unit/type/zset.tcl

@@ -43,6 +43,84 @@ start_server {tags {"zset"}} {
             assert_error "*not*float*" {r zadd myzset nan abc}
         }
 
+        test "ZADD with options syntax error with incomplete pair" {
+            r del ztmp
+            catch {r zadd ztmp xx 10 x 20} err
+            set err
+        } {ERR*}
+
+        test "ZADD XX option without key - $encoding" {
+            r del ztmp
+            assert {[r zadd ztmp xx 10 x] == 0}
+            assert {[r type ztmp] eq {none}}
+        }
+
+        test "ZADD XX existing key - $encoding" {
+            r del ztmp
+            r zadd ztmp 10 x
+            assert {[r zadd ztmp xx 20 y] == 0}
+            assert {[r zcard ztmp] == 1}
+        }
+
+        test "ZADD XX returns the number of elements actually added" {
+            r del ztmp
+            r zadd ztmp 10 x
+            set retval [r zadd ztmp 10 x 20 y 30 z]
+            assert {$retval == 2}
+        }
+
+        test "ZADD XX updates existing elements score" {
+            r del ztmp
+            r zadd ztmp 10 x 20 y 30 z
+            r zadd ztmp xx 5 foo 11 x 21 y 40 zap
+            assert {[r zcard ztmp] == 3}
+            assert {[r zscore ztmp x] == 11}
+            assert {[r zscore ztmp y] == 21}
+        }
+
+        test "ZADD XX and NX are not compatible" {
+            r del ztmp
+            catch {r zadd ztmp xx nx 10 x} err
+            set err
+        } {ERR*}
+
+        test "ZADD NX with non exisitng key" {
+            r del ztmp
+            r zadd ztmp nx 10 x 20 y 30 z
+            assert {[r zcard ztmp] == 3}
+        }
+
+        test "ZADD NX only add new elements without updating old ones" {
+            r del ztmp
+            r zadd ztmp 10 x 20 y 30 z
+            assert {[r zadd ztmp nx 11 x 21 y 100 a 200 b] == 2}
+            assert {[r zscore ztmp x] == 10}
+            assert {[r zscore ztmp y] == 20}
+            assert {[r zscore ztmp a] == 100}
+            assert {[r zscore ztmp b] == 200}
+        }
+
+        test "ZADD INCR works like ZINCRBY" {
+            r del ztmp
+            r zadd ztmp 10 x 20 y 30 z
+            r zadd ztmp INCR 15 x
+            assert {[r zscore ztmp x] == 25}
+        }
+
+        test "ZADD INCR works with a single score-elemenet pair" {
+            r del ztmp
+            r zadd ztmp 10 x 20 y 30 z
+            catch {r zadd ztmp INCR 15 x 10 y} err
+            set err
+        } {ERR*}
+
+        test "ZADD CH option changes return value to all changed elements" {
+            r del ztmp
+            r zadd ztmp 10 x 20 y 30 z
+            assert {[r zadd ztmp 11 x 21 y 30 z] == 0}
+            assert {[r zadd ztmp ch 12 x 22 y 30 z] == 2}
+        }
+
         test "ZINCRBY calls leading to NaN result in error" {
             r zincrby myzset +inf abc
             assert_error "*NaN*" {r zincrby myzset -inf abc}
@@ -77,6 +155,8 @@ start_server {tags {"zset"}} {
         }
 
         test "ZCARD basics - $encoding" {
+            r del ztmp
+            r zadd ztmp 10 a 20 b 30 c
             assert_equal 3 [r zcard ztmp]
             assert_equal 0 [r zcard zdoesntexist]
         }

utils/create-cluster/.gitignore

@@ -1 +1,5 @@
 config.sh
+*.rdb
+*.aof
+*.conf
+*.log

utils/create-cluster/create-cluster

@@ -43,7 +43,7 @@ then
     while [ $((PORT < ENDPORT)) != "0" ]; do
         PORT=$((PORT+1))
         echo "Stopping $PORT"
-        redis-cli -p $PORT shutdown nosave
+        ../../src/redis-cli -p $PORT shutdown nosave
     done
     exit 0
 fi
@@ -54,7 +54,7 @@ then
     while [ 1 ]; do
         clear
         date
-        redis-cli -p $PORT cluster nodes | head -30
+        ../../src/redis-cli -p $PORT cluster nodes | head -30
         sleep 1
     done
     exit 0

utils/hashtable/README

@@ -0,0 +1,13 @@
+Hash table implementation related utilities.
+
+rehashing.c
+---
+
+Visually show buckets in the two hash tables between rehashings. Also stress
+test getRandomKeys() implementation, that may actually disappear from
+Redis soon, however visualizaiton some code is reusable in new bugs
+investigation.
+
+Compile with:
+
+    cc -I ../../src/ rehashing.c ../../src/zmalloc.c ../../src/dict.c -o rehashing_test

utils/hashtable/rehashing.c

@@ -0,0 +1,142 @@
+#include "redis.h"
+#include "dict.h"
+
+void _redisAssert(char *x, char *y, int l) {
+    printf("ASSERT: %s %s %d\n",x,y,l);
+    exit(1);
+}
+
+unsigned int dictKeyHash(const void *keyp) {
+    unsigned long key = (unsigned long)keyp;
+    key = dictGenHashFunction(&key,sizeof(key));
+    key += ~(key << 15);
+    key ^=  (key >> 10);
+    key +=  (key << 3);
+    key ^=  (key >> 6);
+    key += ~(key << 11);
+    key ^=  (key >> 16);
+    return key;
+}
+
+int dictKeyCompare(void *privdata, const void *key1, const void *key2) {
+    unsigned long k1 = (unsigned long)key1;
+    unsigned long k2 = (unsigned long)key2;
+    return k1 == k2;
+}
+
+dictType dictTypeTest = {
+    dictKeyHash,                   /* hash function */
+    NULL,                          /* key dup */
+    NULL,                          /* val dup */
+    dictKeyCompare,                /* key compare */
+    NULL,                          /* key destructor */
+    NULL                           /* val destructor */
+};
+
+void showBuckets(dictht ht) {
+    if (ht.table == NULL) {
+        printf("NULL\n");
+    } else {
+        int j;
+        for (j = 0; j < ht.size; j++) {
+            printf("%c", ht.table[j] ? '1' : '0');
+        }
+        printf("\n");
+    }
+}
+
+void show(dict *d) {
+    int j;
+    if (d->rehashidx != -1) {
+        printf("rhidx: ");
+        for (j = 0; j < d->rehashidx; j++)
+            printf(".");
+        printf("|\n");
+    }
+    printf("ht[0]: ");
+    showBuckets(d->ht[0]);
+    printf("ht[1]: ");
+    showBuckets(d->ht[1]);
+    printf("\n");
+}
+
+int sortPointers(const void *a, const void *b) {
+    unsigned long la, lb;
+
+    la = (long) (*((dictEntry**)a));
+    lb = (long) (*((dictEntry**)b));
+    return la-lb;
+}
+
+void stressGetKeys(dict *d, int times, int *perfect_run, int *approx_run) {
+    int j;
+
+    dictEntry **des = zmalloc(sizeof(dictEntry*)*dictSize(d));
+    for (j = 0; j < times; j++) {
+        int requested = rand() % (dictSize(d)+1);
+        int returned = dictGetSomeKeys(d, des, requested);
+        int dup = 0;
+
+        qsort(des,returned,sizeof(dictEntry*),sortPointers);
+        if (returned > 1) {
+            int i;
+            for (i = 0; i < returned-1; i++) {
+                if (des[i] == des[i+1]) dup++;
+            }
+        }
+
+        if (requested == returned && dup == 0) {
+            (*perfect_run)++;
+        } else {
+            (*approx_run)++;
+            printf("Requested, returned, duplicated: %d %d %d\n",
+                requested, returned, dup);
+        }
+    }
+    zfree(des);
+}
+
+#define MAX1 120
+#define MAX2 1000
+int main(void) {
+    dict *d = dictCreate(&dictTypeTest,NULL);
+    unsigned long i;
+    srand(time(NULL));
+
+    for (i = 0; i < MAX1; i++) {
+        dictAdd(d,(void*)i,NULL);
+        show(d);
+    }
+    printf("Size: %d\n", (int)dictSize(d));
+
+    for (i = 0; i < MAX1; i++) {
+        dictDelete(d,(void*)i);
+        dictResize(d);
+        show(d);
+    }
+    dictRelease(d);
+
+    d = dictCreate(&dictTypeTest,NULL);
+
+    printf("Stress testing dictGetSomeKeys\n");
+    int perfect_run = 0, approx_run = 0;
+
+    for (i = 0; i < MAX2; i++) {
+        dictAdd(d,(void*)i,NULL);
+        stressGetKeys(d,100,&perfect_run,&approx_run);
+    }
+
+    for (i = 0; i < MAX2; i++) {
+        dictDelete(d,(void*)i);
+        dictResize(d);
+        stressGetKeys(d,100,&perfect_run,&approx_run);
+    }
+
+    printf("dictGetSomeKey, %d perfect runs, %d approximated runs\n",
+        perfect_run, approx_run);
+
+    dictRelease(d);
+
+    printf("TEST PASSED!\n");
+    return 0;
+}